A Note on LoRA

Vlad Fomenko, Han Yu, Jongho Lee, Stanley Hsieh, Weizhu Chen
4/7/2024
cs.LGcs.AIcs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

LoRA (Low-Rank Adaptation) has emerged as a preferred method for efficiently adapting Large Language Models (LLMs) with remarkable simplicity and efficacy. This note extends the original LoRA paper by offering new perspectives that were not initially discussed and presents a series of insights for deploying LoRA at scale. Without introducing new experiments, we aim to improve the understanding and application of LoRA.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

LoRA Is Slower Than You Think

91% match2025

LoRA-Mini : Adaptation Matrices Decomposition and Selective Training

90% match1 repo22024

A Survey on LoRA of Large Language Models

89% match1 repo2792024

LoRA$^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models

89% match2024

LoRASuite: Efficient LoRA Adaptation Across Large Language Model Upgrades

88% match1 repo82025

LoRA Land: 310 Fine-tuned LLMs that Rival GPT-4, A Technical Report

88% match1 repo212024

Cite this paper

@article{fomenko2024note,
  title  = {A Note on LoRA},
  author = {Vlad Fomenko and Han Yu and Jongho Lee and Stanley Hsieh and Weizhu Chen},
  year   = {2024},
  eprint = {2404.05086},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.05086v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.05086.svg)](https://codeofpaper.com/paper/2404.05086)

Discussion