QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models

Yuhui Xu, Lingxi Xie, Xiaotao Gu, Xin Chen, Heng Chang +4 more
2/12/2026

Official code: yuhuixu1993/qa-lora (147 stars, MIT, last commit 2024-03-13). 1 more implementation below.

Abstract

Recently years have witnessed a rapid development of large language models (LLMs). Despite the strong ability in many language-understanding tasks, the heavy computational burden largely restricts the application of LLMs especially when one needs to deploy them onto edge devices. In this paper, we propose a quantization-aware low-rank adaptation (QA-LoRA) algorithm. The motivation lies in the imbalanced degrees of freedom of quantization and adaptation, and the solution is to use group-wise operators which increase the degree of freedom of quantization meanwhile decreasing that of adaptation. QA-LoRA is easily implemented with a few lines of code, and it equips the original LoRA with two-fold abilities: (i) during fine-tuning, the LLM's weights are quantized (e.g., into INT4) to reduce time and memory usage; (ii) after fine-tuning, the LLM and auxiliary weights are naturally integrated into a quantized model without loss of accuracy. We apply QA-LoRA to the LLaMA and LLaMA2 model families and validate its effectiveness in different fine-tuning datasets and downstream scenarios. Code will be made available at https://github.com/yuhuixu1993/qa-lora.

View on arXivView PDF

Code Implementations(2)

yuhuixu1993/qa-loraOfficial100%
MIT

Official PyTorch implementation of QA-LoRA

14712PythonSep 26, 20232 years agoMIT
language-model
eltociear/qa-loraCommunity58%
MIT

Pytorch code for paper QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models

2525PythonSep 27, 20233 years agoMIT

Similar Papers

Accurate and Efficient Fine-Tuning of Quantized Large Language Models Through Optimal Balance

91% match1 repo62024

CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initialization

90% match2025

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning

90% match2024

LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits

89% match2025

L4Q: Parameter Efficient Quantization-Aware Fine-Tuning on Large Language Models

89% match2024

LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuning

88% match1 repo1292023

Cite this paper

@article{xu2026qalora,
  title  = {QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models},
  author = {Yuhui Xu and Lingxi Xie and Xiaotao Gu and Xin Chen and Heng Chang and Hengheng Zhang and Zhensu Chen and Xiaopeng Zhang and Qi Tian},
  year   = {2026},
  eprint = {2309.14717},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/pdf/2309.14717},
  journal = {ICLR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.14717.svg)](https://codeofpaper.com/paper/2309.14717)

Discussion