SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs

Wenhua Cheng, Weiwei Zhang, Heng Guo, Haihao Shen
12/4/2025
cs.CLcs.AI

Official code: intel/auto-round (1,503 stars, Apache-2.0, last commit 2026-04-30).

Abstract

Extreme low-bit quantization is critical for efficiently deploying Large Language Models (LLMs), yet it often leads to severe performance degradation at 2-bits and even 4-bits (e.g., MXFP4). We present SignRoundV2, a post-training quantization framework that is highly effective even without mixed-precision. SignRoundV2 introduces (1) a fast sensitivity metric that combines gradient information with quantization-induced deviations to guide layer-wise bit allocation, and (2) a lightweight pre-tuning search for quantization scales to improve extremely low-bit quantization. These components allow SignRoundV2 to close the gap with full-precision models. Extensive experiments indicate that our method sustains competitive accuracy for LLMs, achieving production-grade performance with about 1 percent variance at 4-5 bits and strong results even at 2 bits. The implementation is available at https://github.com/intel/auto-round.

View on arXivView PDF

Code Implementations(1)

intel/auto-roundOfficial100%
Apache-2.0

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

1,503148Shell, PythonJan 4, 20245 months agoApache-2.0
ggufint4llmsmxfp4nvfp4+6 more

Similar Papers

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

88% match1 repo2024

FP4 All the Way: Fully Quantized Training of LLMs

87% match1 repo532025

Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs

87% match2 repos2.7k2023

A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms

87% match2024

LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

87% match2025

LQER: Low-Rank Quantization Error Reconstruction for LLMs

87% match1 repo192024

Cite this paper

@article{cheng2025signroundv,
  title  = {SignRoundV2: Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMs},
  author = {Wenhua Cheng and Weiwei Zhang and Heng Guo and Haihao Shen},
  year   = {2025},
  eprint = {2512.04746},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2512.04746v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2512.04746.svg)](https://codeofpaper.com/paper/2512.04746)

Discussion