Uncertainty quantification in fine-tuned LLMs using LoRA ensembles

Oleksandr Balabanov, Hampus Linander
2/19/2024
cs.LGcs.AIcs.CL

No official code found. Best implementation: oleksandr-balabanov/equivariant-posteriors (4 stars, last commit 2025-03-31).

Abstract

Fine-tuning large language models can improve task specific performance, although a general understanding of what the fine-tuned model has learned, forgotten and how to trust its predictions is still missing. We derive principled uncertainty quantification for fine-tuned LLMs with posterior approximations using computationally efficient low-rank adaptation ensembles. We analyze three common multiple-choice datasets using low-rank adaptation ensembles based on Mistral-7b, and draw quantitative and qualitative conclusions on their perceived complexity and balance between retained prior knowledge and domain specific adaptation during and after fine-tuning. We identify unexpected retention of acquired knowledge during fine-tuning in the overfitting regime.

View on arXivView PDF

Code Implementations(1)

oleksandr-balabanov/equivariant-posteriorsCommunity100%

Generative LLMs & Uncertainty Estimates

42R, NixDec 11, 20231 years ago

Similar Papers

LoRA ensembles for large language model fine-tuning

91% match2023

Large Language Models Must Be Taught to Know What They Don't Know

87% match1 repo532024

A Comparative Analysis of LLM Adaptation: SFT, LoRA, and ICL in Data-Scarce Scenarios

87% match2025

Amortized Bayesian Meta-Learning for Low-Rank Adaptation of Large Language Models

86% match2025

Robust and Efficient Fine-tuning of LLMs with Bayesian Reparameterization of Low-Rank Adaptation

86% match1 repo32024

Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRA

86% match1 repo2025

Cite this paper

@article{balabanov2024uncertainty,
  title  = {Uncertainty quantification in fine-tuned LLMs using LoRA ensembles},
  author = {Oleksandr Balabanov and Hampus Linander},
  year   = {2024},
  eprint = {2402.12264},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.12264v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.12264.svg)](https://codeofpaper.com/paper/2402.12264)

Discussion