ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models

Mingda Li, Xinyu Li, Weinan Zhang, Longxuan Ma
10/15/2025
cs.CLcs.AIcs.LG

No official code found. Best implementation: mingdali6717/ESI-UQ (0 stars, last commit 2025-10-13).

Abstract

Uncertainty Quantification (UQ) is a promising approach to improve model reliability, yet quantifying the uncertainty of Large Language Models (LLMs) is non-trivial. In this work, we establish a connection between the uncertainty of LLMs and their invariance under semantic-preserving intervention from a causal perspective. Building on this foundation, we propose a novel grey-box uncertainty quantification method that measures the variation in model outputs before and after the semantic-preserving intervention. Through theoretical justification, we show that our method provides an effective estimate of epistemic uncertainty. Our extensive experiments, conducted across various LLMs and a variety of question-answering (QA) datasets, demonstrate that our method excels not only in terms of effectiveness but also in computational efficiency.

View on arXivView PDF

Code Implementations(1)

mingdali6717/ESI-UQCommunity58%

This is the official repository for our paper [ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models]

00Shell, PythonOct 13, 20251 years ago

Similar Papers

Improving Uncertainty Quantification in Large Language Models via Semantic Embeddings

90% match2024

Enhancing Trust in Large Language Models with Uncertainty-Aware Fine-Tuning

89% match2024

Measuring Aleatoric and Epistemic Uncertainty in LLMs: Empirical Evaluation on ID and OOD QA Tasks

89% match2025

Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey

89% match2025

Efficient semantic uncertainty quantification in language models via diversity-steered sampling

88% match2025

To Believe or Not to Believe Your LLM

88% match2024

Cite this paper

@article{li2025esi,
  title  = {ESI: Epistemic Uncertainty Quantification via Semantic-preserving Intervention for Large Language Models},
  author = {Mingda Li and Xinyu Li and Weinan Zhang and Longxuan Ma},
  year   = {2025},
  eprint = {2510.13103},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.13103v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.13103.svg)](https://codeofpaper.com/paper/2510.13103)

Discussion