Evaluating Uncertainty Quantification Methods in Argumentative Large Language Models

Kevin Zhou, Adam Dejl, Gabriel Freedman, Lihu Chen, Antonio Rago +1 more
9/26/2025
cs.CLcs.AI

No official code found. Best implementation: CLArg-group/argumentative-llms-uq (0 stars, last commit 2025-09-18).

Abstract

Research in uncertainty quantification (UQ) for large language models (LLMs) is increasingly important towards guaranteeing the reliability of this groundbreaking technology. We explore the integration of LLM UQ methods in argumentative LLMs (ArgLLMs), an explainable LLM framework for decision-making based on computational argumentation in which UQ plays a critical role. We conduct experiments to evaluate ArgLLMs' performance on claim verification tasks when using different LLM UQ methods, inherently performing an assessment of the UQ methods' effectiveness. Moreover, the experimental procedure itself is a novel way of evaluating the effectiveness of UQ methods, especially when intricate and potentially contentious statements are present. Our results demonstrate that, despite its simplicity, direct prompting is an effective UQ strategy in ArgLLMs, outperforming considerably more complex approaches.

View on arXivView PDF

Code Implementations(1)

CLArg-group/argumentative-llms-uqCommunity70%

Code for the "Evaluating Uncertainty Quantification Methods in Argumentative Large Language Models" paper at EMNLP Findings 2025.

01PythonSep 18, 20251 years agoNOASSERTION

Similar Papers

Uncertainty Quantification and Confidence Calibration in Large Language Models: A Survey

90% match2025

Argumentative Large Language Models for Explainable and Contestable Claim Verification

89% match1 repo192024

Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency

88% match2025

Black-box Uncertainty Quantification Method for LLM-as-a-Judge

88% match2024

Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs

87% match1 repo2025

CoT-UQ: Improving Response-wise Uncertainty Quantification in LLMs with Chain-of-Thought

87% match1 repo172025

Cite this paper

@article{zhou2025evaluating,
  title  = {Evaluating Uncertainty Quantification Methods in Argumentative Large Language Models},
  author = {Kevin Zhou and Adam Dejl and Gabriel Freedman and Lihu Chen and Antonio Rago and Francesca Toni},
  year   = {2025},
  eprint = {2510.02339},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2510.02339v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2510.02339.svg)](https://codeofpaper.com/paper/2510.02339)

Discussion