Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts

Taehun Cha, Donghun Lee
9/25/2024
cs.CL

No official code found. Best implementation: AIML-K/HalluDist (4 stars, MIT, last commit 2024-09-26).

Abstract

In this work, we show the pre-trained language models return distinguishable generation probability and uncertainty distribution to unfaithfully hallucinated texts, regardless of their size and structure. By examining 24 models on 6 data sets, we find out that 88-98% of cases return statistically significantly distinguishable generation probability and uncertainty distributions. Using this general phenomenon, we showcase a hallucination-reducing training algorithm. Our algorithm outperforms other baselines by achieving higher faithfulness metrics while maintaining sound general text quality measures.

View on arXivView PDF

Code Implementations(1)

AIML-K/HalluDistCommunity100%
MIT

Code for Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts (EMNLP Findings 2024)

40Shell, PythonSep 25, 20242 years agoMIT

Similar Papers

Why Language Models Hallucinate

88% match1 repo2025

Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directions

88% match2025

Detecting Hallucinations in Large Language Model Generation: A Token Probability Approach

88% match1 repo172024

Improving Uncertainty Estimation through Semantically Diverse Language Generation

87% match1 repo282024

Theoretical Foundations and Mitigation of Hallucination in Large Language Models

87% match2025

Mathematical Analysis of Hallucination Dynamics in Large Language Models: Uncertainty Quantification, Advanced Decoding, and Principled Mitigation

86% match2025

Cite this paper

@article{cha2024pretrained,
  title  = {Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts},
  author = {Taehun Cha and Donghun Lee},
  year   = {2024},
  eprint = {2409.16658},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.16658v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.16658.svg)](https://codeofpaper.com/paper/2409.16658)

Discussion