VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation

Ruiyang Zhang, Hu Zhang, Zhedong Zheng
11/18/2024
cs.CV

No official code found. Best implementation: Ruiyang-061X/VL-Uncertainty (56 stars, last commit 2025-03-18).

Abstract

Given the higher information load processed by large vision-language models (LVLMs) compared to single-modal LLMs, detecting LVLM hallucinations requires more human and time expense, and thus rise a wider safety concerns. In this paper, we introduce VL-Uncertainty, the first uncertainty-based framework for detecting hallucinations in LVLMs. Different from most existing methods that require ground-truth or pseudo annotations, VL-Uncertainty utilizes uncertainty as an intrinsic metric. We measure uncertainty by analyzing the prediction variance across semantically equivalent but perturbed prompts, including visual and textual data. When LVLMs are highly confident, they provide consistent responses to semantically equivalent queries. However, when uncertain, the responses of the target LVLM become more random. Considering semantically similar answers with different wordings, we cluster LVLM responses based on their semantic content and then calculate the cluster distribution entropy as the uncertainty measure to detect hallucination. Our extensive experiments on 10 LVLMs across four benchmarks, covering both free-form and multi-choice tasks, show that VL-Uncertainty significantly outperforms strong baseline methods in hallucination detection.

View on arXivView PDF

Code Implementations(1)

Ruiyang-061X/VL-UncertaintyCommunity71%

🔎Official code for our paper: "VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation".

565Shell, PythonNov 7, 20241 years ago
hallucinationhallucination-detectionhallucination-evaluationlarge-vision-language-modelmulti-modal+7 more

Similar Papers

UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Models

93% match2025

Uncertainty Quantification for Hallucination Detection in Large Language Models: Foundations, Methodology, and Future Directions

91% match2025

The Map of Misbelief: Tracing Intrinsic and Extrinsic Hallucinations Through Attention Patterns

91% match2025

HEDGE: Hallucination Estimation via Dense Geometric Entropy for VQA with Vision-Language Models

90% match1 repo2025

A Head to Predict and a Head to Question: Pre-trained Uncertainty Quantification Heads for Hallucination Detection in LLM Outputs

90% match2025

DynHD: Hallucination Detection for Diffusion Large Language Models via Denoising Dynamics Deviation Learning

90% match2026

Cite this paper

@article{zhang2024vluncertainty,
  title  = {VL-Uncertainty: Detecting Hallucination in Large Vision-Language Model via Uncertainty Estimation},
  author = {Ruiyang Zhang and Hu Zhang and Zhedong Zheng},
  year   = {2024},
  eprint = {2411.11919},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2411.11919v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2411.11919.svg)](https://codeofpaper.com/paper/2411.11919)

Discussion