A Survey on Hallucination in Large Vision-Language Models

Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao +4 more
2/1/2024
cs.CVcs.CLcs.LG

No official code found. Best implementation: lhanchao777/LVLM-Hallucinations-Survey (96 stars, Apache-2.0, last commit 2024-07-26).

Abstract

Recent development of Large Vision-Language Models (LVLMs) has attracted growing attention within the AI landscape for its practical implementation potential. However, ``hallucination'', or more specifically, the misalignment between factual visual content and corresponding textual generation, poses a significant challenge of utilizing LVLMs. In this comprehensive survey, we dissect LVLM-related hallucinations in an attempt to establish an overview and facilitate future mitigation. Our scrutiny starts with a clarification of the concept of hallucinations in LVLMs, presenting a variety of hallucination symptoms and highlighting the unique challenges inherent in LVLM hallucinations. Subsequently, we outline the benchmarks and methodologies tailored specifically for evaluating hallucinations unique to LVLMs. Additionally, we delve into an investigation of the root causes of these hallucinations, encompassing insights from the training data and model components. We also critically review existing methods for mitigating hallucinations. The open questions and future directions pertaining to hallucinations within LVLMs are discussed to conclude this survey.

View on arXivView PDF

Code Implementations(1)

lhanchao777/LVLM-Hallucinations-SurveyCommunity100%
Apache-2.0

This is the first released survey paper on hallucinations of large vision-language models (LVLMs). To keep track of this field and continuously update our survey, we maintain this repository of relevant references.

967Sep 19, 20232 years agoApache-2.0

Similar Papers

A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models

95% match2025

Evaluation and Analysis of Hallucination in Large Vision-Language Models

94% match1 repo192023

Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence

93% match1 repo212024

Analyzing and Mitigating Object Hallucination in Large Vision-Language Models

92% match1 repo1592026

Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models

92% match1 repo152024

Hallucination of Multimodal Large Language Models: A Survey

92% match1 repo1.0k2024

Cite this paper

@article{liu2024survey,
  title  = {A Survey on Hallucination in Large Vision-Language Models},
  author = {Hanchao Liu and Wenyuan Xue and Yifei Chen and Dapeng Chen and Xiutian Zhao and Ke Wang and Liping Hou and Rongjun Li and Wei Peng},
  year   = {2024},
  eprint = {2402.00253},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.00253v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.00253.svg)](https://codeofpaper.com/paper/2402.00253)

Discussion