Mitigating Hallucination in Multimodal LLMs with Layer Contrastive Decoding

Bingkui Tong, Jiaer Xia, Kaiyang Zhou
9/29/2025
cs.CV

Official code: maifoundations/LayerCD (4 stars, MIT, last commit 2025-09-30).

Abstract

Multimodal Large Language Models (MLLMs) have shown impressive perception and reasoning capabilities, yet they often suffer from hallucinations -- generating outputs that are linguistically coherent but inconsistent with the context of the input image, including inaccuracies in objects, attributes, and relations. To address this challenge, we propose a simple approach called Layer Contrastive Decoding (LayerCD). Our design is motivated by the observation that shallow visual features are much more likely than deep visual features to cause an MLLM to hallucinate as they only capture biased, low-level information that is insufficient for high-level reasoning. Therefore, LayerCD aims to filter out hallucinations by contrasting the output distributions generated from visual features of different levels, specifically those from the shallow and deep layers of the vision encoder, respectively. We conduct extensive experiments on two hallucination benchmarks and show that LayerCD significantly outperforms current state-of-the-art. The code for LayerCD is available at https://github.com/maifoundations/LayerCD .

View on arXivView PDF

Code Implementations(1)

maifoundations/LayerCDOfficial100%
MIT

Mitigating Hallucination in Multimodal LLMs with Layer Contrastive Decoding

42CSS, HTMLSep 23, 20251 years agoMIT

Similar Papers

CATCH: Complementary Adaptive Token-level Contrastive Decoding to Mitigate Hallucinations in LVLMs

92% match2024

VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding

91% match1 repo132024

Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding

91% match2 repos18.0k2026

AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models

90% match1 repo222024

Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning

90% match2024

On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models

90% match1 repo112025

Cite this paper

@article{tong2025mitigating,
  title  = {Mitigating Hallucination in Multimodal LLMs with Layer Contrastive Decoding},
  author = {Bingkui Tong and Jiaer Xia and Kaiyang Zhou},
  year   = {2025},
  eprint = {2509.25177},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.25177v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.25177.svg)](https://codeofpaper.com/paper/2509.25177)

Discussion