Hallucination Benchmark in Medical Visual Question Answering

Jinge Wu, Yunsoo Kim, Honghan Wu
1/11/2024
cs.CLcs.AIcs.CV

No official code found. Best implementation: knowlab/halt-medvqa (9 stars, last commit 2024-04-02).

Abstract

The recent success of large language and vision models (LLVMs) on vision question answering (VQA), particularly their applications in medicine (Med-VQA), has shown a great potential of realizing effective visual assistants for healthcare. However, these models are not extensively tested on the hallucination phenomenon in clinical settings. Here, we created a hallucination benchmark of medical images paired with question-answer sets and conducted a comprehensive evaluation of the state-of-the-art models. The study provides an in-depth analysis of current models' limitations and reveals the effectiveness of various prompting strategies.

View on arXivView PDF

Code Implementations(1)

knowlab/halt-medvqaCommunity100%
91Python, Jupyter NotebookApr 1, 20242 years ago

Similar Papers

Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering

91% match2024

MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models

90% match1 repo52025

V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering

88% match2026

HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning

87% match1 repo252024

A Unified Hallucination Mitigation Framework for Large Vision-Language Models

86% match1 repo92024

Towards a Systematic Evaluation of Hallucinations in Large-Vision Language Models

86% match1 repo82024

Cite this paper

@article{wu2024hallucination,
  title  = {Hallucination Benchmark in Medical Visual Question Answering},
  author = {Jinge Wu and Yunsoo Kim and Honghan Wu},
  year   = {2024},
  eprint = {2401.05827},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.05827v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.05827.svg)](https://codeofpaper.com/paper/2401.05827)

Discussion