Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking

Mubashara Akhtar, Oana Cocarascu, Elena Simperl
1/27/2023
cs.CLcs.CV

No official code found. Best implementation: mubasharaak/ChartFC_chartBERT (10 stars, MIT, last commit 2023-06-18).

Abstract

Evidence data for automated fact-checking (AFC) can be in multiple modalities such as text, tables, images, audio, or video. While there is increasing interest in using images for AFC, previous works mostly focus on detecting manipulated or fake images. We propose a novel task, chart-based fact-checking, and introduce ChartBERT as the first model for AFC against chart evidence. ChartBERT leverages textual, structural and visual information of charts to determine the veracity of textual claims. For evaluation, we create ChartFC, a new dataset of 15, 886 charts. We systematically evaluate 75 different vision-language (VL) baselines and show that ChartBERT outperforms VL models, achieving 63.8% accuracy. Our results suggest that the task is complex yet feasible, with many challenges ahead.

View on arXivView PDF

Code Implementations(1)

mubasharaak/ChartFC_chartBERTCommunity63%
MIT

Data and Code for EACL 2023 Paper "Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking"

101Shell, PythonApr 10, 20223 years agoMIT

Similar Papers

ChartCheck: Explainable Fact-Checking over Real-World Chart Images

92% match1 repo102023

RED-DOT: Multimodal Fact-checking via Relevant Evidence Detection

85% match1 repo42023

Multimodal Automated Fact-Checking: A Survey

85% match1 repo5802023

Do LVLMs Understand Charts? Analyzing and Correcting Factual Errors in Chart Captioning

84% match1 repo272023

Multimodal Fact-Checking with Vision Language Models: A Probing Classifier based Solution with Embedding Strategies

83% match1 repo82024

MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models

83% match1 repo122024

Cite this paper

@article{akhtar2023reading,
  title  = {Reading and Reasoning over Chart Images for Evidence-based Automated Fact-Checking},
  author = {Mubashara Akhtar and Oana Cocarascu and Elena Simperl},
  year   = {2023},
  eprint = {2301.11843},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2301.11843v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.11843.svg)](https://codeofpaper.com/paper/2301.11843)

Discussion