HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims

Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, Kunwoo Park
10/16/2024
cs.CLcs.CY

Official code: ssu-humane/HerO (14 stars, last commit 2025-03-18).

Abstract

To tackle the AVeriTeC shared task hosted by the FEVER-24, we introduce a system that only employs publicly available large language models (LLMs) for each step of automated fact-checking, dubbed the Herd of Open LLMs for verifying real-world claims (HerO). For evidence retrieval, a language model is used to enhance a query by generating hypothetical fact-checking documents. We prompt pretrained and fine-tuned LLMs for question generation and veracity prediction by crafting prompts with retrieved in-context samples. HerO achieved 2nd place on the leaderboard with the AVeriTeC score of 0.57, suggesting the potential of open LLMs for verifying real-world claims. For future research, we make our code publicly available at https://github.com/ssu-humane/HerO.

View on arXivView PDF

Code Implementations(1)

ssu-humane/HerOOfficial100%

The code for HerO: a fact-checking pipeline based on open LLMs (the runner-up in AVeriTeC)

142PythonOct 5, 20241 years ago
fact-checkingfact-verificationlarge-language-modelsnatural-language-processingopen-llm

Similar Papers

AIC CTU system at AVeriTeC: Re-framing automated fact-checking as a simple RAG task

87% match1 repo62024

(Fact) Check Your Bias

87% match2025

RealFactBench: A Benchmark for Evaluating Large Language Models in Real-World Fact-Checking

86% match1 repo92025

Multimodal Large Language Models to Support Real-World Fact-Checking

86% match2024

Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs

86% match2025

ClaimCheck: Real-Time Fact-Checking with Small Language Models

86% match2025

Cite this paper

@article{yoon2024hero,
  title  = {HerO at AVeriTeC: The Herd of Open Large Language Models for Verifying Real-World Claims},
  author = {Yejun Yoon and Jaeyoon Jung and Seunghyun Yoon and Kunwoo Park},
  year   = {2024},
  eprint = {2410.12377},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2410.12377v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2410.12377.svg)](https://codeofpaper.com/paper/2410.12377)

Discussion