Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generation

Wei-Lin Chen, Cheng-Kuang Wu, Hsin-Hsi Chen, Chung-Chi Chen
10/23/2023
cs.CL

No official code found. Best implementation: ntunlplab/FECS (7 stars, last commit 2023-11-28).

Abstract

In this paper, we address the hallucination problem commonly found in natural language generation tasks. Language models often generate fluent and convincing content but can lack consistency with the provided source, resulting in potential inaccuracies. We propose a new decoding method called Fidelity-Enriched Contrastive Search (FECS), which augments the contrastive search framework with context-aware regularization terms. FECS promotes tokens that are semantically similar to the provided source while penalizing repetitiveness in the generated text. We demonstrate its effectiveness across two tasks prone to hallucination: abstractive summarization and dialogue generation. Results show that FECS consistently enhances faithfulness across various language model sizes while maintaining output diversity comparable to well-performing decoding algorithms.

View on arXivView PDF

Code Implementations(1)

ntunlplab/FECSCommunity100%
70PythonOct 15, 20232 years ago

Similar Papers

Building Reliable Long-Form Generation via Hallucination Rejection Sampling

88% match1 repo12026

MAP: Mitigating Hallucinations in Large Vision-Language Models with Map-Level Attention Processing

88% match2025

Rethinking CD: A Reproducibility Study and Extension on the Ineffectiveness of Contrastive Decoding at Mitigating Object Hallucinations in MLLMs

88% match2026

Alleviating Hallucinations of Large Language Models through Induced Hallucinations

88% match2 repos72.8k2023

Don't Believe Everything You Read: Enhancing Summarization Interpretability through Automatic Identification of Hallucinations in Large Language Models

88% match2023

Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models

88% match1 repo1362024

Cite this paper

@article{chen2023fidelityenriched,
  title  = {Fidelity-Enriched Contrastive Search: Reconciling the Faithfulness-Diversity Trade-Off in Text Generation},
  author = {Wei-Lin Chen and Cheng-Kuang Wu and Hsin-Hsi Chen and Chung-Chi Chen},
  year   = {2023},
  eprint = {2310.14981},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.14981v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.14981.svg)](https://codeofpaper.com/paper/2310.14981)

Discussion