Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias

Anshuman Chhabra, Hadi Askari, Prasant Mohapatra
1/3/2024
cs.CLcs.AI

No confident code implementation found for this paper.

Abstract

We characterize and study zero-shot abstractive summarization in Large Language Models (LLMs) by measuring position bias, which we propose as a general formulation of the more restrictive lead bias phenomenon studied previously in the literature. Position bias captures the tendency of a model unfairly prioritizing information from certain parts of the input text over others, leading to undesirable behavior. Through numerous experiments on four diverse real-world datasets, we study position bias in multiple LLM models such as GPT 3.5-Turbo, Llama-2, and Dolly-v2, as well as state-of-the-art pretrained encoder-decoder abstractive summarization models such as Pegasus and BART. Our findings lead to novel insights and discussion on performance and position bias of models for zero-shot summarization tasks.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Serial Position Effects of Large Language Models

85% match2024

Assessing LLMs for Zero-shot Abstractive Summarization Through the Lens of Relevance Paraphrasing

85% match1 repo12024

Eliminating Position Bias of Language Models: A Mechanistic Approach

84% match1 repo232026

Mitigate Position Bias in Large Language Models via Scaling a Single Dimension

84% match1 repo122024

On Positional Bias of Faithfulness for Long-form Summarization

84% match1 repo22024

Position-Aware Parameter Efficient Fine-Tuning Approach for Reducing Positional Bias in LLMs

84% match2024

Cite this paper

@article{chhabra2024revisiting,
  title  = {Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias},
  author = {Anshuman Chhabra and Hadi Askari and Prasant Mohapatra},
  year   = {2024},
  eprint = {2401.01989},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.01989v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.01989.svg)](https://codeofpaper.com/paper/2401.01989)

Discussion