Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering

Yucheng Li
4/24/2023
cs.CL

No official code found. Best implementation: liyucheng09/Selective_Context (423 stars, last commit 2024-02-12).

Abstract

Large language models (LLMs) have received significant attention by achieving remarkable performance across various tasks. However, their fixed context length poses challenges when processing long documents or maintaining extended conversations. This paper proposes a method called \textit{Selective Context} that employs self-information to filter out less informative content, thereby enhancing the efficiency of the fixed context length. We demonstrate the effectiveness of our approach on tasks of summarisation and question answering across different data sources, including academic papers, news articles, and conversation transcripts.

View on arXivView PDF

Code Implementations(1)

liyucheng09/Selective_ContextCommunity100%

Compress your input to ChatGPT or other LLMs, to let them process 2x more content and save 40% memory and GPU time.

42325Python, Jupyter NotebookMar 29, 20232 years ago
chatgptllmsnlp

Similar Papers

Compressing Context to Enhance Inference Efficiency of Large Language Models

87% match1 repo4232023

Lost in the Middle: How Language Models Use Long Contexts

85% match1 repo3862023

Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading

85% match2023

Never Lost in the Middle: Mastering Long-Context Question Answering with Position-Agnostic Decompositional Training

84% match1 repo22023

Memorilla: Latent Semantic Memory for LLMs

83% match1 repo12026

MphayaNER: Named Entity Recognition for Tshivenda

83% match2023

Cite this paper

@article{li2023unlocking,
  title  = {Unlocking Context Constraints of LLMs: Enhancing Context Efficiency of LLMs with Self-Information-Based Content Filtering},
  author = {Yucheng Li},
  year   = {2023},
  eprint = {2304.12102},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.12102v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.12102.svg)](https://codeofpaper.com/paper/2304.12102)

Discussion