LLMZip: Lossless Text Compression using Large Language Models

Chandra Shekhara Kaushik Valmeekam, Krishna Narayanan, Dileep Kalathil, Jean-Francois Chamberland, Srinivas Shakkottai
6/6/2023
cs.ITcs.CLcs.LG

No official code found. Best implementation: vcskaushik/LLMzip (66 stars, GPL-3.0, last commit 2025-01-14).

Abstract

We provide new estimates of an asymptotic upper bound on the entropy of English using the large language model LLaMA-7B as a predictor for the next token given a window of past tokens. This estimate is significantly smaller than currently available estimates in \cite{cover1978convergent}, \cite{lutati2023focus}. A natural byproduct is an algorithm for lossless compression of English text which combines the prediction from the large language model with a lossless compression scheme. Preliminary results from limited experiments suggest that our scheme outperforms state-of-the-art text compression schemes such as BSC, ZPAQ, and paq8h.

View on arXivView PDF

Code Implementations(1)

vcskaushik/LLMzipCommunity100%
GPL-3.0
666Shell, PythonJun 22, 20231 years agoGPL-3.0

Similar Papers

Lossless Compression of Large Language Model-Generated Text via Next-Token Prediction

89% match2025

Ranking LLMs by compression

86% match2024

FineZip : Pushing the Limits of Large Language Models for Practical Lossless Text Compression

86% match1 repo202024

Revisiting Data Compression with Language Modeling

86% match2026

Compression Laws for Large Language Models

85% match2025

Llamazip: Leveraging LLaMA for Lossless Text Compression and Training Dataset Detection

85% match2025

Cite this paper

@article{valmeekam2023llmzip,
  title  = {LLMZip: Lossless Text Compression using Large Language Models},
  author = {Chandra Shekhara Kaushik Valmeekam and Krishna Narayanan and Dileep Kalathil and Jean-Francois Chamberland and Srinivas Shakkottai},
  year   = {2023},
  eprint = {2306.04050},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.04050v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.04050.svg)](https://codeofpaper.com/paper/2306.04050)

Discussion