Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward

Arnav Chavan, Raghav Magazine, Shubham Kushwaha, Mérouane Debbah, Deepak Gupta
2/2/2024
cs.LGcs.AIcs.CL

Official code: nyunAI/Faster-LLM-Survey (42 stars, last commit 2024-04-23).

Abstract

Despite the impressive performance of LLMs, their widespread adoption faces challenges due to substantial computational and memory requirements during inference. Recent advancements in model compression and system-level optimization methods aim to enhance LLM inference. This survey offers an overview of these methods, emphasizing recent developments. Through experiments on LLaMA(/2)-7B, we evaluate various compression techniques, providing practical insights for efficient LLM deployment in a unified setting. The empirical analysis on LLaMA(/2)-7B highlights the effectiveness of these methods. Drawing from survey insights, we identify current limitations and discuss potential future directions to improve LLM inference efficiency. We release the codebase to reproduce the results presented in this paper at https://github.com/nyunAI/Faster-LLM-Survey

View on arXivView PDF

Code Implementations(1)

nyunAI/Faster-LLM-SurveyOfficial100%
426Shell, PythonJan 15, 20242 years ago

Similar Papers

Compress, Then Prompt: Improving Accuracy-Efficiency Trade-off of LLM Inference with Transferable Prompt

87% match2023

LLM Inference Unveiled: Survey and Roofline Model Insights

87% match1 repo6502024

A Speed Odyssey for Deployable Quantization of LLMs

87% match2023

xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference

86% match3 repos2.2k2025

Enhancing Inference Efficiency of Large Language Models: Investigating Optimization Strategies and Architectural Innovations

86% match2024

Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques

86% match2025

Cite this paper

@article{chavan2024faster,
  title  = {Faster and Lighter LLMs: A Survey on Current Challenges and Way Forward},
  author = {Arnav Chavan and Raghav Magazine and Shubham Kushwaha and Mérouane Debbah and Deepak Gupta},
  year   = {2024},
  eprint = {2402.01799},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.01799v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.01799.svg)](https://codeofpaper.com/paper/2402.01799)

Discussion