On Pruning State-Space LLMs

Tamer Ghattas, Michael Hassid, Roy Schwartz
2/26/2025
cs.CLcs.LG

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

Recent work proposed state-space models (SSMs) as an efficient alternative to transformer-based LLMs. Can these models be pruned to further reduce their computation costs? We adapt several pruning methods to the SSM structure, and apply them to four SSM-based LLMs across multiple tasks. We find that such models are quite robust to some pruning methods (e.g. WANDA), while using other methods lead to fast performance degradation.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{ghattas2025pruning,
  title  = {On Pruning State-Space LLMs},
  author = {Tamer Ghattas and Michael Hassid and Roy Schwartz},
  year   = {2025},
  eprint = {2502.18886},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2502.18886v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2502.18886.svg)](https://codeofpaper.com/paper/2502.18886)

Discussion