Small Language Models (SLMs) Can Still Pack a Punch: A survey

Shreyas Subramanian, Vikram Elango, Mecit Gungor
1/3/2025
cs.CL

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

As foundation AI models continue to increase in size, an important question arises - is massive scale the only path forward? This survey of about 160 papers presents a family of Small Language Models (SLMs) in the 1 to 8 billion parameter range that demonstrate smaller models can perform as well, or even outperform large models. We explore task agnostic, general purpose SLMs, task-specific SLMs and techniques to create SLMs that can guide the community to build models while balancing performance, efficiency, scalability and cost. Furthermore we define and characterize SLMs' effective sizes, representing increased capability with respect to LLMs.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

What is the Role of Small Models in the LLM Era: A Survey

90% match1 repo1122024

A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness

90% match1 repo2612024

Small Language Models: Survey, Measurements, and Insights

88% match1 repo1092024

A Survey on Collaborative Mechanisms Between Large and Small Language Models

88% match2025

A Primer on Large Language Models and their Limitations

88% match2024

Does Model Size Matter? A Comparison of Small and Large Language Models for Requirements Classification

88% match1 repo2025

Cite this paper

@article{subramanian2025small,
  title  = {Small Language Models (SLMs) Can Still Pack a Punch: A survey},
  author = {Shreyas Subramanian and Vikram Elango and Mecit Gungor},
  year   = {2025},
  eprint = {2501.05465},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.05465v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.05465.svg)](https://codeofpaper.com/paper/2501.05465)

Discussion