LLM Architecture, Scaling Laws, and Economics: A Quick Summary

William H. Press
9/11/2025
cs.GLcs.CLcs.LG

No confident code implementation found for this paper.

Abstract

The current standard architecture of Large Language Models (LLMs) with QKV self-attention is briefly summarized, including the architecture of a typical Transformer. Scaling laws for compute (flops) and memory (parameters plus data) are given, along with their present (2025) rough cost estimates for the parameters of present LLMs of various scales, including discussion of whether DeepSeek should be viewed as a special case. Nothing here is new, but this material seems not otherwise readily available in summary form.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws

86% match2025

Uncovering Scaling Laws for Large Language Models via Inverse Problems

86% match2025

Has LLM Reached the Scaling Ceiling Yet? Unified Insights into LLM Regularities and Constraints

86% match2024

Position: Enough of Scaling LLMs! Lets Focus on Downscaling

85% match1 repo22026

Performance Law of Large Language Models

85% match1 repo52024

DeepSeek LLM: Scaling Open-Source Language Models with Longtermism

85% match1 repo7.3k2024

Cite this paper

@article{press2025architecture,
  title  = {LLM Architecture, Scaling Laws, and Economics: A Quick Summary},
  author = {William H. Press},
  year   = {2025},
  eprint = {2511.11572},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2511.11572v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2511.11572.svg)](https://codeofpaper.com/paper/2511.11572)

Discussion