Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks

Changhyeon Lee, Seulki Lee
2/12/2026

No official code found. Best implementation: eai-lab/SoftmaxOutputApproximation (76 stars, last commit 2024-06-07).

DOISemantic Scholar

Code Implementations(1)

eai-lab/SoftmaxOutputApproximationCommunity66%

[NeurIPS 2023] Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks

761PythonOct 10, 20232 years ago

Cite this paper

@article{lee2026softmax,
  title  = {Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks},
  author = {Changhyeon Lee and Seulki Lee},
  year   = {2026},
  doi    = {10.52202/075280-0664},
  url    = {https://doi.org/10.52202/075280-0664},
  journal = {NEURIPS 2023 2023}
}

Discussion