Position: Do pretrained Transformers Learn In-Context by Gradient Descent?

Lingfeng Shen, Aayush Mishra, Daniel Khashabi
2/12/2026

No confident code implementation found for this paper. 3 possible matches can be shown below.

Semantic Scholar

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Cite this paper

@article{shen2026position,
  title  = {Position: Do pretrained Transformers Learn In-Context by Gradient Descent?},
  author = {Lingfeng Shen and Aayush Mishra and Daniel Khashabi},
  year   = {2026},
  url    = {https://api.semanticscholar.org/CorpusID:703ead78cdfa8e77b7800d29a9163d22891d4adc},
  journal = {ICML 2024 2024}
}

Discussion