Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning

Xinyi Wang, Wanrong Zhu, Michael Stephen Saxon, William Yang Wang
2/12/2026

No official code found. Best implementation: WANGXinyiLinda/concept-based-demonstration-selection (76 stars, MIT, last commit 2024-03-20).

Abstract

In recent years, pre-trained large language models (LLMs) have demonstrated remarkable efficiency in achieving an inference-time few-shot learning capability known as in-context learning. However, existing literature has highlighted the sensitivity of this capability to the selection of few-shot demonstrations. Current understandings of the underlying mechanisms by which this capability arises from regular language model pretraining objectives remain disconnected from the real-world LLMs. This study aims to examine the in-context learning phenomenon through a Bayesian lens, viewing real-world LLMs as latent variable models. On this premise, we propose an algorithm to select optimal demonstrations from a set of annotated data with a small LM, and then directly generalize the selected demonstrations to larger LMs. We demonstrate significant improvement over baselines, averaged over eight GPT models on eight real-world text classification datasets. We also demonstrate the real-world usefulness of our algorithm on GSM8K, a math word problem dataset. Our empirical findings support our hypothesis that LLMs implicitly infer a latent variable containing task information.

View on arXivView PDF

Code Implementations(1)

WANGXinyiLinda/concept-based-demonstration-selectionCommunity100%
MIT

Offical code of the paper Large Language Models Are Implicitly Topic Models: Explaining and Finding Good Demonstrations for In-Context Learning.

766Shell, PythonJan 26, 20232 years agoMIT
deep-learninglarge-language-modelsmachine-learningprompting

Similar Papers

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning

100% match1 repo762023

What Do Language Models Learn in Context? The Structured Task Hypothesis

87% match1 repo112024

Emergent Abilities in Reduced-Scale Generative Language Models

87% match2024

Fine-Tuned In-Context Learners for Efficient Adaptation

87% match2025

Large Language Models Can be Lazy Learners: Analyze Shortcuts in In-Context Learning

87% match2023

How Does the Pretraining Distribution Shape In-Context Learning? Task Selection, Generalization, and Robustness

86% match2025

Cite this paper

@article{wang2026large,
  title  = {Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning},
  author = {Xinyi Wang and Wanrong Zhu and Michael Stephen Saxon and William Yang Wang},
  year   = {2026},
  eprint = {2301.11916},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/2301.11916},
  journal = {NEURIPS 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2301.11916.svg)](https://codeofpaper.com/paper/2301.11916)

Discussion