Membership Inference on LLMs in the Wild

Jiatong Yi, Yanyang Li
1/16/2026
cs.CL

No official code found. Best implementation: simmia2026/SimMIA (7 stars, MIT, last commit 2026-01-19).

Abstract

Membership Inference Attacks (MIAs) act as a crucial auditing tool for the opaque training data of Large Language Models (LLMs). However, existing techniques predominantly rely on inaccessible model internals (e.g., logits) or suffer from poor generalization across domains in strict black-box settings where only generated text is available. In this work, we propose SimMIA, a robust MIA framework tailored for this text-only regime by leveraging an advanced sampling strategy and scoring mechanism. Furthermore, we present WikiMIA-25, a new benchmark curated to evaluate MIA performance on modern proprietary LLMs. Experiments demonstrate that SimMIA achieves state-of-the-art results in the black-box setting, rivaling baselines that exploit internal model information.

View on arXivView PDF

Code Implementations(1)

simmia2026/SimMIACommunity62%
MIT

The code for paper 'Membership Inference on LLMs in the Wild'

70Shell, PythonJan 19, 20269 months agoMIT
black-boxevaluationinference-attackllm-inference

Similar Papers

Scaling Up Membership Inference: When and How Attacks Succeed on Large Language Models

91% match1 repo162024

On the Effectiveness of Membership Inference in Targeted Data Extraction from Large Language Models

90% match2025

Detecting Training Data of Large Language Models via Expectation Maximization

90% match1 repo72024

In-Context Probing for Membership Inference in Fine-Tuned Language Models

89% match1 repo122025

Order of Magnitude Speedups for LLM Membership Inference

89% match2024

Fast-MIA: Efficient and Scalable Membership Inference for LLMs

89% match1 repo152025

Cite this paper

@article{yi2026membership,
  title  = {Membership Inference on LLMs in the Wild},
  author = {Jiatong Yi and Yanyang Li},
  year   = {2026},
  eprint = {2601.11314},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2601.11314v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2601.11314.svg)](https://codeofpaper.com/paper/2601.11314)

Discussion