Text Embeddings Reveal (Almost) As Much As Text

John X. Morris, Volodymyr Kuleshov, Vitaly Shmatikov, Alexander M. Rush
10/10/2023
cs.CLcs.LG

No official code found. Best implementation: vec2text/vec2text (1,137 stars, last commit 2025-12-27).

Abstract

How much private information do text embeddings reveal about the original text? We investigate the problem of embedding \textit{inversion}, reconstructing the full text represented in dense text embeddings. We frame the problem as controlled generation: generating text that, when reembedded, is close to a fixed point in latent space. We find that although a naïve model conditioned on the embedding performs poorly, a multi-step method that iteratively corrects and re-embeds text is able to recover $92\%$ of $32\text{-token}$ text inputs exactly. We train our model to decode text embeddings from two state-of-the-art embedding models, and also show that our model can recover important personal information (full names) from a dataset of clinical notes. Our code is available on Github: \href{https://github.com/jxmorris12/vec2text}{github.com/jxmorris12/vec2text}.

View on arXivView PDF

Code Implementations(1)

vec2text/vec2textCommunity100%

utilities for decoding deep representations (like sentence embeddings) back to text

1,137118Shell, PythonFeb 25, 20239 months agoNOASSERTION

Similar Papers

Universal Zero-shot Embedding Inversion

80% match1 repo322025

ALGEN: Few-shot Inversion Attacks on Textual Embeddings using Alignment and Generation

79% match1 repo12025

Text Simplification with Sentence Embeddings

76% match2025

Rep2Text: Decoding Full Text from a Single LLM Token Representation

76% match2025

Improving Text Embeddings with Large Language Models

76% match2023

Cropping outperforms dropout as an augmentation strategy for training self-supervised text embeddings

75% match2025

Cite this paper

@article{morris2023text,
  title  = {Text Embeddings Reveal (Almost) As Much As Text},
  author = {John X. Morris and Volodymyr Kuleshov and Vitaly Shmatikov and Alexander M. Rush},
  year   = {2023},
  eprint = {2310.06816},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.06816v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.06816.svg)](https://codeofpaper.com/paper/2310.06816)

Discussion