RetVec: Resilient and Efficient Text Vectorizer

Elie Bursztein, Marina Zhang, Owen Vallis, Xinyu Jia, Alexey Kurakin
2/1/2023

No official code found. Best implementation: google-research/retvec (292 stars, Apache-2.0, TensorFlow, last commit 2025-04-04).

Abstract

This paper describes RETVec, an efficient, resilient, and multilingual text vectorizer designed for neural-based text processing. RETVec combines a novel character encoding with an optional small embedding model to embed words into a 256-dimensional vector space. The RETVec embedding model is pre-trained using pair-wise metric learning to be robust against typos and character-level adversarial attacks. In this paper, we evaluate and compare RETVec to state-of-the-art vectorizers and word embeddings on popular model architectures and datasets. These comparisons demonstrate that RETVec leads to competitive, multilingual models that are significantly more resilient to typos and adversarial text attacks. RETVec is available under the Apache 2 license at https://github.com/google-research/retvec.

View on arXivView PDF

Code Implementations(1)

google-research/retvecCommunity100%
TensorFlowApache-2.0

RETVec is an efficient, multilingual, and adversarially-robust text vectorizer.

29225HTML, ShellFeb 1, 20221 years agoApache-2.0
deep-learningnatural-language-processingnlppythontensorflow+1 more

Similar Papers

Lightweight Adaptation of Neural Language Models via Subspace Embedding

80% match2023

On the Robustness of Text Vectorizers

79% match2023

From Word Vectors to Multimodal Embeddings: Techniques, Applications, and Future Directions For Large Language Models

79% match2024

DIVE: Embedding Compression via Self-Limiting Gradient Updates

78% match2026

PDA: Text-Augmented Defense Framework for Robust Vision-Language Models against Adversarial Image Attacks

78% match2026

RETSim: Resilient and Efficient Text Similarity

78% match2 repos4.3k2023

Cite this paper

@article{bursztein2023retvec,
  title  = {RetVec: Resilient and Efficient Text Vectorizer},
  author = {Elie Bursztein and Marina Zhang and Owen Vallis and Xinyu Jia and Alexey Kurakin},
  year   = {2023},
  eprint = {2302.09207},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/pdf/2302.09207},
  journal = {NEURIPS 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2302.09207.svg)](https://codeofpaper.com/paper/2302.09207)

Discussion