Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing

Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani +3 more
2/25/2024
cs.CL

Official code: UCSB-NLP-Chang/SemanticSmooth (24 stars, MIT, last commit 2024-06-09).

Abstract

Aligned large language models (LLMs) are vulnerable to jailbreaking attacks, which bypass the safeguards of targeted LLMs and fool them into generating objectionable content. While initial defenses show promise against token-based threat models, there do not exist defenses that provide robustness against semantic attacks and avoid unfavorable trade-offs between robustness and nominal performance. To meet this need, we propose SEMANTICSMOOTH, a smoothing-based defense that aggregates the predictions of multiple semantically transformed copies of a given input prompt. Experimental results demonstrate that SEMANTICSMOOTH achieves state-of-the-art robustness against GCG, PAIR, and AutoDAN attacks while maintaining strong nominal performance on instruction following benchmarks such as InstructionFollowing and AlpacaEval. The codes will be publicly available at https://github.com/UCSB-NLP-Chang/SemanticSmooth.

View on arXivView PDF

Code Implementations(1)

UCSB-NLP-Chang/SemanticSmoothOfficial100%
MIT

Implementation of paper 'Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing'

245PythonMar 1, 20242 years agoMIT
large-language-modelsllm-jailbreaks

Similar Papers

SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks

90% match1 repo1352023

Safeguarding Large Language Models in Real-time with Tunable Safety-Performance Trade-offs

90% match1 repo12025

Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective

90% match2024

An Interpretable N-gram Perplexity Threat Model for Large Language Model Jailbreaks

89% match1 repo122024

Jailbreaking Large Language Models Through Alignment Vulnerabilities in Out-of-Distribution Settings

89% match1 repo42024

Jailbreak Attacks and Defenses Against Large Language Models: A Survey

89% match2024

Cite this paper

@article{ji2024defending,
  title  = {Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing},
  author = {Jiabao Ji and Bairu Hou and Alexander Robey and George J. Pappas and Hamed Hassani and Yang Zhang and Eric Wong and Shiyu Chang},
  year   = {2024},
  eprint = {2402.16192},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.16192v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.16192.svg)](https://codeofpaper.com/paper/2402.16192)

Discussion