HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang +7 more
2/6/2024
cs.LGcs.AIcs.CL

Official code: centerforaisafety/HarmBench (1,062 stars, MIT, last commit 2024-08-16).

Abstract

Automated red teaming holds substantial promise for uncovering and mitigating the risks associated with the malicious use of large language models (LLMs), yet the field lacks a standardized evaluation framework to rigorously assess new methods. To address this issue, we introduce HarmBench, a standardized evaluation framework for automated red teaming. We identify several desirable properties previously unaccounted for in red teaming evaluations and systematically design HarmBench to meet these criteria. Using HarmBench, we conduct a large-scale comparison of 18 red teaming methods and 33 target LLMs and defenses, yielding novel insights. We also introduce a highly efficient adversarial training method that greatly enhances LLM robustness across a wide range of attacks, demonstrating how HarmBench enables codevelopment of attacks and defenses. We open source HarmBench at https://github.com/centerforaisafety/HarmBench.

View on arXivView PDF

Code Implementations(1)

centerforaisafety/HarmBenchOfficial100%
MIT

HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

1,062168MDX, ShellFeb 2, 20242 years agoMIT

Similar Papers

Automated Progressive Red Teaming

89% match1 repo62024

RedBench: A Universal Dataset for Comprehensive Red Teaming of Large Language Models

89% match1 repo292026

Holistic Automated Red Teaming for Large Language Models through Top-Down Test Case Generation and Multi-turn Interaction

87% match1 repo172024

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

87% match2025

Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)

87% match1 repo502024

Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models

87% match2025

Cite this paper

@article{mazeika2024harmbench,
  title  = {HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal},
  author = {Mantas Mazeika and Long Phan and Xuwang Yin and Andy Zou and Zifan Wang and Norman Mu and Elham Sakhaee and Nathaniel Li and Steven Basart and Bo Li and David Forsyth and Dan Hendrycks},
  year   = {2024},
  eprint = {2402.04249},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2402.04249v2},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2402.04249.svg)](https://codeofpaper.com/paper/2402.04249)

Discussion