Teams of LLM Agents can Exploit Zero-Day Vulnerabilities

Yuxuan Zhu, Antony Kellermann, Akul Gupta, Philip Li, Richard Fang +2 more
6/2/2024
cs.MAcs.AI

No official code found. Best implementation: uiuc-kang-lab/HPTSA (12 stars, last commit 2026-02-23).

Abstract

LLM agents have become increasingly sophisticated, especially in the realm of cybersecurity. Researchers have shown that LLM agents can exploit real-world vulnerabilities when given a description of the vulnerability and toy capture-the-flag problems. However, these agents still perform poorly on real-world vulnerabilities that are unknown to the agent ahead of time (zero-day vulnerabilities). In this work, we show that teams of LLM agents can exploit real-world, zero-day vulnerabilities. Prior agents struggle with exploring many different vulnerabilities and long-range planning when used alone. To resolve this, we introduce HPTSA, a system of agents with a planning agent that can launch subagents. The planning agent explores the system and determines which subagents to call, resolving long-term planning issues when trying different vulnerabilities. We construct a benchmark of 14 real-world vulnerabilities and show that our team of agents improve over prior agent frameworks by up to 4.3X.

View on arXivView PDF

Code Implementations(1)

uiuc-kang-lab/HPTSACommunity71%

Teams of LLM Agents can Exploit Zero-Day Vulnerabilities

120Python, DockerfileApr 11, 20258 months ago

Similar Papers

CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilities

87% match1 repo2822026

Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents

85% match1 repo3052026

Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents

85% match1 repo3052024

OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety

85% match2025

STRIATUM-CTF: A Protocol-Driven Agentic Framework for General-Purpose CTF Solving

84% match2026

The Dark Side of LLMs: Agent-based Attacks for Complete Computer Takeover

84% match2025

Cite this paper

@article{zhu2024teams,
  title  = {Teams of LLM Agents can Exploit Zero-Day Vulnerabilities},
  author = {Yuxuan Zhu and Antony Kellermann and Akul Gupta and Philip Li and Richard Fang and Rohan Bindu and Daniel Kang},
  year   = {2024},
  eprint = {2406.01637},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.01637v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.01637.svg)](https://codeofpaper.com/paper/2406.01637)

Discussion