Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot

Antonio López Martínez, Alejandro Cano, Antonio Ruiz-Martínez
1/12/2025
cs.CRcs.AI

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

The advent of Generative Artificial Intelligence (GenAI) has brought a significant change to our society. GenAI can be applied across numerous fields, with particular relevance in cybersecurity. Among the various areas of application, its use in penetration testing (pentesting) or ethical hacking processes is of special interest. In this paper, we have analyzed the potential of leading generic-purpose GenAI tools-Claude Opus, GPT-4 from ChatGPT, and Copilot-in augmenting the penetration testing process as defined by the Penetration Testing Execution Standard (PTES). Our analysis involved evaluating each tool across all PTES phases within a controlled virtualized environment. The findings reveal that, while these tools cannot fully automate the pentesting process, they provide substantial support by enhancing efficiency and effectiveness in specific tasks. Notably, all tools demonstrated utility; however, Claude Opus consistently outperformed the others in our experimental scenarios.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Hacking, The Lazy Way: LLM Augmented Pentesting

91% match2024

Breaking Android with AI: A Deep Dive into LLM-Powered Exploitation

87% match2025

Towards Automated Penetration Testing: Introducing LLM Benchmark, Analysis, and Improvements

86% match1 repo232024

AI-Augmented Ethical Hacking: A Practical Examination of Manual Exploitation and Privilege Escalation in Linux Environments

86% match2024

CIPHER: Cybersecurity Intelligent Penetration-testing Helper for Ethical Researcher

86% match1 repo402024

AI-Enhanced Ethical Hacking: A Linux-Focused Experiment

86% match2024

Cite this paper

@article{martnez2025generative,
  title  = {Generative Artificial Intelligence-Supported Pentesting: A Comparison between Claude Opus, GPT-4, and Copilot},
  author = {Antonio López Martínez and Alejandro Cano and Antonio Ruiz-Martínez},
  year   = {2025},
  eprint = {2501.06963},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2501.06963v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2501.06963.svg)](https://codeofpaper.com/paper/2501.06963)

Discussion