A Reinforcement-Learning Agent for Web Penetration Testing

A purpose-built vulnerable web application and a reinforcement-learning agent that learns — through trial and error, the paradigm behind game-playing AI — to discover all three planted vulnerabilities. The trained agent outperforms a random baseline sixfold and generates a structured findings report.

Watch the agent run →
A safe replay of the learned attack path — nothing runs live.

Defensive extension: JADEPUFFER detector →
Detects the signatures of agentic (AI-driven) database ransomware.

Summary

Method

Penetration testing follows a natural sequence — reconnaissance, then exploitation — which is modeled here as a Markov decision process. At each step the agent selects one of twelve actions: crawl the site, attempt a SQL-injection login, attempt an XSS search, request another user's profile, or one of several unproductive actions (incorrect passwords, ordinary searches, missing pages). It receives a reward of +10 whenever an action empirically confirms a new vulnerability, a small +2 for performing reconnaissance first, and −1 per step, so shorter, more efficient attack paths score higher.

Critically, the agent is not told which actions are productive. Using Q-learning, it maintains a table of the value of each action in each state, explores (randomly at first), and updates that table from the rewards received. Over a few hundred episodes the table converges to an effective strategy.

The evaluation is grounded in real behavior: every action is an actual HTTP request to the live application, and a vulnerability is counted only when the real response confirms it — a login page returning "Welcome" without a valid password, a search page reflecting an injected <script> tag, or a profile page disclosing another user's private record.

Training progression

The chart tracks the agent's mean episode reward across training. It begins in the negatives — exploring ineffectively and reaching the step limit without significant findings — and rises to approximately +28, the reward of the efficient four-step attack. The dashed line marks the performance of a purely random agent for comparison.

Trained agent vs. random baseline

This comparison is the central justification for the reinforcement-learning approach. Once trained, the agent discovers all three vulnerabilities in 4 actions, on every run. A random agent — drawing from the same twelve actions — requires roughly 25 actions on average and discovers all three in only half of its sessions. Learning converts undirected exploration into an efficient, repeatable strategy.

Automated reporting

On completion, the agent generates a structured penetration-testing report: each finding with its severity, OWASP category, impact, and remediation (parameterized queries, output escaping, proper authorization checks). The report is currently template-generated, with a clearly defined integration point where a language model would produce the narrative — a planned v2 enhancement.

Limitations and future work (v2)

This is a compact sandbox: three known vulnerabilities, twelve actions, one application — appropriately sized to understand reinforcement learning end to end, but an educational model rather than a production scanner. The first v2 component is already published: a defensive detector for AI-driven database ransomware (JADEPUFFER), which pivots the project from offense to defense. Subsequent work includes additional vulnerability classes, a second application to evaluate generalization, and integrating a language model into the reporting and payload-generation stages.