pentest-ai

<div align="center"> <img src="assets/transparentbanner.png" alt="pentest-ai" width="640"> <h1>pentest-ai</h1>

The pentest tool that proves its findings. No oracle, no badge.

Website · Install · Why verification · Docs · Benchmarks · Agents · Discord

</div>

⚠️ Offensive tooling, authorized testing only. By installing you accept the AUP and Terms. Full text in Responsible use ↓

ptai is an AI-driven pentest tool that re-runs every exploit to confirm it. It runs recon, logs in, and chains findings into multi-step attack paths, but it does not ask you to trust the results. The way TruffleHog confirms a leaked secret by logging in with it, ptai confirms a web finding by re-running the exploit: a finding stays a candidate until a machine oracle reproduces it N out of N, and only then does it earn a VERIFIED badge. Third-party scanner output (nuclei, nikto, zap) is held back until an oracle re-proves it. Scanner noise is what trains teams to ignore their tools, so the report carries only what ptai could prove, each VERIFIED finding with a portable proof capsule you can replay yourself.

Today 14 vulnerability classes are oracle-verified. On a deliberately-vulnerable test honeypot, 23 findings verify across those classes at 100% precision with zero false positives. On a stock OWASP Juice Shop, 12 verify in a single scan. Runs on your laptop. No cloud, no telemetry.

See it work

<p align="center"> <img src="assets/ptai-juiceshop.gif" alt="ptai scanning OWASP Juice Shop: findings flip from candidate to oracle-VERIFIED, 12 verified in one scan" width="900"> </p> <p align="center"><em>Scanning a stock OWASP Juice Shop: 12 findings oracle-verified in a single scan. Findings are real; timing is paced for watchability.</em></p>

Reproduce the core idea yourself in two minutes, no target of your own:

pip install ptai && ptai demo

ptai demo scans a bundled vulnerable app and reports 4 findings, 4 oracle-VERIFIED, replays one live from a proof capsule (replay 3/3), then runs the same routes hardened and reports 0 findings. The only thing that changed between the two runs is the fix, so the findings appear and disappear with the vulnerability, not because the tool went quiet. Two minutes, no API key, no target of your own. Re-prove any capsule yourself with ptai replay.

Honest numbers. The honeypot run (23 verified across 14 classes, 100% precision, zero false positives) and the Juice Shop run (12 verified in one scan) are individual reproducible benchmarks, not field false-positive rates. The oracle gate buys precision, not catch rate: it removes false positives, it does not raise detection. Juice Shop is the most-studied vulnerable app on the internet, so read its raw volume as breadth and the verified count as the precision story; the honeypot, with bugs we wrote ourselves, is the honest signal. The honeypot harness (tests/honeypot/) and a clean-app zero-FP gate (tests/cleanapp/) ship in the repo, so the claims are reproducible rather than screenshots.

What's new in 1.1.0

Verification coverage roughly doubled, and a scan no longer reports zero on a target it knocked over mid-run. Every VERIFIED finding comes from a named machine oracle, never an LLM assertion, enforced in code: a verdict that cannot name its oracle is rejected. This release adds:

• Ten new oracle classes (14 total). Trusted-header bypass, JWT alg:none, host-header poisoning, XXE, type confusion, stored XSS, sequential IDOR, mass assignment, non-blind SSRF, and SQLi login-bypass, joining SQLi (boolean/blind), BOLA/IDOR, reflected XSS, open redirect, and path traversal. Each oracle has a control that must fail on a safe target, so a non-vulnerable app abstains instead of earning a badge.
• Verification resilience. An aggressive sweep could knock a fragile single-container target over, after which the verify phase failed every oracle and reported 0 despite valid, replayable recipes. It now waits for the target to answer again before re-proving, which took an OWASP Juice Shop scan from 0 to 12 oracle-verified.
• Scope safety. Active tools (sqlmap, dalfox) are host-locked to the engagement target; the scan no longer feeds third-party URLs scraped from a page's content to attack tools.
• Portable proof capsules with ptai replay, a live TUI that flips verdicts to VERIFIED on screen, and a CI gate (--fail-on verified) that breaks a build only on proven findings.

On a real target: OWASP Juice Shop

Pointed at a stock OWASP Juice Shop, ptai oracle-verifies 12 findings in a single scan: JWT alg:none accepted on protected endpoints, BOLA cross-user reads, sequential IDOR, and type confusion, each re-proven by a machine oracle, not asserted. It detects more than it verifies (SQLi auth-bypass on /rest/user/login, UNION SQLi on /rest/products/search, XXE disclosing /etc/passwd, mass assignment, password-reset bypass); only the verified subset reaches the report. Drive it through Claude Code over MCP with no API key, or standalone.

Honesty caveat. Juice Shop is the most-documented vulnerable app on the internet, so the LLM and the probe authors both have a head start. Against a novel target the catch rate is whatever the curated probe library covers (60+ web probes today, growing each release); the LLM coordinates and reasons about results, it doesn't replace the probes. A private honeypot harness in tests/honeypot/ measures coverage against bugs we wrote ourselves and is asserted in CI (tests/honeypot/test_mcp_honeypot_e2e.py); its numbers are lower than Juice Shop, and that's the point. We publish both. See the full Juice Shop benchmark vs ZAP / Nuclei / HexStrike.

Install

pip install ptai

Path 1: Drive it from Claude Code (no API key)

If you already pay for Claude Pro / Max / Team, your subscription IS the LLM. Wire ptai in as an MCP server:

claude mcp add pentest-ai -- ptai mcp

Restart Claude Code, then ask:

"Run an authenticated pentest against staging.acme.com. Login is at /login, password is in $APP_PASS."

What hits the network: ptai's tools and probes execute locally against your target. Your prompts and the tool output that Claude Code reads go through Anthropic's API, same as any Claude Code session. If you need an air-gapped path, see Path 3 (Ollama / on-prem LLM).

Claude Code drives ptai via these MCP tools (47 of them today):

• list_tools / run_tool: list and invoke any of 200+ wrapped security tools
• plan_tools / ensure_tools_installed: get the canonical tool list for an engagement, batched install
• list_probes / run_probe: 60 SPA-aware probes for OWASP Top 10 bug classes
• http_request: raw HTTP under a hard scope guard for novel chains
• start_engagement / get_findings / get_attack_chains: the engagement record
• plus test_web_app, test_active_directory, test_cloud, test_api_security, and the rest

Path 2: Other MCP clients (Cursor, VS Code Copilot, Codex, Claude Desktop)

ptai setup --mcp

Auto-detects every MCP-compatible client you have installed and writes their config files. Restart the client and the same 47 tools are there.

Path 3: Standalone CLI when you DON'T have an MCP client

If you're using Claude Code, Cursor, Codex, or Claude Desktop, use Path 1 or 2 above and skip this section. No API key needed there.

Path 3 is for CI/CD pipelines, scheduled cron jobs, air-gapped terminals, and users without an MCP client. The standalone CLI has no LLM of its own, so you bring one via env var:

export ANTHROPIC_API_KEY=sk-ant-...           # Claude (best results)
# or
export OPENAI_API_KEY=sk-...                  # OpenAI
# or, fully local, no cloud
export PENTEST_AI_LLM_PROVIDER=ollama         # Ollama (default localhost:11434)
# or, any of 300+ models via LiteLLM (OpenRouter, Azure, DeepSeek, Groq, Mistral, ...)
pip install litellm

ptai start https://your-target.com

Hitting an OpenAI-compatible endpoint (DeepSeek cloud, Groq, Together AI, vLLM, etc.)? Set OPENAI_BASE_URL + PENTEST_AI_MODEL and use the openai provider. Full recipes for every provider - including custom model names, troubleshooting, and the LiteLLM-300+ list - live in docs/llm-providers.md.

Spending cap (Path 3 only)

The standalone agent loop drives its own LLM, so runaway loops cost real money. ptai caps spend per engagement at $10 USD by default. A normal Sonnet 4.6 web-app sweep with prompt caching finishes well under that; an Opus 4.7 deep run can blow past it.

Change it via env var (no CLI flag - env var is the only knob):

export PTAI_PRICE_LIMIT=25        # raise to $25
export PTAI_PRICE_LIMIT=0         # unlimited (logs a warning)
unset PTAI_PRICE_LIMIT            # back to the $10 default

If the cap fires mid-engagement, the engagement is marked aborted_cost_limit and its checkpoint is preserved. Raise the cap and resume from where it stopped:

export PTAI_PRICE_LIMIT=25
ptai resume <engagement_id>

Paths 1 and 2 (MCP) don't use this cap - your AI client (Claude Code, Cursor, etc.) handles its own LLM billing.

Installing security tools

ptai wraps 200+ external tools. Three ways to get them on the box:

# 1. Zero-config (recommended). At engagement start, the planner predicts
#    which tools the LLM will need and asks ONCE to install the missing
#    ones. Decline once and the answer persists in
#    ~/.pentest-ai/install-preferences.json.
ptai start https://target.example.com

# 2. Batch install upfront. Skips the engagement-time prompt entirely.
ptai setup --tier core            # ~6 essentials, ~30s
ptai setup --tier recommended     # + fuzzers, crawlers, password tools, ~5m
ptai setup --tier full            # everything, ~30m

# 3. Install specific tools by name.
ptai setup --per-tool wpscan,dalfox,paramspider
ptai setup --wizard               # interactive picker

In non-interactive contexts (PTAI_NON_INTERACTIVE=1 or no TTY) ptai uses what's on PATH and logs (rather than prompts) for anything missing.

<details> <summary><strong>Other paths</strong>: REST API, MCP composition, HITL teleoperation, cloud workspace, public benchmarks</summary>

HTTP REST API (for dashboards and integrations)

pip install ptai[api]
ptai serve --port 8888

Endpoints: /health, /version, /agents, /tools, /engagements (list, detail, findings, chains, detection rules, SARIF export). Write endpoints (POST /engagements, POST /engagements/{id}/abort) require Authorization: Bearer $PENTEST_AI_API_TOKEN. Live event stream at `WS /engageme