Augustus - LLM Vulnerability Scanner

Test large language models against 210+ adversarial attacks covering prompt injection, jailbreaks, encoding exploits, and data extraction.

Augustus is a Go-based LLM vulnerability scanner for security professionals. It tests large language models against a wide range of adversarial attacks, integrates with 28 LLM providers, and produces actionable vulnerability reports.

Unlike research-oriented tools, Augustus is built for production security testing — concurrent scanning, rate limiting, retry logic, and timeout handling come out of the box.

Why Augustus
Features
Quick Start
Supported Providers
Usage
How It Works
Architecture
Configuration
FAQ
Troubleshooting
Contributing
Security
Support
License

Why Augustus

| Feature | Augustus | garak | promptfoo | |---------|----------|-------|-----------| | Language | Go | Python | TypeScript | | Single binary | Yes | No | No | | Concurrent scanning | Goroutine pools | Multiprocessing pools | Yes | | LLM providers | 28 | 35+ | 80+ | | Probe types | 210+ | 160+ | 119 plugins + 36 strategies | | Enterprise focus | Yes | Research | Yes |

Features

| Feature | Description | |---------|-------------| | 210+ Vulnerability Probes | 47 attack categories: jailbreaks, prompt injection, adversarial examples, data extraction, safety benchmarks, agent attacks, and more | | 28 LLM Providers | OpenAI, Anthropic, Azure, Bedrock, Vertex AI, Ollama, and 22 more with 43 generator variants | | 90+ Detectors | Pattern matching, LLM-as-a-judge, HarmJudge (arXiv:2511.15304), Perspective API, unsafe content detection | | 7 Buff Transformations | Encoding, paraphrase, poetry (5 formats, 3 strategies), low-resource language translation, case transforms | | Flexible Output | Table, JSON, JSONL, and HTML report formats | | Production Ready | Concurrent scanning, rate limiting, retry logic, timeout handling | | Single Binary | Go-based tool compiles to one portable executable | | Extensible | Plugin-style registration via Go init() functions |

Attack Categories

Jailbreak attacks: DAN, DAN 11.0, AIM, AntiGPT, Grandma, ArtPrompts
Prompt injection: Encoding (Base64, ROT13, Morse), Tag smuggling, FlipAttack, Prefix/Suffix injection
Adversarial examples: GCG, PAIR, AutoDAN, TAP (Tree of Attack Prompts), TreeSearch, DRA
Multi-turn attacks: Crescendo (gradual escalation), GOAT (adaptive technique switching)
Data extraction: API key leakage, Package hallucination, PII extraction, LeakReplay
Context manipulation: RAG poisoning, Context overflow, Multimodal attacks, Continuation, Divergence
Format exploits: Markdown injection, YAML/JSON parsing attacks, ANSI escape, Web injection (XSS)
Evasion techniques: Obfuscation, Character substitution, Translation-based attacks, Phrasing, ObscurePrompt
Safety benchmarks: DoNotAnswer, RealToxicityPrompts, Snowball, LMRC
Agent attacks: Multi-agent manipulation, Browsing exploits
Security testing: Guardrail bypass, AV/spam scanning, Exploitation (SQLi, code exec), BadChars

Warning: The lmrc probe uses profane and offensive language as part of its jailbreak testing. Use only in authorized testing environments.

Quick Start

Installation

Requires Go 1.25.3 or later.

go install github.com/praetorian-inc/augustus/cmd/augustus@latest

Or build from source:

git clone https://github.com/praetorian-inc/augustus.git
cd augustus
make build

Basic Usage

export OPENAI_API_KEY="your-api-key"
augustus scan openai.OpenAI \
  --probe dan.Dan_11_0 \
  --detector dan.DAN \
  --verbose

Example Output

+--------------+-------------+--------+-------+--------+
| PROBE        | DETECTOR    | PASSED | SCORE | STATUS |
+--------------+-------------+--------+-------+--------+
| dan.Dan_11_0 | dan.DAN     | false  | 0.85  | VULN   |
| dan.STAN     | dan.STAN    | true   | 0.10  | SAFE   |
| dan.AntiDAN  | dan.AntiDAN | true   | 0.05  | SAFE   |
+--------------+-------------+--------+-------+--------+

List Available Capabilities

# List all registered probes, detectors, generators, harnesses, and buffs
augustus list

Supported Providers

Augustus includes 28 LLM provider categories with 43 generator variants:

| Provider | Generator Name(s) | Notes |--------------------|---------------------------|- | OpenAI | openai.OpenAI, | Anthropic | anthropic.Anthropic | Azure OpenAI | azure.AzureOpenAI | AWS Bedrock | bedrock.Bedrock | Google Vertex AI | vertex.Vertex | Cohere | cohere.Cohere | Replicate | replicate.Replicate | HuggingFace | huggingface.InferenceAPI, | Together AI | together.Together | Anyscale | anyscale.Anyscale | Groq | groq.Groq | Mistral | mistral.Mistral | Fireworks | fireworks.Fireworks | DeepInfra | deepinfra.DeepInfra | NVIDIA NIM | nim.NIM, | NVIDIA NeMo | nemo.NeMo | NVIDIA NVCF | nvcf.NvcfChat, | NeMo Guardrails | guardrails.NeMoGuardrails | IBM watsonx | watsonx.WatsonX | LangChain | langchain.LangChain | LangChain Serve | langchain_serve.LangCh | Rasa | rasa.RasaRest | GGML | ggml.Ggml | Function | function.Single, | Ollama | ollama.Ollama, | LiteLLM | litellm.LiteLLM | REST API | rest.Rest | Test | test.Blank, | -------------------------------| openai.OpenAIReasoning | GPT-3.5, GPT-4, GPT-4 Turbo, o1/o3 reasoning models | | Claude 3/3.5/4 (Opus, Sonnet, Haiku) | | Azure-hosted OpenAI models | | Claude, Llama, Titan models | | PaLM, Gemini models | | Command, Command R models | | Cloud-hosted open models | huggingface.InferenceEndpoint, huggingface.Pipeline, huggingface.LLaVA | HF Inference API, endpoints, pipelines, multimodal | | Fast inference for OSS models | | Llama and Mistral hosting | | Ultra-fast LPU inference | | Mistral API models | | Production inference platform | | Serverless GPU inference | nim.NVOpenAICompletion, nim.NVMultimodal, nim.Vision | NVIDIA AI endpoints, multimodal | | NVIDIA NeMo framework | nvcf.NvcfCompletion | NVIDIA Cloud Functions | | NVIDIA NeMo Guardrails | | IBM watsonx.ai platform | | LangChain LLM wrapper | ainServe | LangChain Serve endpoints | | Rasa conversational AI | | GGML local model inference | function.Multiple | Custom function generators | ollama.OllamaChat | Local model hosting | | Unified API proxy | | Custom REST endpoints (SSE support) | test.Repeat, test.Lipsum, test.Nones, test.Single, test.BlankVision | Testing and development |

All providers are available in the compiled binary. Configure via environment variables or YAML configuration files. See Configuration for setup details.

Usage

Single Probe

# Test for DAN jailbreak
augustus scan openai.OpenAI \
  --probe dan.Dan_11_0 \
  --detector dan.DAN \
  --config-file config.yaml \
  --verbose

Multiple Probes

# Use glob patterns to run related probes
augustus scan openai.OpenAI \
  --probes-glob "dan.*,goodside.*,grandma.*" \
  --detectors-glob "*" \
  --config-file config.yaml \
  --output batch-results.jsonl

# Run all probes against Claude
augustus scan anthropic.Anthropic \
  --all \
  --config '{"model":"claude-3-opus-20240229"}' \
  --timeout 60m \
  --output comprehensive-scan.jsonl \
  --html comprehensive-report.html

Buff Transformations

Apply prompt transformations to test evasion techniques:

# Apply base64 encoding buff to all probes
augustus scan openai.OpenAI \
  --all \
  --buff encoding.Base64 \
  --config '{"model":"gpt-4"}'

# Apply poetry transformation
augustus scan anthropic.Anthropic \
  --probes-glob "dan.*" \
  --buff poetry.MetaPrompt \
  --config '{"model":"claude-3-opus-20240229"}'

# Chain multiple buffs
augustus scan openai.OpenAI \
  --all \
  --buffs-glob "encoding.*,paraphrase.*" \
  --output buffed-results.jsonl

Output Formats

# Table format (default) - human-readable
augustus scan openai.OpenAI --probe dan.Dan_11_0

Augustus

Install / Use

README