YAPP - Yet Another Pentest Parser

One parser to rule them all

A powerful Python library and CLI tool for parsing and processing multiple pentesting tool outputs.

YAPP is a comprehensive solution for parsing pentesting tool outputs (Nessus, Nmap, and Burp-soon) into structured JSON, with advanced consolidation capabilities, excel outputting, and both programmatic and command-line interfaces. Built as an extensible framework with modularity, efficiency, and ease of use in mind.

🎯 Why YAPP Exists

The pentesting industry has a multi-faceted tooling problem (CAPDEV). Most businesses treat capabilities development as an afterthought - they'd rather hire more people to combat performance and resource issues, rather than fix their piss-poor workflows and capabiltiies.

• Why? 💰C.R.E.A.M.💰
  • Unbilled consultant = cost to business
    • This is a bad mindset as business teaches us that you need to spend money to make money.
• Result?
  • Consultants working unpaid overtime because basic data processing eats half their day
  • Delays, lots of them. These delays then cause a snowball effect where the next engagement is impacted.
  • Cutting corners to meet deadlines.
  • Burnout & general negative vibes.

<img width="536" height="470" alt="image" src="https://github.com/user-attachments/assets/a0f96d93-f908-42f6-85a3-deed0b37c6a5" />

✨ Features

🔧 Multi-Tool Support

• Nessus XML: Full vulnerability parsing with consolidation and API formatting
• Nmap XML: Service discovery with port filtering, flat JSON output, and a comparison spreadsheet between two scans
• Extensible Framework: Easy to add new parsers following established patterns
• Auto-Detection: Automatically identifies file types

🖥️ Dual Interface Design

• CLI Tool: Beautiful command-line interface with colored output and tool-specific options
• Python Library: Clean programmatic API for integration into your projects
• In-Memory Processing: Parse raw XML strings directly — no filesystem required (ideal for DB/API integration via process_data())
• One External Dependency: It used to be 0 deps but openpyxl is needed for xlsx

📊 Advanced Nessus Processing

• Parse Nessus XML files into structured JSON/Python dictionaries
• Advanced consolidation engine with smart vulnerability grouping
• Plugin output pattern matching and filtering
• Rule-based vulnerability categorization
• API-ready output formatting with entity limiting
• Inline Excel report generation (-x flag) — no more two-step workflow
• Combined single-file output (-sf flag) — all selected outputs in one JSON

🗺️ Comprehensive Nmap Support

• Parse Nmap XML into structured format with service details
• Port status filtering (open, closed, filtered)
• Flat JSON output for legacy tool compatibility
• Service enumeration and script output capture
• Compare two Nmap XML files and output differences into spreadsheet

🎯 Intelligence & Analytics

• Track vulnerabilities globally and per host (Nessus)
• Comprehensive statistics and metrics for both tools
• Multiple FQDN support per host
• Detailed vulnerability information (CVE, CVSS, affected systems)
• Human-readable output with plugin/service names
• Consolidation rule logging -- easily debug consolidation rules by seeing what hasn't matched and why

⚡ Performance

Benchmark Results (Nessus):

• File Size: 118 MB Nessus XML (1005 hosts, 214 findings, 17 remediations)
• Processing Time: 9.18 seconds total (5.76s processing + 0.45s I/O)
• Throughput: ~13 MB/second (total) / ~20 MB/second (processing only)
• Memory Efficient: Low memory footprint with streaming parser
• Includes: Full parsing + consolidation engine + API formatting + JSON output (3 files)

Tested on: WSL2 (Debian) on Windows host

🚀 Installation

For CLI Usage Only

# Install globally with pipx (recommended for CLI-only usage)
git clone https://github.com/FlyingPhish/YetAnotherPentestParser && cd YetAnotherPentestParser
pipx install .

OR

pipx install git+https://github.com/FlyingPhish/YetAnotherPentestParser.git
# pipx install git+https://github.com/FlyingPhish/YetAnotherPentestParser.git@branch

For Programmatic Usage

# In your virtual environment
pip install git+https://github.com/FlyingPhish/YetAnotherPentestParser.git
# pip install git+https://github.com/FlyingPhish/YetAnotherPentestParser.git@branch

Upgrading

# When installed using pipx
pipx upgrade yapp

# When installed using pip
pip install git+https://github.com/FlyingPhish/YetAnotherPentestParser.git --force-reinstall

💡 Usage

🖥️ Command Line Interface

usage: yapp [-h] [--version] {parse,excel,compare} ...

YAPP - Swiss Army Knife for Pentester File Processing

positional arguments:
  {parse,excel,compare}  Available commands
    parse                Parse and process pentesting files (Nessus/Nmap/JSON)
    excel                Generate Excel report from YAPP JSON output
    compare              Compare two Nmap XML scans

options:
  -h, --help       show this help message and exit
  --version        show program's version number and exit

🖥️ Command Line Interface - Parse

yapp parse -h

options:
  -h, --help            show this help message and exit
  -i, --input-file INPUT_FILE
                        Path to input file (Nessus .nessus, Nmap .xml, Consolidated JSON)
  -t, --file-type {auto,nessus,nmap,consolidated_json}
                        Input file type (default: auto-detect)
  -of, --output-folder OUTPUT_FOLDER
                        Output folder path (default: ./output)
  -on, --output-name OUTPUT_NAME
                        Output base name (default: timestamp_<original-name>)
  --no-output           Skip writing files, only display results

Output options:
  -sf, --single-file    Write all selected outputs into one combined JSON file
                        instead of separate files

Nessus options:
  -c, --consolidate     Generate consolidated findings file
  -a, --api-output      Generate API-ready JSON (requires --consolidate)
  -r, --rules-file RULES_FILE
                        Custom consolidation rules file
  -el, --entity-limit ENTITY_LIMIT
                        Max entities per API finding
  --log-exclusions      Enable detailed exclusion logging

Nmap options:
  -s, --port-status {all,open,closed,filtered}
                        Filter by port status (default: all)
  -fj, --flat-json      Generate flat JSON format

Excel options:
  -x, --excel           Also generate Excel report (Nessus: requires -c;
                        or consolidated JSON input)

🖥️ Command Line Interface - Excel

yapp excel -h

options:
  -h, --help            show this help message and exit
  -i, --input-file INPUT_FILE
                        Path to YAPP JSON file (consolidated, combined, or parsed)
  -of, --output-folder OUTPUT_FOLDER
                        Output folder path (default: ./output)
  -on, --output-name OUTPUT_NAME
                        Custom output filename (without extension)

🖥️ Command Line Interface - Compare

yapp compare -h

options:
  -h, --help            show this help message and exit
  -ff, --first-file FIRST_FILE
                        Path to first Nmap XML file
  -lf, --last-file LAST_FILE
                        Path to second Nmap XML file
  -of, --output-folder OUTPUT_FOLDER
                        Output folder path (default: ./output)
  -on, --output-name OUTPUT_NAME
                        Custom output filename (without extension)

🔬 Nessus Consolidation Engine

The consolidation engine intelligently groups related vulnerabilities, reducing noise and improving vulnerability management efficiency.

Features:

• Smart Pattern Matching: Regex patterns for vulnerability names and plugin output
• Plugin Output Filtering: Search actual Nessus plugin output content
• Flexible Grouping: Group by IP, port, service, or custom criteria
• Rule-Based Configuration: JSON rules for different vulnerability types
• Advanced Logic: AND/OR pattern matching, exclusion rules
• Entity Limiting: Control API output size with configurable entity limits

Common Consolidation Rules:

• Outdated Software: Group software with version update patterns
• Certificate Issues: Consolidate SSL/TLS certificate problems
• Weak Encryption: Group protocol and cipher vulnerabilities
• JavaScript Libraries: Separate web application library issues
• Operating System: Group OS-specific updates and patches

📊 Excel Report Generation

Transform consolidated vulnerability data into structured Excel workbooks for easy analysis and validation.

<img width="689" height="555" alt="Excel Output CLI" src="https://github.com/user-attachments/assets/4c5b380d-4e0d-4c0a-b3e7-ce957fcc14de" />

Features:

• Matrix Layout: One worksheet per vulnerability with Yes/No plugin indicators
• Service-Level Detail: Each row shows FQDN, IP, Port, and which plugins affected it
• Consolidation Validation: Quickly verify which plugins were grouped together
• Analyst-Friendly Format: Familiar spreadsheet format for review and sign-off
• Consistent Naming: Output filename follows the unified {base}_Report.xlsx convention

Workflow:

Option A — Inline (single command):

yapp parse -i scan.nessus -c -x          # Parse, consolidate, and generate Excel in one shot
yapp parse -i scan.nessus -c -a -x       # Full pipeline with API output + Excel
yapp parse -i scan.nessus -c -a -x -sf   # Everything above, combined into one JSON + Excel

**Option B — Standalone (from existing JSON)