Grinder Framework

:mag_right: Internet-connected Devices Census Python Framework

Screenshot

<div align="center"> <img src="https://raw.githubusercontent.com/sdnewhop/grinder/master/docs/images/screenshot.png" alt="Grinder Framework Interface"> <p align="center"><i>The basic CLI interface of the Grinder Framework</i></p> </div>

Table of Contents

1. Description
2. Slides
3. Grinder Workflow
4. Grinder Map
   • Screenshots
   • Description
5. Requirements
   • Legend
   • :pushpin: Basic
   • :pushpin: Accounts
   • Additional scanning
   • TLS configuration
6. Current Features
   • Already Implemented
   • :fire: Additional Modules
   • Development and Future Updates
7. :pushpin: Grinder Installing
   • Setup and Configure Environment
   • Running Grinder Map Server Locally
8. Building and Running in Docker
   • Description
   • Services and Images
   • Provided Scripts and Preparations
   • Environment
   • :pushpin: Building
   • :pushpin: Running
9. Tests
10. CLI Interface
    • Help on Command Line Arguments
11. Wiki
12. :pushpin: Usage Examples
    • Show Help
    • Basic Enumeration
    • Enumeration with Limited Results
    • Enumeration with Nmap Scanning
    • Enumeration with Additional Analytics, Map and Plots
    • Enumeration with Analytics from Vulners
    • Enumeration with TLS Configuration and Attacks Scanning
    • Enumeration with Additional Filtering
    • Enumeration with Additional Custom Scripts
    • Enumeration with Additional Debug Information
13. :pushpin: Add Your Own Queries
    • Description
    • Queries Template
    • Queries Example

Description

The Grinder framework was created to automatically enumerate and fingerprint different hosts on the Internet using various back-end systems: search engines (such as Shodan or Censys) for discovering hosts and NMAP engine for fingerprinting and specific checks. Also, Grinder supports Vulners API to get information about available public exploits and vulnerabilities, documents related to found vulnerabilities and other features.

The Grinder framework can be used in many different areas of research, as a connected Python module in your project or as an independent ready-to-use from the box tool.

Slides

1. One framework to rule them all: a framework for Internet-connected device census. PHDays 2019. (Talk Page, Slides)
2. One Framework to rule them all: A framework for Internet-connected Device Census. OFFZONE 2019. (Talk Page, Slides)

Grinder Workflow

Grinder Map

Screenshots

<div align="center"> <img src="https://raw.githubusercontent.com/sdnewhop/grinder/master/docs/images/map_1.png" alt="Grinder Framework Map (1)"> <p align="center"><i>The Grinder Framework can easily build an interactive map with found hosts in your browser</i></p> <img src="https://raw.githubusercontent.com/sdnewhop/grinder/master/docs/images/map_2.png" alt="Grinder Framework Map (2)"> <p align="center"><i>Also, the Grinder Framework can show you some basic information</i></p> <img src="https://raw.githubusercontent.com/sdnewhop/grinder/master/docs/images/map_3.png" alt="Grinder Framework Map (3)"> <p align="center"><i>...And some additional information</i></p> </div>

Description

To visualize gained data, the Grinder Framework provides an interactive world map with all results. Grinder map back-end that was written in Flask supports additional REST API methods to get more information about all scanned hosts or some particular host from the map, also it is possible to show some additional information about host interactively from the map.

For example, the hosts will be automatically checked for availability with ping from back-end, also for every host many additional features are available: current host can be directly opened in Shodan, Censys, and ZoomEye web interfaces, the host can be shown on Google Maps with all available information about geolocation. Also, it is possible to make an IP lookup or open raw information in JSON directly in a browser or from your application with provided API methods.

Requirements

Legend

:heavy_exclamation_mark: required
:heavy_plus_sign: not required to run (or required only for additional modules)

Basic

• :heavy_exclamation_mark: Python 3.6+
• :heavy_exclamation_mark: python3-tk library
• :heavy_exclamation_mark: FreeType library (Python 3.8+)

Accounts

• :heavy_exclamation_mark: Shodan and Censys accounts
  Required to collect hosts, both free and full accounts are suitable. Also, it's possible to use only one account (Censys or Shodan, Shodan is preferable).
• :heavy_plus_sign: Vulners account
  Required to make additional reports on vulnerabilities and exploits. If this feature is not required for you, you can use Grinder without Vulners account.

Additional scanning

• :heavy_plus_sign: Nmap Security Scanner 7.60+
  Version 7.60 and newer has been tested with currently used in Grinder scripts (ssl-cert.nse, vulners.nse, etc.).

TLS configuration

• :heavy_plus_sign: Java 8
  Required to build TLS-Attacker and TLS-Scanner.
• :heavy_plus_sign: TLS-Attacker 3.0
  Required only for TLS scanning.
• :heavy_plus_sign: TLS-Scanner 2.9
  Required only for TLS scanning.

Current Features

Already Implemented

• :mag: Collecting hosts and additional information using Shodan and Censys search engines
• :rocket: Scanning ports and services with boosted multi-processed Nmap Scanner wrapper
• :syringe: Scanning vulnerabilities and additional information about them with Vulners database and Shodan CVEs database
• :memo: Retrieving information about SSL certificates
• :key: Scanning for SSL/TLS configuration and supported cipher suites
• :key: Scanning for SSL/TLS bugs, vulnerabilities and attacks
• :earth_asia: Building an interactive map with information about the hosts found
• :bar_chart: Creating plots and tables based on the collected results
• :wrench: Custom scanning scripts support (in LUA or Python3)
• :chart_with_downwards_trend: Confidence filtering system support
• :chart_with_upwards_trend: Special vendors scanning and filtering support
• :bulb: Searching for documents, security bulletins, public exploits and many more things based on detected by Grinder vulnerabilities and software

Additional Modules

:rocket: Note #1: You can run multiple Python scripts simultaneously per multiple hosts, so you can build your own chain of scripts and checks to get the most information from your hosts. Feel free to add your modules with PR or give us an idea with feature issue.

:construction: Note #2: Multiple NSE scripts running task is still in WIP status. So, NSE scripts will be ran consistently, one after one. New NSE script engine for Grinder is comming up, stay tuned.

DICOM Patient Info Getter

Location: py_scripts/dicom_getter/dicom_getter.py
Description: This module allows you to grab different patient information (including files) from medical servers

HTTP Raw Response Grabber

Location: py_scripts/http_response_grabber/http_response_grabber.py
Description: This module allows you to grab the HTTP response (headers + body) in decoded raw-bytes format

HTTP Status

Location: py_scripts/http_status/http_status.py
Description: This module allows you to check the HTTP status of the resource

SCP (SSH) Grabber

Location: py_scripts/scp_grabber/scp_grabber.py
Description: This module allows you to grab and download files from d