<img src="https://raw.githubusercontent.com/NBPub/BeatLog/main/BeatLog/static/favicon.png" title="BeatLog!"> BeatLog

Overview | Documentation

Background
Features
Installation
- Docker
- Parameters
- Data Sources
- Installation Extras +
Setup
- Setup Guide +
Development
- Feedback
- In Progress
- Pre-Release Notes
  - Detailed Changelog +
- Local Installation +

Background

BeatLog parses NGINX reverse proxy and fail2ban logs into readable tables and reports. Use BeatLog to assess server traffic and tailor fail2ban filters.

Features

BeatLog is a Python based web application, and utilizes a PostgreSQL database for storage. Log files are parsed line-by-line using regex. BeatLog provides default patterns for parsing, but each log's regex scheme can be customized.

In addition to parsing data from log files, it categorizes each IP address as coming from "Home" (same IP as server) or "Outside" (anywhere else). Location information (coordinates, city, country) can be added to all "Outside" entries, using MaxMind's GeoLite2 database (SWAG users, see: mod installation). If only coordinates are provided from the MaxMind database, locations can be ascertained using the Nominatim reverse geocoding API, based on OpenStreetMap data.

See the BeatLog Documentation for a full description of features. The Database, Report, and Visitor Map are briefly highlighted here.

Database

Data is saved in a PostgreSQL database and can be used for your own purposes. See the Processed Data section in the Setup Guide to see the table and field schema used for parsed log data, and the Database Explorer page to see how data can be queried and viewed within BeatLog. A simple JSON API provides daily summaries and bandwidth statistics.

<details><summary>Database Query - fail2ban Log</summary>

dataview

</details>

Adminer can be installed to facilitate interaction with the database.

Report |demo|

A report synthesizes all log data from the previous few days or a custom date range. Charts are integrated using CanvasJS, and Bootstrap is used for tables and styling. Within the report, you can:

Analyze home and outside connections against fail2ban finds, bans, and ignores to assess efficacy of fail2ban filters.
Scrutinize traffic from frequent visitors and monitor popular client requests
Known Devices can be identified and separated from other outside connections

Documentation

Visitor Map |demo|

Visitor locations are plotted onto an interactive map using LeafletJS and OpenStreetMap tiles.

Tool tips show location names and total connections or unique visitors (IPs) over the selected time range
Location marker sizes are scaled by total connections or unique visitors
Tabular data is presented beneath the map

Documentation

Installation

BeatLog docker images are created via workflows with two different tags. "Stable" images are built and pushed with each release, and "Latest" images are built and pushed with each commit. Therefore, the stable or arm32v7-stable tags are recommended, unless there are pending updates that are desired.

Current release: alpha-0.1.7, In development: alpha-0.1.8 | Architecture | Latest Tags | Stable Tags | | :----: | --- | --- | | x86-64 | latest | stable, alpha-0.1.7 | | arm64 | latest | stable, alpha-0.1.7 | | armhf | (built on request) | arm32v7-stable, arm32v7-alpha-0.1.7 |

A PostgreSQL database is required, and can be included in the same docker deployment, as shown below. Or, connect to an existing database, by providing connection settings under environment:.

Logs and other files are added to the container via volumes. See the data sources section for specific files in the mounted directories and their usage. In the example below, the specified directories, and their contents, will be available within a created /import/ directory.

It is important to mount files that may change (log turnover, changed fail2ban settings, MaxMindDB updates) indirectly via their parent directories. Directly mounted files will not update within the container.

The compose parameters are detailed in the next section. Optional healthcheck and adminer additions are shown on the Installation Extras page.

With an existing, connectable database, depends_on: and the following lines are not needed. Ensure a database with the name specified in BeatLog's environment exists.

Docker Compose

---
version: "2.1"
services:
  beatlog:
    image: nbpub/beatlog:stable
    container_name: beatlog
    user: 1000:1000 # optional	
    ports:
      - 5000:8000 # access from 5000 instead of 8000, for demonstration
    environment:
      - TZ=Pacific/Galapagos
      - db_host=<IP>
      - db_password=changeme	  
      - FLASK_SECRET_KEY=<secretkey>
      - check_IP=12
      - check_Log=3	  
    volumes:
      - /path_to/swag_config/log:/import/log # NGINX and fail2ban logs
      - /path_to/swag_config/fail2ban:/import/fail2ban # fail2ban jail.local
      - /path_to/swag_config/geoip2db:/import/geoip2db # MaxMindDB
    depends_on:
      - db
  db:
    image: postgres:15
    container_name: beatlog_db
    ports:
      - 5432:5432
    environment:
      - POSTGRES_USER=beatlog
      - POSTGRES_PASSWORD=changeme
      - TZ=Pacific/Galapagos	  
    volumes:
      - /path_to/beatlog_conifg/db:/var/lib/postgresql/data # recommended
    restart: unless-stopped

Parameters

Container images are configured using parameters passed at runtime (such as those above). These parameters are separated by a colon and indicate <external>:<internal> respectively. For example, setting ports to 5433:5432 would expose port 5432 from inside the container to be accessible from the host's IP on port 5433 outside the container.

Sensitive data can be passed to compose using secrets, if desired.

<details><summary>Docker Compose Parameters</summary>

| Parameter | Function | | :----: | --- | | user | | | 1000:1000 | Optional setting to change the user used for the docker container. See also | | ports | | | 5000:8000 | Example of changing external access port. Internal port, 8000, should not be changed. | | **envir

BeatLog

Install / Use

README