</div> # LogUtil - Real-time Nginx Log Analyzer <div align="center"> <a href="https://crates.io/crates/logutil" target="_blank"> <img src="https://img.shields.io/crates/v/logutil?style=for-the-badge" alt="crates.io version"> </a> <a href="https://crates.io/crates/logutil" target="_blank"> <img src="https://img.shields.io/crates/d/logutil?style=for-the-badge" alt="crates.io downloads"> </a> <a href="https://github.com/s00d/logutil/blob/master/LICENSE" target="_blank"> <img src="https://img.shields.io/crates/l/logutil?style=for-the-badge" alt="crates.io license"> </a> <a href="https://github.com/s00d/logutil" target="_blank"> <img src="https://img.shields.io/github/stars/s00d/logutil?style=for-the-badge" alt="GitHub stars"> </a> </div>

Overview

LogUtil is a powerful, real-time log analysis tool written in Rust that provides an interactive terminal interface for monitoring and analyzing Nginx access logs. It offers comprehensive log parsing, real-time monitoring, and detailed analytics with a beautiful TUI (Terminal User Interface).

LogUtil in action

Key Features

🔍 Real-time Log Monitoring

Live tail functionality that monitors log files as they grow
Automatic detection and processing of new log entries
Real-time updates without manual refresh

📊 Comprehensive Analytics

IP Address Analysis: Track requests by IP addresses with detailed statistics
URL Analysis: Monitor most accessed URLs and their patterns
Request Type Tracking: Categorize requests by HTTP methods (GET, POST, etc.)
Domain Analysis: Track requests by domain names
Time-based Analytics: Analyze request patterns over time

🎨 Interactive TUI Interface

9 Main Tabs: Overview, Requests, Detailed, Sparkline, Heatmap, Security, Performance, Errors, and Bots
Overview Tab: Shows top IPs and URLs with real-time statistics
Requests Tab: Searchable log entries with pagination
Detailed Tab: Drill-down view for specific IP addresses
Sparkline Tab: Real-time request timeline visualization
Heatmap Tab: Hourly request patterns across multiple days
Security Tab: Detection of suspicious IPs and attack patterns
Performance Tab: Response time analysis and slow request identification
Errors Tab: HTTP error code analysis and categorization
Bots Tab: Bot and crawler detection and classification

⚡ Performance Optimizations

Efficient memory management with automatic cleanup
Optimized for large log files (handles 10,000+ entries)
Minimal CPU usage during real-time monitoring
Configurable cleanup policies

🔧 Flexible Configuration

Customizable regex patterns for different log formats
Support for various date formats
Configurable top N entries display
Optional automatic cleanup of outdated entries

📁 Interactive File Selection Mode

When you run LogUtil without specifying a log file, it launches an interactive file selection mode:

File Selector Features:

📂 Directory Navigation: Browse through folders with intuitive navigation
📄 File Browser: View all files with clear icons (📁 for folders, 📄 for files, ⬆️ for parent directory)
🔍 Visual Selection: Highlighted current selection with clear visual feedback
⚡ Quick Access: Navigate with arrow keys and select with Enter

Settings Configuration: After selecting a log file, you'll see an interactive settings screen where you can configure:

📊 Analysis Parameters: Enable/disable specific analysis tabs (Security, Performance, Errors, Bots, Sparkline, Heatmap)
🔧 Processing Options: Set count limits, regex patterns, date formats
⚙️ Real-time Settings: Configure cleanup policies and monitoring options
▶️ Start Analysis: Launch the analysis with your custom configuration

Usage:

# Launch interactive mode
logutil

# Or specify a file directly
logutil /path/to/access.log

🎮 Keyboard Shortcuts

Navigation:

Tab / T: Switch to next tab
Shift+Tab / Shift+T: Switch to previous tab
↑/↓: Navigate through lists and options
←/→: Switch between panels (in tabs with multiple panels)
Enter: Select item or toggle boolean values
Esc: Go back or cancel

File Selector:

↑/↓: Navigate through files and folders
Enter: Select file or enter folder
Esc: Go back to parent directory or exit

Settings:

↑/↓: Navigate through settings
Enter: Edit setting or toggle boolean values
Esc: Cancel editing or go back

TUI Controls:

Q / Ctrl+C: Quit application
Enter: Copy selected item to clipboard (Overview tab)

📋 Command Line Examples

Interactive mode with pre-configured parameters:

# Launch file selector with all analysis tabs enabled
cargo run "" --enable-security --enable-performance --enable-errors --enable-bots --enable-sparkline --enable-heatmap --count=1000

# Launch with custom settings
cargo run "" --enable-security --enable-performance --top=20 --count=500

# Launch with specific analysis tabs only
cargo run "" --enable-security --enable-errors --count=2000

Direct file analysis with all features:

# Analyze with all tabs enabled
logutil access.log --enable-security --enable-performance --enable-errors --enable-bots --enable-sparkline --enable-heatmap --count=1000

# Security-focused analysis
logutil access.log --enable-security --enable-errors --top=50

# Performance monitoring
logutil access.log --enable-performance --enable-sparkline --enable-heatmap

Installation

Quick Install (Recommended)

Download the latest release for your platform:

Linux (x86_64):

curl -L -o /usr/local/bin/logutil https://github.com/s00d/logutil/releases/latest/download/logutil-x86_64-unknown-linux-gnu
chmod +x /usr/local/bin/logutil

Linux (ARM64):

curl -L -o /usr/local/bin/logutil https://github.com/s00d/logutil/releases/latest/download/logutil-aarch64-unknown-linux-gnu
chmod +x /usr/local/bin/logutil

macOS:

curl -L -o /usr/local/bin/logutil https://github.com/s00d/logutil/releases/latest/download/logutil-x86_64-apple-darwin
chmod +x /usr/local/bin/logutil

Build from Source

Install Rust:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rust-lang.org | sh

Clone and Build:

git clone https://github.com/s00d/logutil.git
cd logutil
cargo build --release

Install:

sudo cp target/release/logutil /usr/local/bin/

Usage

Basic Usage

Monitor a log file in real-time:

logutil /var/log/nginx/access.log

Analyze the entire log file:

logutil /var/log/nginx/access.log --count=-1

Show only the last 1000 lines:

logutil /var/log/nginx/access.log --count=1000

Advanced Usage

Custom regex pattern:

logutil /var/log/nginx/access.log --regex='^(\S+) - - \[(.*?)\] "(\S+) (\S+) HTTP/\d+\.\d+" (\d+) (\d+) "([^"]*)" "([^"]*)"$'

Show top 20 entries:

logutil /var/log/nginx/access.log --top=20

Disable automatic cleanup:

logutil /var/log/nginx/access.log --no-clear

Custom date format:

logutil /var/log/nginx/access.log --date-format="%d/%b/%Y:%H:%M:%S %z"

Load regex from file:

logutil /var/log/nginx/access.log --regex=/path/to/regex.txt

Tab Management

By default, only the core tabs (Overview, Requests, Detailed) are enabled. Additional tabs can be enabled using command-line flags:

Enable Security tab (detect suspicious activity, attacks, etc.):

logutil /var/log/nginx/access.log --enable-security

Enable Performance tab (monitor response times, slow requests):

logutil /var/log/nginx/access.log --enable-performance

Enable Errors tab (track error codes and failed requests):

logutil /var/log/nginx/access.log --enable-errors

Enable Bots tab (detect bot traffic and crawlers):

logutil /var/log/nginx/access.log --enable-bots

Enable Sparkline tab (real-time request rate visualization):

logutil /var/log/nginx/access.log --enable-sparkline

Enable Heatmap tab (hourly traffic patterns visualization):

logutil /var/log/nginx/access.log --enable-heatmap

Enable all tabs:

logutil /var/log/nginx/access.log --enable-security --enable-performance --enable-errors --enable-bots --enable-sparkline --enable-heatmap

Console Output Mode

Show top URLs in console:

logutil /var/log/nginx/access.log --show-urls --top=10

Show top IPs in console:

logutil /var/log/nginx/access.log --show-ips --top=10

Interactive Interface

Navigation

Tab or t: Switch between tabs (Overview → Requests → Detailed → Sparkline → Heatmap)
↑/↓: Navigate through lists
←/→: Switch between panels or pages
Enter: Copy selected item to clipboard (in Overview tab)
q or Ctrl+C: Quit the application

Tabs Overview

1. Overview Tab

Left Panel: Top IP addresses with request counts and last update times
Right Panel: Top URLs with request types, domains, and statistics
Bottom Panel: Full URL display for selected URL
Navigation: Use arrow keys to switch between panels

2. Requests Tab

Search Functionality: Type to filter log entries
Pagination: Navigate through large result sets
Real-time Updates: New requests appear automatically

3. Detailed Tab

IP List: Select an IP address to view its details
Request Details: View all requests from the selected IP
Drill-down Analysis: Understand tr

Logutil

Install / Use

README