OTP Supervisor Educational Platform

A comprehensive, production-grade educational platform for learning OTP (Open Telecom Platform) supervision concepts through hands-on experimentation and real-time visualization.

Overview

This project provides an advanced learning environment for OTP concepts, combining theoretical knowledge with practical implementation. Built with Phoenix LiveView, it offers:

Supervisors - Interactive Management Interactive supervisor management dashboard with real-time visualization

System Dashboard - Real-time Monitoring System-wide monitoring interface with analytics and health metrics

Arsenal - Operations Dashboard Advanced operations dashboard for OTP process management and debugging

Documentation - Learning Resources Comprehensive documentation and educational resources for OTP concepts

🔬 Interactive Sandbox Environments - Isolated experimentation without system risk
📊 Real-time Analytics - Production-grade telemetry and monitoring systems
🎯 Educational Focus - Progressive complexity with comprehensive documentation
⚡ Live Visualization - Dynamic supervision tree updates and process management
🛠️ Advanced Tooling - Message tracing, process introspection, and system analysis
🏗️ Production Patterns - Real-world OTP design patterns and best practices

🚀 Quick Start

mix setup                    # Install dependencies and setup database
mix phx.server              # Start server → http://localhost:4000
# OR
iex -S mix phx.server       # Start with interactive console

Key Interfaces

/supervisors - Interactive supervisor management dashboard
/system - System-wide analytics and monitoring
/api/v1/* - RESTful API for programmatic access

🔧 Developer Status & Distributed Development

Current Implementation Status

✅ Completed Features:

Distributed tooling foundation with mode switching
Single-node simulator for development without real cluster
Cluster state manager with real-time topology tracking
Distributed Arsenal operations (ClusterHealth, NodeInfo, ClusterTopology, ProcessList)
Comprehensive test suite with proper OTP synchronization

🚧 In Progress:

Horde-specific Arsenal operations
Enhanced existing Arsenal operations for distributed support
Distributed sandbox management system

Distributed Development Setup

Option 1: Single Node with Simulation (Recommended for Development)

# Start the server normally - simulation mode is enabled by default
mix phx.server
# OR with interactive console
iex -S mix phx.server

Option 2: Multi-Node Cluster Setup

# Terminal 1: Start primary node
./scripts/start_node1.sh
# This runs: iex --sname superlearner1 --cookie secret_cluster_cookie -S mix phx.server
# Web interface: http://localhost:4000

# Terminal 2: Start secondary node (if needed)
./scripts/start_node2.sh
# This runs: iex --sname superlearner2 --cookie secret_cluster_cookie -S mix

Testing Distributed Features

From IEx Console (when running ./scripts/start_node1.sh):

# Check cluster status
Node.list()                                    # List connected nodes
Node.self()                                    # Current node name

# Test distributed tooling
OTPSupervisor.Distributed.ToolManager.get_mode()           # Current mode
OTPSupervisor.Distributed.ToolManager.get_cluster_status() # Cluster info

# Test cluster state management
OTPSupervisor.Distributed.ClusterStateManager.get_cluster_topology()
OTPSupervisor.Distributed.ClusterStateManager.get_process_distribution()

# Test single-node simulation
OTPSupervisor.Distributed.SingleNodeSimulator.simulation_enabled?()
OTPSupervisor.Distributed.SingleNodeSimulator.enable_simulation(3)
OTPSupervisor.Distributed.SingleNodeSimulator.get_simulated_topology()

Example Output from Real Multi-Node Test:

iex(superlearner1@U2401)> Node.connect(:"superlearner2@U2401")
[info] Node joined cluster: superlearner2@U2401
true

iex(superlearner1@U2401)> Node.list()
[:superlearner2@U2401]

iex(superlearner1@U2401)> OTPSupervisor.Distributed.ToolManager.get_cluster_status()
%{
  nodes: [:superlearner1@U2401, :superlearner2@U2401],
  mode: :multi_node,
  connected_nodes: [:superlearner2@U2401],
  current_node: :superlearner1@U2401,
  tools: []
}

iex(superlearner1@U2401)> OTPSupervisor.Distributed.ClusterStateManager.get_cluster_topology()
%{
  nodes: [:superlearner1@U2401, :superlearner2@U2401],
  connected_nodes: [:superlearner2@U2401],
  current_node: :superlearner1@U2401,
  total_nodes: 2,
  cluster_name: :superlearner_cluster,
  formation_time: ~U[2025-07-15 01:27:17.152676Z]
}

iex(superlearner1@U2401)> OTPSupervisor.Distributed.ClusterStateManager.get_process_distribution()
%{
  superlearner1@U2401: [#PID<0.0.0>, #PID<0.1.0>, ...], # 598 processes
  superlearner2@U2401: [#PID<27148.0.0>, #PID<27148.1.0>, ...] # Similar count
}

From Command Line (test distributed Arsenal operations):

# Test cluster health endpoint
curl http://localhost:4000/api/v1/cluster/health

# Test cluster topology
curl http://localhost:4000/api/v1/cluster/topology

# Test node information (replace with actual node name)
curl http://localhost:4000/api/v1/cluster/nodes/superlearner1@U2401/info

# Test distributed process list
curl http://localhost:4000/api/v1/cluster/processes

Example JSON Response from Cluster Health:

{
  "data": {
    "overall_status": "healthy",
    "partition_status": "healthy",
    "nodes_total": 2,
    "nodes_healthy": 2,
    "nodes_unhealthy": 0,
    "cluster_uptime": "37s",
    "node_statuses": {
      "superlearner1@U2401": {
        "status": "up",
        "health_score": 100,
        "cpu_status": "normal",
        "memory_status": "normal",
        "issues": []
      },
      "superlearner2@U2401": {
        "status": "up", 
        "health_score": 100,
        "cpu_status": "normal",
        "memory_status": "normal",
        "issues": []
      }
    },
    "performance_metrics": {
      "total_processes": 1194,
      "memory_usage": {
        "superlearner1@U2401": {"total": 86914960, "processes": 24422864},
        "superlearner2@U2401": {"total": 86401672, "processes": 24214256}
      }
    },
    "recommendations": []
  },
  "success": true,
  "timestamp": "2025-07-15T01:27:54.227947Z"
}

Development Commands for Distributed Features

Testing & Quality Assurance

# Run distributed-specific tests
mix test test/otp_supervisor/distributed/

# Run all tests
mix test

# Check for compiler warnings and dialyzer issues
mix compile --warnings-as-errors
mix dialyzer

# Format code
mix format

Debugging Distributed Issues

# Check Erlang statistics (useful for understanding system behavior)
mix run -e "IO.inspect(:erlang.statistics(:reductions)); IO.inspect(:erlang.statistics(:io)); IO.inspect(:erlang.statistics(:garbage_collection)); IO.inspect(:erlang.statistics(:run_queue))"

# Start with debugging enabled
MIX_ENV=dev LOG_LEVEL=debug ./scripts/start_node1.sh

Cluster Management Scripts

# Check cluster status
./scripts/cluster_status.sh

# Stop nodes gracefully
./scripts/stop_node1.sh
./scripts/stop_node2.sh

# Kill processes on specific ports (if needed)
./scripts/kill_beam_on_port.sh 4000

Distributed Architecture Overview

Distributed Development Tooling
├── ToolManager                    # Central coordinator for distributed tools
├── SingleNodeSimulator           # Development mode simulation
├── ClusterStateManager           # Real-time cluster topology tracking
└── Arsenal Operations
    ├── ClusterHealth             # GET /api/v1/cluster/health
    ├── ClusterTopology           # GET /api/v1/cluster/topology  
    ├── NodeInfo                  # GET /api/v1/cluster/nodes/:node/info
    └── ProcessList               # GET /api/v1/cluster/processes

Configuration Notes

LibCluster: Configured for automatic node discovery
Horde: Available for distributed process registry and supervision
Simulation Mode: Automatically enabled in single-node development
Multi-Node Mode: Automatically detected when nodes connect

🧪 Distributed Test Infrastructure

For comprehensive distributed testing capabilities, see:

README_DISTRIBUTED_TESTING.md - Professional distributed testing framework
README_ROBUST_CLUSTER_TESTING.md - WSL-compatible automated cluster management

Quick Test Commands

# Professional distributed test cluster management
mix cluster.test status          # Check cluster status
mix cluster.test start           # Start test cluster
mix cluster.test health          # Comprehensive health check
mix cluster.test run             # Full automated test cycle
mix cluster.test stop            # Stop cluster

# Run distributed tests
mix test --only real_nodes       # Real multi-node tests
mix test --only simulation       # Simulation-based tests

Key Features:

✅ Automated Cluster Management - No more manual server scripts
✅ Robust Error Handling - Addresses all distributed Erlang startup issues
✅ Code Synchronization - Prevents testing against stale code
✅ Health Monitoring - Comprehensive cluster validation
✅ Professional Tooling - Production-grade CLI interface

📋 Essential Commands

Development & Testing

# Server Management
mix phx.server              # Start Phoenix server
iex -S mix phx.server       # Start with IEx console
mix test                    # Run comprehensive test suite
mix test --cover            # Run tests with coverage analysis

# Code Quality
mix format                  # Format code
mix compile --warnings-as-errors
mix dialyzer               # Static analysis (if configured)

# Datab

Superlearner

Install / Use

README