Project Synapse 🧠

🌟 Multi-Branch Repository: This repository showcases Project Synapse in two complementary implementations:

• 🎓 educational-simulation: A comprehensive educational simulation with detailed documentation, perfect for understanding multi-agent architecture concepts without running live services.
• 🚀 working-app: A fully functional, production-ready implementation with Docker, Kubernetes, and monitoring. You are currently viewing this branch.

A production-ready multi-agent system showcasing Agent Communication Protocol (ACP) and Model Context Protocol (MCP) capabilities through a collaborative research workflow.

"Where artificial intelligence meets production-grade architecture"

🎯 Project Overview

Project Synapse is a comprehensive multi-agent system built with modern async Python, featuring specialized agents that work together to investigate complex research questions. The system demonstrates advanced patterns in agent communication, secure tool integration, and production deployment practices.

Key Highlights

• 🚀 Production-Ready: Docker containerization with resource management and health monitoring
• ⚡ High Performance: Async/await architecture with measured performance metrics
• 🔒 Security-First: MCP Roots implementation with filesystem access controls
• 📊 Observable: Comprehensive logging and real-time monitoring
• 🏗️ Scalable: RabbitMQ message bus with connection pooling

🌟 Architecture Features

Production Infrastructure

| Component | Technology | Performance | Status | |-----------|------------|-------------|---------| | HTTP Servers | FastAPI v0.104.1 | 1,447 RPS file ops | ✅ Optimized | | Message Bus | RabbitMQ 3.13.7 | 557 RPS API calls | ✅ High Availability | | Containerization | Docker Compose | <131MB per service | ✅ Resource Managed | | Agent Coordination | Async Python | 19.8 RPS search ops | ✅ Production Ready |

Agent Ecosystem

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│  Orchestrator   │◄──►│  Search Agent   │◄──►│ Extraction Agent│
│  (Coordinator)  │    │  (Discovery)    │    │  (Processing)   │
└─────────────────┘    └─────────────────┘    └─────────────────┘
         │                       │                       │
         ▼                       ▼                       ▼
┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│ Fact Checker    │    │  Synthesis      │    │  File Save      │
│ (Validation)    │    │  (Generation)   │    │  (Storage)      │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌─────────────────┐
                       │  Logger Agent   │
                       │  (Monitoring)   │
                       └─────────────────┘

� Quick Start

Prerequisites

• Docker and Docker Compose
• Git

Production Deployment

# Clone the repository
git clone https://github.com/YanCotta/project-synapse.git
cd project-synapse

# Deploy with optimized configuration (includes monitoring)
docker-compose -f docker-compose.optimized.yml up --build

# Monitor system performance
python scripts/monitor_system.py

Kubernetes Deployment

# Deploy to Kubernetes cluster
./k8s/deploy.sh

# Access monitoring
kubectl port-forward svc/grafana 3000:3000
kubectl port-forward svc/prometheus 9090:9090

Development Setup

# Alternative: Local development
pip install -r requirements.txt
python async_main.py

📊 Performance Metrics

Measured Performance (Production Testing)

| Metric | Primary Server | Filesystem Server | RabbitMQ | |--------|---------------|-------------------|----------| | Response Time | 0.9ms avg | 0.8ms avg | 1.4ms avg | | P95 Latency | 3.5ms | 2.1ms | 2.6ms | | Throughput | 668.4 RPS | 882.7 RPS | 557 RPS | | Success Rate | 100% | 100% | 100% |

System Resources

• Memory Usage: ~131MB per service (optimized)
• Container Startup: 20-30% faster with optimization
• Connection Efficiency: 40-60% improvement with pooling
• Network Isolation: Custom bridge network (172.20.0.0/16)

🏗️ System Architecture

Production Infrastructure Stack

Container Orchestration

• Docker Compose: Multi-service deployment with health checks
• Resource Limits: CPU and memory constraints for production stability
• Health Monitoring: Automated health checks with retry mechanisms
• Network Isolation: Secure service communication

Message Bus (RabbitMQ)

• High Availability: Production-grade message broker
• Connection Pooling: Efficient connection reuse
• Performance Tuning: Memory watermarks and optimization
• Authentication: Secure credential management

HTTP Infrastructure

• FastAPI Servers: Async HTTP servers with streaming SSE
• Connection Pooling: 50 total connections, 10 per host
• Error Handling: Comprehensive error responses
• Health Endpoints: Service status monitoring

Security Architecture

MCP Roots Implementation

• Filesystem Boundaries: Restricted access to approved directories
• Path Validation: Comprehensive security checks
• Access Logging: Security event monitoring
• Error Handling: Secure error responses

� Configuration Management

Docker Compose Production Configuration

version: '3.8'
services:
  rabbitmq:
    image: rabbitmq:3.13.7-management
    environment:
      RABBITMQ_VM_MEMORY_HIGH_WATERMARK: 0.8
    deploy:
      resources:
        limits: { memory: 512M, cpus: "0.5" }
    healthcheck:
      test: ["CMD", "rabbitmq-diagnostics", "ping"]
      interval: 30s
      timeout: 10s
      retries: 3

Performance Optimization

• Connection Pooling: 30-second keep-alive with automatic cleanup
• Resource Limits: Memory and CPU constraints
• Health Checks: Automated service monitoring
• Network Optimization: Custom bridge network for isolation

📁 Project Structure

project-synapse/
├── src/
│   ├── agents/              # 7 specialized async agents
│   │   ├── async_orchestrator.py    # Central coordinator
│   │   ├── async_search_agent.py    # Web search capabilities
│   │   ├── async_extraction_agent.py # Content extraction
│   │   ├── async_fact_checker_agent.py # Validation services
│   │   ├── async_synthesis_agent.py  # Report generation
│   │   ├── async_file_save_agent.py  # Secure file operations
│   │   ├── async_logger_agent.py     # System monitoring
│   │   └── async_base_agent.py       # Common agent functionality
│   ├── mcp_servers/         # Production MCP servers
│   │   ├── fastapi_primary_server.py    # Web tools with progress
│   │   └── fastapi_filesystem_server.py # Secure file operations
│   ├── message_bus/         # RabbitMQ message bus implementation
│   │   └── rabbitmq_bus.py          # Async message routing
│   └── protocols/           # Communication schemas
│       ├── acp_schema.py        # Agent Communication Protocol
│       └── mcp_schemas.py       # Model Context Protocol
├── k8s/                     # Kubernetes deployment manifests  
│   ├── configmap.yaml          # Environment configuration
│   ├── rabbitmq-deployment.yaml # RabbitMQ message broker
│   ├── primary-server-deployment.yaml # Primary MCP server
│   ├── filesystem-server-deployment.yaml # Filesystem MCP server
│   ├── agents-deployment.yaml   # Agent application
│   └── deploy.sh               # Deployment automation script
├── monitoring/              # Observability and monitoring
│   ├── prometheus.yml          # Prometheus configuration
│   ├── grafana_dashboard.json  # Pre-built Grafana dashboard
│   ├── grafana-datasources.yml # Grafana data source config
│   └── grafana-dashboards.yml  # Dashboard provisioning config
├── scripts/                 # Performance and monitoring tools
│   ├── performance_test.py     # Load testing framework
│   ├── optimize_performance.py # Performance optimization
│   ├── monitor_system.py       # Real-time monitoring
│   ├── health_check.py         # System health validation
│   └── integration_test.py     # End-to-end testing
├── docs/                    # Comprehensive documentation
│   ├── ARCHITECTURE.md         # System architecture guide
│   ├── ACP_SPEC.md            # ACP protocol specification
│   └── MCP_IN_DEPTH.md        # MCP implementation guide
├── docker-compose.optimized.yml # Production deployment
├── async_main.py            # System entry point
└── requirements.txt         # Python dependencies

🔬 Technical Documentation

Protocol Specifications

• Agent Communication Protocol (ACP): Custom messaging protocol with type safety
• Model Context Protocol (MCP): Tool integration with progress notifications
• System Architecture: Complete architectural documentation

Production Considerations

• Performance Testing: Load testing with baseline metrics
• Resource Management: Memory and CPU optimization
• Health Monitoring: Automated service health checks
• Security: MCP Roots and access control implementation

�️ Development Workflow

Performance Testing

# Run performance baseline tests
python scripts/performance_test.py

# Monitor system resources
python scripts/monitor_system.py

# Optimize performance settings
python scripts/optimize_performance.py

Health Monitoring

The system includes comprehensive health monitoring:

• Service Health: HTTP health endpoints for all services
• Message Bus Status: RabbitMQ connection and queue monitoring
• Agent Status: Real-time agent activity tracking
• Resource Usage: Memory and CPU utilization monitoring

🎯 Use Cases

Research Workflow Automation

The system demonstrates automated research workflows:

1. Query Processing: Complex research question analysis
2. Web Search: Distributed search across multiple sources
3. Content Extraction: Intelligent content processing
4. Fact Checking: Automated claim validation
5. Report Synthesis: Comprehensive report generation
6. Secure Storage: MCP Roots-protected file operations

Agent Coordination Patterns

• Command & Control: Orchestrator coordinating specialized agents
• Peer Review: Fact-checking and validation workflows
• Publish-Subscribe: System-wide event monitoring
• Request-Response: Sophisticated inter-agent communication

🔮 Production Deployment

Scalability Features

• Horizontal Scaling: Multiple worker processes per service
• Resource Isolation: Container-based deployment
• Health Monitoring: Automated failure detection
• Connection Pooling: Efficient resource utilization

Monitoring and Observability

• Real-time Metrics: Performance monitoring with baseline comparison
• Health Dashboards: Service status and resource utilization
• Error Tracking: Comprehensive error logging and reporting
• Performance Analytics: Response time and throughput analysis

📜 License

MIT License - see LICENSE for details.

🎯 Key Features

✅ Production-Ready: Docker containerization with Kubernetes deployment
✅ High Performance: Measured metrics with optimization framework
✅ Secure Architecture: MCP Roots implementation with access controls
✅ Observable Systems: Prometheus metrics and Grafana dashboards
✅ Scalable Design: Async architecture with connection pooling
✅ Complete Documentation: Architecture guides and implementation details
✅ Production Monitoring: Real-time metrics and alerting capabilities

---

Deploy a production-grade multi-agent system:

# Dock