Auto-News - Indonesian Stock Market Analysis Platform

Comprehensive Indonesian stock market analysis platform with News Pipeline + Technical Analysis + MCP Integration - all running in Docker containers.

Features:

• Automated news scraping and semantic search (Milvus vector database)
• Technical analysis API (RSI, MACD, Bollinger Bands, Smart Money Flow)
• Trading strategy screening (PREOPEN, BPJS, BSJP, Day Trade setups)
• MCP server for AI assistant integration (Claude Desktop/Code)
• RESTful API for programmatic access

Connect any MCP-compatible AI assistant (Claude, or other LLMs) and ask: "Show me today's Indonesian stock market news" to get instant market insights!

What's New

Major Updates

🐳 Unified Docker Architecture

• All services now run in containers via docker-compose.yml at project root
• idx-stock-api container - Internal API for data processing (port 13052)
• idx-report-server container - Public web UI for reports (port 3131, read-only)
• Persistent data volumes for seamless updates and restarts
• No more manual dependency installation!

🚀 Comprehensive Stock API

• 20+ REST endpoints for news, stock data, technical analysis, and screening
• Port 13052 - New unified API server (replaces old port 13051)
• Technical indicators: RSI, MACD, Bollinger Bands, Volume analysis
• Bandarmology/smart money flow detection
• Trading strategy screening (PREOPEN, BPJS, BSJP, Day Trade)
• Market context endpoints (global sentiment, trading time, stock lists)

🔌 Enhanced MCP Integration

• MCP server now runs inside Docker via docker exec
• Shares same /app/data volume as FastAPI server
• 5 MCP tools for news pipeline management
• Seamless integration with Claude Desktop/Code

🔐 Security Features

• Optional CLAUDE_SECRET_KEY authentication for sensitive endpoints
• Localhost/Docker network IP whitelisting for /api/news/analyze (openclaw integration)
• Claude CLI credentials stored in persistent volumes
• Environment-based configuration via .env file

📁 Reorganized Project Structure

• Pipeline scripts moved to src/helper/
• Stock API code in src/stock_api/
• Data persistence in volumes/ directory
• Docker-compose at project root (no more rag/ subdirectory)

📋 For detailed migration guide from older versions, see the changelog at the end of this README.

Table of Contents

• Complete Workflow
• Quick Start
  • Installation
  • Option 1: Use with MCP-Compatible AI Assistants
  • Option 2: Manual CLI Usage
  • Option 3: REST API Usage
• MCP Integration Details
• REST API Integration
• Files Overview
• Manual Step-by-Step Usage
• Web Server Features
• Configuration
• Daily Automation with Cron
• Troubleshooting
• Dependencies
• Example Output
• Features
• Future Enhancements
• Disclaimer

Architecture Overview

┌───────────────────────────────────────────────────────────────────────┐
│                        Docker Compose Stack                            │
├───────────────────────────────────────────────────────────────────────┤
│                                                                         │
│  ┌──────────────────┐  ┌───────────────────────┐  ┌────────────────┐ │
│  │  Milvus Stack    │  │  Stock API Container  │  │ Report Server  │ │
│  │  ─────────────   │  │  (idx-stock-api)      │  │ (public web)   │ │
│  │  • Milvus DB     │←─┤  ─────────────────    │  │ ──────────────  │ │
│  │  • etcd          │  │  • FastAPI (13052)    │  │ • Web UI       │ │
│  │  • MinIO         │  │    [Internal Only]    │  │   (port 3131)  │ │
│  │  • Attu UI       │  │  • News Pipeline      │  │ • Read-only    │ │
│  └──────────────────┘  │  • Technical APIs     │  │   /app/data    │ │
│                         │  • MCP Server         │←─┤ • Auto-refresh │ │
│                         └───────────────────────┘  └────────────────┘ │
│                                                                         │
│                         volumes/news_data/ (shared)                    │
│                         ├─ news.txt                                    │
│                         ├─ news_condensed.txt                          │
│                         └─ daily_report.md  ←─────(read-only)          │
└───────────────────────────────────────────────────────────────────────┘
                                    ↑
                    ┌───────────────┼───────────────┐
                    │               │               │
             ┌──────▼─────┐  ┌──────▼─────┐  ┌──────▼─────┐
             │   MCP      │  │  REST API  │  │  Browser   │
             │  Clients   │  │   Clients  │  │  (Public)  │
             │  (Claude)  │  │  (curl/app)│  │  :3131     │
             └────────────┘  └────────────┘  └────────────┘

Complete Workflow

News Pipeline:

1. Scrape unlimited articles → 2. Index in Milvus → 3. Semantic search → 4. AI analysis

Stock Analysis:

1. Fetch OHLCV from Yahoo Finance → 2. Calculate indicators → 3. Generate signals → 4. Return analysis

Why This Architecture?

Containerized: All services run in Docker - consistent environment, easy deployment

Security-First: Stock API (port 13052) stays internal, only report server (port 3131) exposed publicly

Separation of Concerns:

• idx-stock-api - Internal API for data processing (news + stock analysis)
• idx-report-server - Public web server with read-only access to reports

Semantic Search: Milvus vector database handles 500+ articles efficiently (no token limits)

MCP Integration: Connect AI assistants directly to your stock analysis infrastructure

Quick Start

Installation

# 1. Clone the repository
git clone <your-repo-url>
cd auto-news

# 2. Create Docker network (required for inter-container communication)
docker network create tunnel

# 3. (Optional) Configure security for Claude analysis endpoint
cp .env.example .env
# Edit .env and set CLAUDE_SECRET_KEY for production use

# 4. Start all services (Milvus + Stock API)
docker-compose up -d

# 5. Verify services are running
docker ps
# You should see: etcd, minio, milvus-standalone, attu, idx-stock-api, idx-report-server

# 6. Check services health
curl http://localhost:13052/        # Stock API (internal)
curl http://localhost:3131/         # Report Server (public web UI)

# 7. (Optional) Configure Claude CLI for /api/news/analyze endpoint
# This allows the API to analyze news using Claude
docker exec -it idx-stock-api su appuser -c "claude /login"
# Follow the prompts to authenticate with your Claude account

Service Ports:

• 13052 - Stock API (keep internal, for API/MCP access only)
• 3131 - Report Server (safe to expose publicly, read-only web UI)
• 19530 - Milvus (internal only)
• 1233 - Attu UI (optional, for database management)

Notes:

• The Stock API container includes all dependencies (Python packages, embedding models, etc.) - no need to install anything locally unless you want to run scripts outside Docker.
• The Report Server is a minimal (~50MB) Alpine container with read-only access to reports.
• Claude CLI configuration is optional and only needed if you want to use the /api/news/analyze endpoint for automated AI analysis.

Option 1: Use with MCP-Compatible AI Assistants (Recommended!)

Connect any MCP-compatible AI to your news pipeline - now running inside Docker!

1. Ensure Docker containers are running:

   docker ps | grep idx-stock-api
   # Should show the stock-api container
   

2. Configure your MCP client - Add to your MCP client's config file:

   • Claude Code: ~/.claude/claude_code_config.json
   • Claude Desktop: ~/Library/Application Support/Claude/claude_desktop_config.json
   • Other MCP clients: Check your client's documentation for config location

   {
     "mcpServers": {
       "stock-news": {
         "type": "stdio",
         "command": "docker",
         "args": ["exec", "-i", "idx-stock-api", "python3", "/app/mcp_server.py"]
       }
     }
   }
   

   What this does: Uses docker exec to run the MCP server inside the idx-stock-api container, allowing it to share the same data volume (/app/data) and network as the FastAPI server.

3. Restart your MCP client (Claude Desktop, Claude Code, etc.)

4. Ask your AI assistant:

   • "Show me today's Indonesian stock market news"
   • "Get the latest news and check the status"
   • "Read the condensed news report"
   • "Check which files already exist in the data folder"

That's it! Your AI assistant will automatically call the MCP tools which communicate with the FastAPI server running in Docker.

Available MCP Tools:

• get_news_status - Check pipeline status
• run_news_pipeline_async - Start news scraping in background
• run_news_pipeline_sync - Run news scraping synchronously
• check_existing_files - See which reports already exist
• read_news_report - Read condensed news output

MCP Compatibility: This server implements the Model Context Protocol standard and works with any MCP-compatible client, not just Claude.

Example Results (Simplified):

I'll get today's Indonesian stock market news for you.

**KEY MARKET HIGHLIGHTS:**

**IHSG:** Dropped 0.71% to 8,584.78 (Dec 23), opened weak today at 8,607 then slipped to 8,583

**FOREIGN FLOW:** Net buy Rp2.5T in session 1 today - targeting mining/commodity stocks

**TOP MOVERS:**
- **BUMI:** -3.55% (heavy profit-taking after 14% jump Mon)
- **INCO:** +12.39% (foreign accumulation Rp101B on Tue, +1.7% opening today)
- **ANTM:** +74B foreign buy (nickel sector hot)
- **BBCA:** -1.83% to Rp8,025 (pressure despite 3.73% div yield)

**SENTIMENT:**
- Santa Rally stalling - window dressing + profit-taking collide
- Mining stocks magnets for foreign capital
- IHSG target 9,000 still possible per IPOT but needs big-cap banking/infra surge
- Investors cautious pre-Christmas holiday

**TRADEABLE SETUPS:**
- **INCO/ANTM:** Bandarmology strong, foreign accumulation continues
- **Banks (BBCA/BMRI):** Undervalued on dividend play but profit-taking risk
- **Infrastructure:** TLKM, BREN, TOWR seeing net buy flows

**RISK:** Volume thin, profit-taking ahead of long holiday

Option 2: Manual CLI Usage

# Run complete pipeline (scrape → index → semantic search)
bash prepare_news.sh

# Analyze with Claude Code CLI
bash run_daily_analysis.sh

# View report in browser (report server is already running!)
open http://localhost:3131

The report server container automatically displays volumes/news_data/daily_report.md when you visit http://localhost:3131

Option 3: REST API Usage

The Stock API server runs automatically in Docker (no manual startup needed!):

# API is already running on port 13052
curl http://localhost:13052/

# View interactive API documentation
open http://localhost:13052/docs

The API provides comprehensive News Pipeline + Stock Analysis + Screening endpoints:

News Pipeline Endpoints:

• POST /api/news/get - Run news pipeline asynchronously (returns immediately)
• POST /api/news/get/sync - Run news pipeline synchronously (waits for completion)
• GET /api/news/status - Check pipeline status
• GET /api/news/read - Read condensed news report (plain text)
• GET /api/news/read/json - Get report as JSON with metadata
• `GET /api/news/analy