OSINT Skill

Early Beta — functional but APIs change, Apify actors rotate, and edge cases are still being discovered.

Версия на русском / Russian version

Systematic intelligence gathering on individuals. From a name or handle to a scored dossier with psychoprofile, career map, and entry points.

Compatible Agents

Works with any AI coding agent that supports the SKILL.md format:

| Agent | Install | |-------|---------| | Claude Code | cp -r osint/ ~/.claude/skills/osint/ | | OpenClaw | Copy to workspace skills/ directory | | Codex | Point to osint/SKILL.md in your agent config | | OpenCode | Copy to project skills/ directory | | Any SKILL.md agent | Place osint/ folder where your agent reads skills |

The skill uses standard tools (bash, curl, node, python3) and writes instructions in Markdown — no vendor lock-in.

Features

• Phased pipeline (0 → 1 → 1.5 → 2 → 3 → 4 → 5 → 6): from quick search to deep research
• Swarm Mode: coordinates 3-5 parallel sub-agents on Sonnet for speed
• 55+ Apify actors embedded: Instagram (12), Facebook (14), TikTok (14), YouTube (5), Google Maps (4), LinkedIn, and more
• Psychoprofile: MBTI / Big Five based on content analysis (YouTube transcripts, Telegram messages, blogs)
• Confidence Scoring: every fact gets graded A/B/C/D by number of independent confirmations
• Internal Intelligence: checks Telegram history, email, vault contacts BEFORE going external
• Research Escalation: 4 levels from free to $0.50, from seconds to minutes
• Budget tracking: spends ≤$0.50 without asking, asks permission above that

Quick Start

# Clone the repo
git clone https://github.com/smixs/osint-skill.git
cd osint-skill

# Copy to your agent's skills directory (example for Claude Code)
cp -r osint/ ~/.claude/skills/osint/

# Run self-diagnostics to check what's available
bash osint/scripts/diagnose.sh

Requirements

Required

| Tool | Purpose | Install | |------|---------|---------| | curl | HTTP requests to APIs | Pre-installed on macOS/Linux | | python3 | JSON parsing, MCP client | Pre-installed on macOS/Linux | | jq | JSON processing | brew install jq / apt install jq |

For Apify actors (55+ platforms)

| Tool | Purpose | Install | |------|---------|---------| | Node.js 18+ | Runs run_actor.js (embedded Apify runner) | nodejs.org |

Optional

| Tool | Purpose | Install | |------|---------|---------| | mcpc | Dynamic actor discovery in Apify Store | npm install -g @apify/mcpc |

API Keys & Services

The skill uses graceful degradation — the more API keys you provide, the deeper it can dig. You need at least one search API to get started.

Free Tier

| Service | Env Variable | What It Does | Get It | |---------|-------------|--------------|--------| | Brave Search | (built into Claude Code) | 2,000 queries/month, basic web search | Built-in, nothing needed | | Jina AI | JINA_API_KEY | URL → markdown reader, search, deepsearch | jina.ai/api-key | | Apify | APIFY_API_TOKEN | Instagram, TikTok, YouTube, LinkedIn scraping. Free tier ~$5/month | console.apify.com | | Parallel AI | PARALLEL_API_KEY | AI-powered search with reasoning and citations | platform.parallel.ai |

Paid (recommended)

| Service | Env Variable | What It Does | Cost | Get It | |---------|-------------|--------------|------|--------| | Perplexity API | PERPLEXITY_API_KEY | Sonar (fast AI answers), Deep Research | ~$5/month | perplexity.ai/settings/api | | Exa AI | EXA_API_KEY | Semantic search, people/company research | ~$5/month | dashboard.exa.ai | | Tavily | TAVILY_API_KEY | Agent-optimized search, $0.005/request basic | ~$5/month | app.tavily.com |

Advanced

| Service | Env Variable | What It Does | Cost | Get It | |---------|-------------|--------------|------|--------| | Bright Data | BRIGHTDATA_MCP_URL | CAPTCHA bypass, authwall bypass, Facebook scraping, Yandex search | ~$10/month+ | brightdata.com/mcp |

Setting Up Keys

Option 1 — Environment variables (recommended):

export PERPLEXITY_API_KEY="pplx-..."
export EXA_API_KEY="exa-..."
export APIFY_API_TOKEN="apify_api_..."
export JINA_API_KEY="jina_..."
export TAVILY_API_KEY="tvly-..."
export PARALLEL_API_KEY="..."
export BRIGHTDATA_MCP_URL="https://mcp.brightdata.com/..."

Option 2 — File fallback (supported by some scripts):

<workspace>/scripts/apify-api-token.txt
<workspace>/scripts/jina-api-key.txt
<workspace>/scripts/parallel-api-key.txt
<workspace>/scripts/brightdata-mcp-url.txt

How It Works

Research Phases

Phase 0: Tooling Self-Check      → diagnose.sh, check available tools
Phase 1: Seed Collection         → parallel search across all engines
Phase 1.5: Internal Intelligence → Telegram, email, vault (BEFORE external sources)
Phase 2: Platform Extraction     → LinkedIn, Instagram, Facebook, TikTok, YouTube...
Phase 3: Cross-Reference         → facts verified, graded A/B/C/D
Phase 4: Psychoprofile           → MBTI, Big Five, communication style
Phase 5: Completeness Check      → 9 mandatory checks + Depth Score 1-10
Phase 6: Dossier Output          → formatted dossier from template

Research Escalation (cheap → expensive)

Level 1: Quick Answers      → Perplexity Sonar, Brave, Tavily, Exa    (~$0.00)
Level 2: Source Verification → Jina read, Parallel extract             (~$0.01)
Level 3: Social Media        → Apify scrapers, Bright Data             (~$0.01-0.10)
Level 4: Deep Research       → Perplexity Deep, Exa Deep, Jina Deep   (~$0.05-0.50)

Embedded Scripts

| Script | Purpose | |--------|---------| | diagnose.sh | Self-diagnostics for all tools and APIs | | perplexity.sh | search / sonar / deep research | | tavily.sh | search / deep / extract | | exa.sh | search / company / people / crawl / deep | | first-volley.sh | Parallel search across all engines at once | | merge-volley.sh | Deduplicate and group search results | | apify.sh | LinkedIn / Instagram / any actor / store search | | run-actor.sh | Universal Apify runner (55+ actors, polling, CSV/JSON export) | | run_actor.js | Node.js engine powering run-actor.sh | | jina.sh | read URL / search / deepsearch | | parallel.sh | search / extract | | brightdata.sh | scrape / search / search-geo / search-yandex | | mcp-client.py | Lightweight MCP client for Bright Data (stdlib only) |

Project Structure

osint/
├── SKILL.md                          # Main skill file (452 lines)
├── references/
│   ├── tools.md                      # Full catalog of 55+ Apify actors + all tools
│   ├── platforms.md                  # Platform-specific extraction guide
│   ├── content-extraction.md         # YouTube/podcast/blog extraction
│   └── psychoprofile.md              # MBTI/Big Five methodology
├── assets/
│   └── dossier-template.md           # Output dossier template
└── scripts/
    ├── diagnose.sh                   # Self-check
    ├── run-actor.sh                  # Universal Apify runner (bash wrapper)
    ├── run_actor.js                  # Apify runner engine (Node.js, embedded)
    ├── package.json                  # ESM support for run_actor.js
    ├── apify.sh                      # Apify shortcuts
    ├── perplexity.sh                 # Perplexity API
    ├── tavily.sh                     # Tavily API
    ├── exa.sh                        # Exa AI API
    ├── jina.sh                       # Jina AI API
    ├── parallel.sh                   # Parallel AI API
    ├── brightdata.sh                 # Bright Data MCP
    ├── mcp-client.py                 # MCP client (Python, stdlib only)
    ├── first-volley.sh               # Parallel first search
    └── merge-volley.sh               # Result merging

Known Issues (Beta)

• Shell injection: user input is interpolated into JSON without jq escaping. Do not run with untrusted input.
• macOS: first-volley.sh uses tail --pid (Linux-only). Parallel searches work on macOS, but timeout logic may not trigger.
• Apify actors: actor IDs can change or get removed without notice. Use apify.sh store-search to find alternatives.
• Inconsistent key loading: Perplexity, Tavily, and Exa only load from env vars (no file fallback, unlike Apify/Jina/Parallel).

Credits

• Apify Actor Runner (run_actor.js) embedded from apify/agent-skills (MIT License)
• Actor catalog based on apify-ultimate-scraper v1.3.0

License

MIT