Jarvis

A private AI voice assistant that lives on your computer - not in someone else's cloud. Leave it running 24/7 and treat it like a third person in the room: say "Jarvis" anywhere in your sentence ("what do you think, Jarvis?" or "Jarvis, look at this error") and get thoughtful, contextual responses. It remembers your preferences, helps with code, tracks your health goals, searches the web, and connects to 500+ tools via MCP.

🔒 100% local processing. No subscriptions. No data harvesting. Automatic redaction of sensitive info.

---

Support Jarvis

---

<p align="center"> <img src="docs/img/face.png" alt="Jarvis Face" width="400"> </p> <p align="center"> <img src="docs/img/memory-viewer-memories.png" alt="Memory Viewer - Conversations" width="400"> <img src="docs/img/memory-viewer-meals.png" alt="Memory Viewer - Nutrition" width="400"> </p>

Why Jarvis?

🔒 Your data stays yours - 100% local AI processing. No cloud, no subscriptions, no data harvesting. Automatic redaction of sensitive info. This is non-negotiable.

🗣️ A third person in the room - Unlike voice assistants that only respond to rigid commands, Jarvis understands conversations. It maintains a short temporary rolling context of what's being discussed, so when you ask "Jarvis, what do you think?" it knows exactly what you're talking about. Have it chime into discussions with friends, help debug code while you talk through problems, or weigh in on decisions.

🧠 Never forgets - Unlimited memory across conversations. Context-aware personalities (developer/business/life coach). Learns your preferences over time.

🔌 Extensible - MCP integration connects Jarvis to 500+ tools: smart home, GitHub, Slack, databases, and more.

📊 Transparent progress - We track what works (and what doesn't) with automated evals. See current accuracy →

🚧 Known limitations: Jarvis is under active development. Primary development happens on macOS. Windows/Linux support may lag behind. We're building in the open, issues and contributions welcome!

• Voice-only for now—no text chat interface yet (#35)
• No mobile apps (#17)
• "Stop" commands during speech sometimes get filtered as echo (#24)

<details> <summary><strong>See it in action</strong> (example conversations)</summary>

Chiming into conversations (the magic moment):

👤 Alice: I wonder what the weather will be like tomorrow
👤 Bob: Yeah, we should check before planning the picnic
👤 Alice: Jarvis, what do you think?

✨ Working on it: what do you think about the weather tomorrow for the picnic
🧰 Tool: getWeather…

🤖 Jarvis (life)
Tomorrow looks great for a picnic! Sunny with highs around 72°F...

Jarvis understood the entire conversation and gave a contextual answer - no need to repeat the question.

Debugging with screen analysis:

✨ Working on it: i'm getting this weird graphql error
🧰 Tool: screenshot…
🧰 MCP: vscode:findReferences…

🤖 Jarvis (developer)
I can see the error on your screen - "Cannot return null for non-nullable field".
Your User type has `email: String!` but the resolver is returning null...

Health-aware advice:

✨ Working on it: should i order pizza tonight considering my health goals
🧰 Tool: fetchMeals…

🤖 Jarvis (life)
Looking at your week - you mentioned wanting to lose 10 pounds by March...
Today you've had about 860 calories so far. Given your 1,800 calorie target,
you've got room for pizza! But maybe consider thin crust to save calories.

Personalized news search:

✨ Working on it: what are some news from today that might interest me?
  🧠 Looking back at our past conversations…
  🌐 Searching the web…

🤖 Jarvis (business)
Here's a quick snapshot of today's headlines...

Natural wake word placement (say "Jarvis" anywhere):

👂 Heard: give me a random topic to discuss jarvis
✨ Working on it: give me a random topic to discuss

🤖 Jarvis (life)
How about the Fermi Paradox? Given the vast number of stars...

</details>

Quick Install

1. Install Prerequisites

| Platform | Requirement | |----------|-------------| | All | Ollama | | Windows | Visual C++ Redistributable (most PCs already have this) |

2. Download Jarvis

Get the latest from GitHub Releases:

| Platform | Download | Run | |----------|----------|-----| | Windows | Jarvis-Windows-x64.zip | Extract → Run Jarvis.exe | | macOS | Jarvis-macOS-arm64.zip | Extract → Move to Applications → Right-click → Open | | Linux | Jarvis-Linux-x64.tar.gz | tar -xzf → Run ./Jarvis/Jarvis |

Click the system tray icon → Start Listening → Say "Jarvis" and talk!

<p align="center"> <img src="docs/img/setup-wizard-initial-check.png" alt="Setup - Initial Check" width="200"> <img src="docs/img/setup-wizard-model.png" alt="Setup - Model Selection" width="200"> <img src="docs/img/setup-wizard-whisper.png" alt="Setup - Whisper" width="200"> <img src="docs/img/setup-wizard-complete.png" alt="Setup - Complete" width="200"> </p> <p align="center"> <img src="docs/img/logs.png" alt="Real-time Logs" width="500"> </p>

Features

• Conversational Awareness - Understands ongoing discussions. Ask "Jarvis, what do you think?" and it knows what you're talking about. Works naturally in multi-person conversations.
• Unlimited Memory - Never forgets. Searches across all your conversation history. Memory Viewer GUI included.
• Smart Personalities - Adapts tone: Developer (debugging), Business (planning), Life Coach (health/wellness)
• Built-in Tools - Screenshot OCR, web search (with auto-fetch), weather, file access, nutrition tracking, location awareness
• Natural Voice - Say "Jarvis" anywhere in your sentence, interrupt with "stop", follow up without repeating the wake word
• MCP Integration - Connect to 500+ external tools (Home Assistant, GitHub, Slack, etc.)

System Requirements

| Hardware | RAM | Model | |----------|-----|-------| | Most users | 8GB+ | llama3.2:3b (default) | | High-end | 16GB+ | gpt-oss:20b |

The setup wizard will guide you through model selection and installation on first launch.

Configuration

Most users won't need to change anything. Config file: ~/.config/jarvis/config.json

<details> <summary><strong>Speech Recognition (Whisper)</strong></summary>

Language Modes

• English Only (default, better accuracy): "whisper_model": "small.en"
• Multilingual (99 languages): "whisper_model": "small"

Model Sizes

| Model | English | Multilingual | Size | VRAM | Speed | |-------|---------|--------------|------|------|-------| | Tiny | tiny.en | tiny | ~75MB | ~1GB | ~10x | | Base | base.en | base | ~150MB | ~1GB | ~7x | | Small | small.en | small | ~500MB | ~2GB | ~4x | | Medium | medium.en | medium | ~1.5GB | ~5GB | ~2x | | Large V3 Turbo | - | large-v3-turbo | ~1.6GB | ~6GB | ~8x |

Speed is relative to original large model. Source

</details> <details> <summary><strong>Voice Interface (Advanced)</strong></summary>

LLM Intent Judge - Jarvis uses llama3.2:3b for intelligent voice intent classification (echo detection, query extraction, stop commands). This model is automatically installed alongside your chosen chat model during setup. The intent judge cannot be disabled but gracefully falls back to simpler text matching if Ollama is unavailable.

Audio-Level Wake Word Detection (optional) - More precise wake word timing using openWakeWord:

{
  "audio_wake_enabled": true,
  "audio_wake_threshold": 0.5
}

Requires openWakeWord library. Falls back to text detection if unavailable.

</details> <details> <summary><strong>Text-to-Speech</strong></summary>

Piper TTS (default) - Neural TTS that auto-downloads on first use (~60MB):

• Works out of the box - no setup required
• High-quality British English male voice (en_GB-alan-medium)
• Fast local synthesis with exact duration tracking

To use different Piper voices, download from HuggingFace and set:

{
  "tts_piper_model_path": "~/.local/share/jarvis/models/piper/en_GB-alan-medium.onnx"
}

Chatterbox - AI voice with emotion control (requires running from source):

{ "tts_engine": "chatterbox" }

Voice cloning with Chatterbox - add a 3-10 second .wav sample:

{
  "tts_engine": "chatterbox",
  "tts_chatterbox_audio_prompt": "/path/to/voice.wav"
}

</details> <details> <summary><strong>MCP Tool Integration</strong></summary>

Connect Jarvis to external tools via MCP servers:

{
  "mcps": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": { "GITHUB_TOKEN": "your-token" }
    }
  }
}

Popular integrations:

• Home Assistant - Voice control for smart home
• Google Workspace - Gmail, Calendar, Drive, Docs
• GitHub - Issues, PRs, workflows
• Notion - Knowledge management
• Slack/Discord - Team communication
• Databases - MySQL, PostgreSQL, MongoDB
• Composio - 500+ apps in one integration

See full MCP setup guide below.

</details>

MCP Integrations

<details> <summary><strong>Home Assistant</strong> - Smart home voice control</summary>

1. Add MCP Server integration