<div align="center">

Jarvis AI Assistant

Your Personal Voice-Controlled AI Companion

<img src="https://raw.githubusercontent.com/vannu07/jarvis/main/frontend/assets/img/logo.ico" alt="Jarvis Logo" width="150"/>

A production-ready voice assistant with facial recognition authentication, built on modern Python architecture and web technologies.

Features • Installation • Usage • Documentation • Contributing

---

<img src="https://user-images.githubusercontent.com/74038190/212284100-561aa473-3905-4a80-b561-0d28506553ee.gif" width="700"> </div>

Overview

Jarvis is an intelligent voice assistant that combines speech recognition, natural language processing, and computer vision to provide a seamless user experience. The system features biometric authentication, hotword detection, and extensive integration with popular platforms.

<div align="center">

Key Features

| Voice Control | Face Recognition | Hotword Detection | Web Integration | |:---:|:---:|:---:|:---:| | Advanced speech-to-text | Secure biometric auth | Always-on wake word | Modern responsive UI |

</div>

Core Capabilities

<table> <tr> <td width="50%">

Voice & AI

• Real-time Speech Recognition using Google STT
• Natural Language Processing with Hugging Face
• Text-to-Speech with customizable voices
• Audio Visualization in real-time
• Wake Word Detection ("Jarvis", "Alexa")

</td> <td width="50%">

Smart Integrations

• WhatsApp Automation (messages, calls, video)
• YouTube Control via voice commands
• System Control (apps, windows, shortcuts)
• Contact Management with voice lookup
• Web Browsing through voice
• Weather Forecasts via OpenWeatherMap API

</td> </tr> </table>

---

<div align="center">

Technology Stack

Backend Technologies

Frontend Technologies

AI & ML

Tools & Libraries

</div>

---

<div align="center">

System Architecture

graph TD
    A[Web Frontend] -->|Eel Bridge| B[Main Process]
    B --> C[Speech Recognition]
    B --> D[Face Authentication]
    B --> E[Hotword Detection]
    C --> F[Command Parser]
    F --> G[Feature Handlers]
    G --> H[SQLite Database]
    G --> I[WhatsApp Integration]
    G --> J[YouTube Control]
    G --> K[AI Chatbot]

    %% Consistent style for all nodes
    style A fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style B fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style C fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style D fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style E fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style F fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style G fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style H fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style I fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style J fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121
    style K fill:#ede7f6,stroke:#4a148c,stroke-width:1px,color:#212121

</div>

---

Prerequisites

<table> <tr> <td width="50%">

System Requirements

OS: Windows 10/11, Linux, macOS
Python: 3.10+
RAM: 4GB minimum
Storage: 500MB free space

</td> <td width="50%">

Hardware

Microphone: Required for voice input
Webcam: Required for face recognition
Internet: Active connection needed
Audio Output: Speakers/Headphones

</td> </tr> </table>

---

Installation

Step 1: Clone Repository

git clone https://github.com/vannu07/jarvis.git
cd jarvis

Step 2: Setup Virtual Environment

<table> <tr> <td width="50%">

Windows

python -m venv venv
venv\Scripts\activate

</td> <td width="50%">

Linux/Mac

python3 -m venv venv
source venv/bin/activate

</td> </tr> </table>

Step 3: Install Dependencies

pip install -r requirements.txt

Step 4: Configure Environment

Create a .env file:

# API Keys
HUGGINGFACE_TOKEN=your_token_here
PORCUPINE_ACCESS_KEY=your_key_here
NEWSAPI_KEY=your_newsapi_key
OPENWEATHERMAP_API_KEY=your_openweathermap_api_key

# Voice Settings
TTS_RATE=150
TTS_VOICE=0

# Recognition Settings
FACE_CONFIDENCE_THRESHOLD=50
HOTWORD_SENSITIVITY=0.5

Step 5: Train Face Recognition (Optional)

python backend/auth/trainer.py

<div align="center">

Quick Start

python run.py

Jarvis will launch at http://localhost:8000

</div>

---

Usage

Voice Commands

<table> <tr> <td width="33%">

System Control

Jarvis, open Chrome
Jarvis, launch VS Code
Jarvis, close window
Jarvis, shutdown computer

</td> <td width="33%">

Media Control

Jarvis, play Metallica
Jarvis, pause video
Jarvis, next song
Jarvis, volume up

</td> <td width="33%">

Communication

Jarvis, message John
Jarvis, call Sarah
Jarvis, video call Mike
Jarvis, open WhatsApp

</td> </tr> <tr> <td width="33%">

Weather & Information

Jarvis, weather in London
Jarvis, forecast for Tokyo
Jarvis, weather in New York
Jarvis, forecast Paris

</td> <td width="66%" colspan="2">

AI Assistant

Jarvis, what is the capital of France?
Jarvis, tell me a joke
Jarvis, help me with Python

</td> </tr> </table>

Keyboard Shortcuts

<div align="center">

| Shortcut | Action | |:--------:|:------:| | Win + J (Windows) | Manual Activation | | Cmd + J (macOS) | Manual Activation | | Ctrl + Q | Quit Application | | F11 | Fullscreen Toggle |

</div>

Wake Words

Say "Jarvis" or "Alexa" followed by your command

---

Weather Feature

Jarvis integrates with OpenWeatherMap API to provide real-time weather updates and forecasts.

Setup OpenWeatherMap API

1. Sign up for a free API key at OpenWeatherMap
2. Add your API key to the .env file:

   OPENWEATHERMAP_API_KEY=your_api_key_here
   

Weather Commands

Current Weather:

Jarvis, weather in London
Jarvis, what's the weather in Tokyo
Jarvis, weather for New York

Weather Forecast (3-5 days):

Jarvis, forecast for Paris
Jarvis, forecast in Mumbai
Jarvis, weather forecast for Berlin

Features

• ✅ Current temperature, feels-like temperature, and conditions
• ✅ Humidity, wind speed, and atmospheric pressure
• ✅ 3-5 day weather forecast with daily min/max temperatures
• ✅ Graceful handling of invalid city names
• ✅ Clean console output with detailed information
• ✅ Voice responses for hands-free operation

Standalone Usage

You can also use the weather module independently:

python weather_fetcher.py

Then use commands like:

• weather London - Get current weather
• forecast Tokyo - Get 5-day forecast
• exit - Quit the application

Testing

Run the weather module tests:

python -m testing.weather_test

---

Project Structure

jarvis/
├── backend/
│   ├── auth/
│   │   ├── haarcascade_frontalface_default.xml
│   │   ├── recognize.py        # Face recognition
│   │   ├── trainer.py          # Model training
│   │   └── trainer/            # Trained models
│   ├── command.py              # Command parser
│   ├── config.py               # Configuration
│   ├── db.py                   # Database ops
│   ├── feature.py              # Feature handlers
│   └── helper.py               # Utilities
├── frontend/
│   ├── assets/
│   │   ├── audio/              # Sound files
│   │   ├── img/                # Images & icons
│   │   └── vendor/             # Third-party libs
│   ├── index.html              # Main UI
│   ├── style.css               # Styles
│   ├── script.js               # Parti