Shadow (Case Study)

An open-source background coding agent. Designed to understand, reason about, and contribute to existing codebases. Licensed for open-source use under MIT License

Sets up isolated execution environments for AI agents to work on GitHub repositories with tools to understand code, edit files, and much more.

Agent Environment (The Shadow Realm)

GitHub repository integration with branch management
Pull request generation with AI-authored commits
Real-time task status tracking and progress updates
Automatic workspace setup and cleanup on Micro-VMs
Kata QEMU containers for hardware-level isolation

Code Generation & Understanding

Multi-provider LLM support (Anthropic, OpenAI, OpenRouter)
Streaming chat interface with real-time responses
Tool execution with file operations, terminal commands, and code search
Memory system for repository-specific knowledge retention
Semantic code search, background processing
Lightweight Shadow Wiki generation for comprehensive codebase documentation
Custom rules for Shadow code generation

Execution Modes

Shadow supports two execution modes through an abstraction layer:

Local Mode

Direct filesystem execution on the host machine

Remote Mode (For Deployment)

Hardware-isolated execution in Kata QEMU containers
True VM isolation via QEMU hypervisor
Kubernetes orchestration with bare metal nodes

Mode selection is controlled by NODE_ENV and AGENT_MODE environment variables.

Development Setup

Repository Structure

Frontend (apps/frontend/) - Next.js application with real-time chat interface, terminal emulator, file explorer, and task management
Server (apps/server/) - Node.js orchestrator handling LLM integration, WebSocket communication, task initialization, and API endpoints
Sidecar (apps/sidecar/) - Express.js service providing REST APIs for file operations within isolated containers
Website (apps/website/) - Marketing and landing page
Database (packages/db/) - Prisma schema and PostgreSQL client with comprehensive data models
Types (packages/types/) - Shared TypeScript type definitions for the entire platform
Command Security (packages/command-security/) - Security utilities for command validation and sanitization
ESLint Config (packages/eslint-config/) - Shared linting rules
TypeScript Config (packages/typescript-config/) - Shared TypeScript configurations

Prerequisites

Node.js 22
PostgreSQL

Installation

Clone the repository and install dependencies:

git clone <repository-url>
cd shadow
npm install

Set up environment variables:

# Copy example environment files
cp apps/server/.env.template apps/server/.env
cp apps/frontend/.env.template apps/frontend/.env
cp packages/db/.env.template packages/db/.env

Configure the database:

# Create local PostgreSQL database
psql -U postgres -c "CREATE DATABASE shadow_dev;"

# Update packages/db/.env with your database URL
DATABASE_URL="postgres://postgres:@127.0.0.1:5432/shadow_dev"

# Generate Prisma client and push schema
npm run generate
npm run db:push

Start development servers:

# Start all services
npm run dev

# Or start specific services
npm run dev --filter=frontend
npm run dev --filter=server
npm run dev --filter=sidecar

Environment Configuration

Set variables in the following files:

Frontend: apps/frontend/.env.local
Server: apps/server/.env
Database: packages/db/.env

Quick start (local, no GitHub App install)

Use a personal GitHub token so the GitHub selector works instantly without installing our app.

Create a GitHub Personal Access Token with scopes: repo, read:org.
Add env vars:

An easy way to do this is run the ./setup-script.sh which will take in your input variables and autoset them in the right places for you! If you would like to do it manually, follow the instructions below

apps/server/.env

# Required
DATABASE_URL="postgres://postgres:@127.0.0.1:5432/shadow_dev"
BETTER_AUTH_SECRET="dev-secret"

GITHUB_PERSONAL_ACCESS_TOKEN=ghp_xxx

# Local mode
NODE_ENV=development
AGENT_MODE=local

# Optional: Pinecone for semantic search
PINECONE_API_KEY="" # TODO: Set this to your Pinecone API key
PINECONE_INDEX_NAME="shadow"

# Workspace directory for local agent:
WORKSPACE_DIR= # TODO: Set this to your local workspace directory

apps/frontend/.env.local

# Point frontend to your server if needed
NEXT_PUBLIC_SERVER_URL=http://localhost:4000

# Marks environment; any value other than "production" enables local behavior
NEXT_PUBLIC_VERCEL_ENV=development

GITHUB_PERSONAL_ACCESS_TOKEN=ghp_xxx
GITHUB_CLIENT_ID=
GITHUB_CLIENT_SECRET=

# Optional (only if you want OAuth login locally)
BETTER_AUTH_SECRET=dev-secret

packages/db/.env

DATABASE_URL="postgres://postgres:@127.0.0.1:5432/shadow_dev"
DIRECT_URL="postgres://postgres:@127.0.0.1:5432/shadow_dev"

With GITHUB_PERSONAL_ACCESS_TOKEN set on the server and NEXT_PUBLIC_VERCEL_ENV not equal to production, the backend uses your PAT for repo/branch/issue queries. The frontend's GitHub selector works immediately.

Development Commands

Linting and Formatting

# Lint all packages and apps
npm run lint

# Format code with Prettier
npm run format

# Type checking
npm run check-types

Database Operations

# Generate Prisma client from schema
npm run generate

# Push schema changes to database (for development)
npm run db:push

# Reset database and push schema (destructive)
npm run db:push:reset

# Open Prisma Studio GUI
npm run db:studio

# Run migrations in development
npm run db:migrate:dev

Building and Deployment

# Build all packages and apps
npm run build

# Build specific app
npm run build --filter=frontend
npm run build --filter=server
npm run build --filter=sidecar

Tool System

Shadow provides a comprehensive set of tools for AI agents:

File Operations

read_file - Read file contents with line range support
edit_file - Write and modify files
search_replace - Precise string replacement
delete_file - Safe file deletion
list_dir - Directory exploration

Code Search

grep_search - Pattern matching with regex
file_search - Fuzzy filename search
semantic_search - AI-powered semantic code search

Terminal & Execution

run_terminal_cmd - Command execution with real-time output
Command validation and security checks

Task Management

todo_write - Structured task management
add_memory - Repository-specific knowledge storage
list_memories - Retrieve stored knowledge

Development Guidelines

Code Organization

TypeScript throughout with strict type checking
Shared configurations via packages
Clean separation between execution modes
WebSocket event compatibility across frontend/backend

Security

Command validation in all execution modes
Path traversal protection
Workspace boundary enforcement
Container isolation in remote mode

Important Notes

Always test both local and remote modes for production features
Keep initialization steps mode-aware and properly abstracted
Maintain WebSocket event compatibility across frontend/backend changes
Remote mode requires Amazon Linux 2023 nodes for Kata Containers compatibility

Contributing

Fork the repository
Create a feature branch
Make your changes with proper TypeScript types
Test in both local and remote modes
Submit a pull request

We're excited to see what you've built with Shadow!

Ishaan Dey — X

Rajan Agarwal — X

Elijah Kurien — X

Shadow

Install / Use

README