MCPMacControl

Give your AI eyes and hands on your Mac.

MCPMacControl is an MCP server that turns any AI assistant into an agentic computer user — it can see your screen, move the mouse, type on the keyboard, and run shell commands across any macOS application, not just the browser.

Works with Claude Code, Cursor, Windsurf, and any MCP-compatible client.

No scripting. No AppleScript. Just tell the AI what you want in plain language and watch it drive your Mac.

What can your AI do with it?

• Test your app like a QA engineer — click through flows, fill forms, verify visual state
• Automate legacy software that has no API — navigate menus, extract data from any window
• Debug UI issues by looking at screenshots and reproducing the problem step by step
• Drive native apps — Xcode, Figma, Finder, Mail, Excel — anything with a window
• Run interactive terminal sessions — full PTY with TUI support (vim, htop, docker logs)

"Take a screenshot of Safari, click the search bar, and type anthropic.com"

That's a real prompt. The AI does the rest.

Install

Download (recommended)

Download MCPMacControl.app.zip from the latest release, unzip, and move to /Applications/.

Release builds are signed and notarized with an Apple Developer ID certificate, so macOS permissions persist across updates and no Gatekeeper warnings appear.

Build from source

git clone https://github.com/sstraus/mcpmaccontrol
cd mcpmaccontrol
make build
make install

Configure your MCP client

Add to your MCP client's config (e.g. ~/.claude.json for Claude Code):

{
  "mcpServers": {
    "mac-control": {
      "command": "/Applications/MCPMacControl.app/Contents/MacOS/mcpmaccontrol"
    }
  }
}

Then ask your AI: "Take a screenshot of Safari and click the search box"

Why not Playwright / browser automation?

Browser automation tools only control browsers. MCPMacControl controls everything on your Mac — native apps, Electron apps, system dialogs, menu bars, the Dock. If it has pixels on screen, Claude can see it and interact with it.

It also includes full PTY shell sessions, so Claude can run vim, docker, ssh, or any interactive terminal tool — not just headless commands.

Why an .app bundle?

Unlike most MCP servers, MCPMacControl needs macOS Screen Recording and Accessibility permissions. macOS grants these permissions per-application, not per-binary. When a plain binary runs from a terminal, macOS attributes the permission to the terminal app (iTerm, Terminal.app), not to the binary itself. This means:

• Permissions don't persist when the binary is rebuilt
• Every terminal app that launches the binary needs separate permission grants
• There's no way for users to manage the binary's permissions in System Settings

Wrapping the binary in a signed .app bundle gives it its own identity in macOS privacy settings. The binary still communicates via stdio like any MCP server - the .app bundle is just how macOS identifies it for permission management.

That's why the command path is /Applications/MCPMacControl.app/Contents/MacOS/mcpmaccontrol instead of /usr/local/bin/mcpmaccontrol.

Permissions

On first launch, macOS will prompt for two permissions:

| Permission | Purpose | |------------|---------| | Screen Recording | Screenshot capture | | Accessibility | Mouse/keyboard control |

Grant both in System Settings > Privacy & Security. The app registers itself automatically.

Features

• Visual feedback — menu bar icon, native popover showing current operation, sound + orange border flash before automation
• Input safety — verifies the target app is focused before sending keystrokes, preventing input from reaching the wrong window
• App context inheritance — focus propagates to subsequent actions in a batch, so you write app once
• Signed .app bundle — macOS permissions persist across updates, no Gatekeeper warnings
• Region capture — screenshot only what you need to save tokens
• Auto-exit when the AI client disconnects

Tools

| Tool | Description | |------|-------------| | help | Built-in documentation - call first! | | list_windows | Find windows by app name | | capture_window | Screenshot window or region with click coordinates | | capture_screen | Screenshot entire screen | | do | Execute actions: click, type, key, scroll, focus, minimize, etc. | | shell | PTY shell sessions: spawn, send_input, get_snapshot, resize, close, list | | processes | List running processes with filtering for debugging |

Built-in Help

The AI calls help() to learn the API:

• help() → Overview and workflow
• help("actions") → All action types for do()
• help("shell") → Shell/PTY documentation
• help("examples") → Usage examples

How it works

Claude operates in a see-think-act loop — the same way a human would use a computer:

1. capture_window("Safari")         → Screenshot saved to temp file
2. [Claude reads the image]         → "I see a search bar at (400, 50)"
3. do([                             → Click, type, press Enter
     {type:"click", app:"Safari", x:400, y:50},
     {type:"type", text:"anthropic.com"},
     {type:"key", key:"enter"}
   ])
4. capture_window("Safari")         → Verify the result

Pixel coordinates in the screenshot map directly to do() action coordinates — no conversion needed.

Region capture

Capture a portion of a window to save tokens:

capture_window("Safari", region_x: 0, region_y: 0, region_width: 400, region_height: 300)

Click coordinates: click_x = region_x + x_in_image, click_y = region_y + y_in_image.

The do Tool

Execute one or more actions in sequence:

do({"actions": [
  {"type": "click", "app": "Safari", "x": 400, "y": 50},
  {"type": "type", "text": "anthropic.com"},
  {"type": "key", "key": "enter"}
]})

Action Types

| Action | Required | Optional | Description | |--------|----------|----------|-------------| | click | app, x, y | button, double | Click at position | | move | app, x, y | | Move mouse | | type | text | | Type text string | | key | key | modifiers | Press key (supports "cmd+shift+g" compound syntax) | | scroll | app, x, y, delta_y/delta_x | | Scroll in window | | wait | ms | | Pause (milliseconds) | | focus | app | | Bring window to front | | minimize | app | | Minimize to dock | | restore | app | | Restore from dock | | close | app | | Close window | | resize | app, width, height | | Resize window |

Examples

Click:

{"type": "click", "app": "Safari", "x": 100, "y": 50}
{"type": "click", "app": "Finder", "x": 200, "y": 100, "button": "right"}
{"type": "click", "app": "Finder", "x": 200, "y": 100, "double": true}

Keyboard:

{"type": "type", "text": "Hello World"}
{"type": "key", "key": "enter"}
{"type": "key", "key": "cmd+v"}
{"type": "key", "key": "cmd+shift+g"}
{"type": "key", "key": "v", "modifiers": ["cmd"]}

Scroll:

{"type": "scroll", "app": "Safari", "x": 400, "y": 300, "delta_y": -100}

Window Control:

{"type": "focus", "app": "Safari"}
{"type": "minimize", "app": "Finder"}
{"type": "restore", "app": "Finder"}
{"type": "resize", "app": "Safari", "width": 1024, "height": 768}
{"type": "close", "app": "TextEdit"}

Shell Sessions

Run interactive terminal sessions with full PTY and terminal emulation (supports TUI apps like vim, htop):

1. shell(action: "spawn")                                          → Get session ID
2. shell(action: "send_input", session_id: "ID", input: "ls")     → Send text
3. shell(action: "send_input", session_id: "ID", special_key: "enter")
4. shell(action: "get_snapshot", session_id: "ID", format: "ansi") → Get screen
5. shell(action: "close", session_id: "ID")                        → Cleanup

| Action | Parameters | Description | |--------|------------|-------------| | spawn | command?, cwd?, cols?, rows? | Start session (default: /bin/bash) | | send_input | session_id, input?, special_key?, wait_ms? | Send text or keys | | get_snapshot | session_id, format? | Get screen state (text or ansi) | | resize | session_id, cols?, rows? | Resize terminal | | list | | List active sessions | | close | session_id | Close session |

Headless Mode

Run without menu bar (for CI/SSH):

{
  "mcpServers": {
    "mac-control": {
      "command": "/Applications/MCPMacControl.app/Contents/MacOS/mcpmaccontrol",
      "env": {
        "MCPMACCONTROL_HEADLESS": "1"
      }
    }
  }
}

Image Optimization

Screenshots are optimized for AI vision:

• WebP lossy format (default quality 25, tunable 1-100 via quality parameter; use 50+ for small glyphs/icons)
• Scaled to window point dimensions (coordinates match clicks)
• Use region capture to further reduce size and tokens

Development

make build          # Build signed .app bundle
make test           # Run tests
make install        # Install to /Applications
make clean          # Remove build artifacts
make verify-sign    # Check code signature

To sign with your own Apple Developer certificate:

SIGN_IDENTITY="Developer ID Application: YOUR NAME (TEAMID)" make build

Requirements

• macOS 12+
• Go 1.24+ (for building)

License

MIT