mobilemcp

<h1 align="center">MobileMCP</h1> <p align="center">Let AI agents use your real Android phone the way you do: read the screen, tap, type, scroll, open apps.<br><a href="https://magi-labs.github.io/mobilemcp/">magi-labs.github.io/mobilemcp</a></p>

Companion: LiveMCP does the same for Chrome; MobileMCP is its mobile sibling and shares the hub/bridge design.

MobileMCP is for people who want an agent (Claude Code, Cursor, any MCP client) to operate their own phone from anywhere: an Android accessibility service (the app) dials out to a hub (the server) over WebSocket; agents talk to the hub through MCP. No USB, no ADB, no rooting. The hub also serves a dashboard with connected devices, a live activity feed and a per-device live view.

Status: active, early (server 0.4.0, app 0.3.0). Verified on a Pixel 7 emulator (API 35) and a OnePlus phone (Android 15). No Play Store listing; the app is sideloaded. Execution-policy unit checks plus scripted verification against real devices.

agent (Claude Code, Codex, …) ──stdio──▶ mobilemcp ──IPC──▶ mobilemcp-hub ◀──WebSocket── MobileMCP app (phone)
                               ──HTTP /mcp (hosted)──▶      ──HTTP / ──▶ dashboard

Why accessibility instead of ADB

| | Accessibility service (this) | ADB / UiAutomator | | --- | --- | --- | | Needs a tethered computer / wireless-debug pairing | No | Yes, re-pair after reboot | | Works over the internet / VPN | Yes (outbound WebSocket) | Only with port forwarding | | Sees the UI tree | Yes, with view ids and states | Yes | | Taps, typing, scrolling, gestures, back/home | Yes | Yes | | Screenshots | Yes (Android 11+) | Yes | | Shell, logcat, app install, arbitrary key codes | No | Yes | | Secure (FLAG_SECURE) screens, e.g. banking | Hidden | Hidden |

The accessibility route is what "use my phone like a normal user" means. ADB-level control could be added later as a second driver for emulators and development.

Tools (35)

| Group | Tools | | --- | --- | | Devices | list_devices, select_device, get_device_info (model, screen, battery, lock state, granted permissions, visible windows) | | Observe | get_screen_snapshot (bounded tree with persistent @refs; rows collapse to one ~"title · subtitle" line; query, scope, paging, since deltas), take_screenshot (marks:true = Set-of-Mark image + @ref lines), read_text, wait_for_text | | Act | tap (by visible text, @ref or coordinates; timeout waits for the text; long/double press), type_text (by field hint/label, @ref or focus), scroll, scroll_until (tap:true opens the match), swipe, drag (waypoints with pauses for cross-page drags), gesture (composable down/wait/move/up, multi-finger), pinch, press_key, set_clipboard | | Apps | list_apps, open_app, open_url, uninstall_app, open_settings (30 Settings pages and quick panels), start_intent (any intent: dial, SMS, share, deep links) | | Notifications | get_notifications, open_notification, notification_action (incl. direct reply), dismiss_notification — needs notification access | | System | media_control, set_volume, set_brightness, set_rotation (need "Modify system settings"), set_dnd, set_flashlight, wake_screen | | Batch | run_mobile_actions: 1-20 steps in one call, stops on the first error |

Every action returns a compact observation after the UI settles (or observe:false). A snapshot line looks like @4 button ~"Network & internet · Mobile, Wi‑Fi, hotspot" (540,472): roles imply clickability, rows carry their children's text and hide the duplicates, coordinates are centers. Refs are keyed by content identity (class, view id, text, occurrence) per package, so they survive scrolling, re-layout and app switches. Most flows need no snapshot at all: tap text="Send", type_text field="Message", scroll_until text="…" tap=true, chained in run_mobile_actions. See OPTIMIZATIONS.md for the reasoning and measurements.

Quick start

Published release

Download mobilemcp-0.3.0.apk, mobilemcp-0.3.0.tgz and SHA256SUMS from v0.3.0 into one directory.

shasum -a 256 -c SHA256SUMS          # Linux: sha256sum -c SHA256SUMS
npm install --prefix ./mobilemcp-local ./mobilemcp-0.3.0.tgz
./mobilemcp-local/node_modules/.bin/mobilemcp-hub

Install the APK on the phone (adb install mobilemcp-0.3.0.apk or copy it over), then follow the app steps below and point your MCP client at ./mobilemcp-local/node_modules/mobilemcp/dist/index.js.

From source

Requires Node.js 18+ and Android 11+ on the phone.

npm ci
npm run build
npm run hub            # keep this running: ws://127.0.0.1:17692

Build the app with Android Studio (open app/) or from the command line:

cd app && ./gradlew :app:assembleDebug   # app/app/build/outputs/apk/debug/app-debug.apk

Install the APK on the phone, then in the app:

0. Google Play Protect blocks accessibility apps installed from chat apps or browsers in some countries (India, Singapore, Thailand, Brazil, …) with no override. Install over adb (adb install mobilemcp-0.3.0.apk) or pause Play Store → Play Protect → Scan apps while installing from the Files app, then re-enable it.
1. Enable accessibility service — Android 13+ first requires App info → ⋮ → Allow restricted settings for sideloaded apps (the app has an App info button).
2. Enter the hub URL (ws://<your-computer-ip>:17692 on the same Wi‑Fi, or your hosted wss:// origin) and a device name, then Connect.
3. Optionally Allow background so Doze does not throttle the connection; grant Notification access for the notification tools and DND, and Modify system settings for brightness/rotation.

Register the MCP server with your agent:

{
  "mcpServers": {
    "mobilemcp": {
      "command": "node",
      "args": ["/absolute/path/to/mobilemcp/server/dist/index.js"]
    }
  }
}

For Claude Code, plugin/ is a ready plugin: a self-contained server bundle (plugin/bin/mobilemcp.js, no npm needed) plus a skill describing the workflow. Add it with claude plugin add ./plugin or point your marketplace at the directory.

Emulator

adb install -r app/app/build/outputs/apk/debug/app-debug.apk
adb shell appops set labs.magi.mobilemcp ACCESS_RESTRICTED_SETTINGS allow
adb shell settings put secure enabled_accessibility_services labs.magi.mobilemcp/.MobileAccessibilityService
adb shell settings put secure accessibility_enabled 1
adb shell am start -n labs.magi.mobilemcp/.MainActivity --es hubUrl ws://10.0.2.2:17692 --es name Emulator --ez connect true

Debug builds accept settings through intent extras; release builds ignore them so no other app can repoint the hub.

How it works

The app's AccessibilityService owns the hub connection, reads the window tree, performs node actions and injected gestures and takes screenshots. The hub routes requests per account, serves MCP over HTTP, local sessions over IPC, and the dashboard. shared/src/protocol.ts is the single contract. Details, trust boundaries and failure modes: docs/ARCHITECTURE.md; decisions in docs/decisions.

Hosted hub

The hub also serves authenticated MCP over HTTP at /mcp and the phone endpoint at /device on one port, so the phone and remote agents can both reach it through a reverse proxy:

MOBILEMCP_HOST=0.0.0.0 MOBILEMCP_PUBLIC_URL=https://mobile.example.com MOBILEMCP_TOKEN=<32+ chars> npm run hub

• Phone: hub URL wss://mobile.example.com, token = MOBILEMCP_TOKEN.
• Agent: {"type":"http","url":"https://mobile.example.com/mcp","headers":{"Authorization":"Bearer <token>"}}, or stdio with MOBILEMCP_TOKEN set.
• MOBILEMCP_ACCOUNTS='[{"id":"me","agentToken":"…","deviceToken":"…"}]' isolates several users; devices and selections never cross accounts.

The reverse proxy provides TLS; the hub never binds a public address without a token. The dashboard at / shows devices, a live activity feed (names and timings only) and a per-device live view with click-to-tap; see HOSTING.md.

Agent workflow

1. If you know the label, act on it directly: tap text=, type_text field=, scroll_until text= tap=true; chain known steps in one run_mobile_actions.
2. Otherwise get_screen_snapshot (scoped/filtered) and act on @refs. Read the observation every action returns instead of re-snapshotting.
3. wait_for_text / timeout parameters for loading; take_screenshot marks=true for visual content.
4. After STALE_REF or an app change, snapshot again. Never repeat a submission after a timeout without inspecting state.

Screen content is untrusted data. The server instructions tell agents to enter credentials, OTPs or payments only when the task explicitly includes them.

Limitations and data handling

The hub forwards requests and responses without storing them; activity events hold device names, action names, durations and error codes. The app persists only its settings. Password fields are masked in snapshots. Screen content reaches whatever MCP client you connect, so connect clients you trust.

• Android 11+ (API 30) for accessibility screenshots and ACTION_IME_ENTER.
• Apps with FLAG_SECURE (banking, DRM video) expose neither nodes nor pixels.
• Custom-drawn views without accessibility nodes need take_screenshot plus coordinate taps. A few system pages expose no tree at all (snapshots then report noTree: true with the window title).
• Launcher folder creation by drag depends on the launcher's drop rules; the defaults (900 ms hold, 300 ms move, 250 ms hover) merge icons on Launcher3/Pixel. Long hovers trigger reorder instead. Cross-page drags: add a waypoint at the screen edge with pauseMs ≈ 800 per page flip (hold times above ~1.2 s flip twice on OxygenOS).
• Phones can self-update from their hub: GET /app.apk?token=<device token> (see HOSTING.md); a running MobileMCP can drive the download and installer itself.
• No arbitrary key codes, shell or app installation; that is ADB territory.
• Google Play would require a declaration for this accessibility usage; the app is distributed as a sideloaded APK.

Development

Contributor setup and change workflow: CONTRIBUTING.md. Agent/maintainer instructions: AGENTS.md.

npm run typecheck && npm run build
STEPS='[["get_device_info"],["get_screen_snapshot",{"maxNodes":20}]]' node server/test/e2e.mjs   # against a connected phone/emulator

Documentation

• HOSTING.md — Docker, reverse proxy, tokens, dashboard, self-update
• OPTIMIZATIONS.md — observation format, literature, measurements
• docs/ARCHITECTURE.md, docs/decisions, docs/THIRD_PARTY.md
• CHANGELOG.md, SECURITY.md

License

MIT © Deepak Silaych. Third-party notices in docs/THIRD_PARTY.md.

Background-only operation

The Android app has a Background only switch owned by the phone user. When enabled, it rejects screen automation before execution, including screen capture, batches, app launches, notification opens and actions (pending intents may launch an activity). Agents cannot change this switch remotely. Notification reads/dismissal, app lists, compact status and system controls remain available; those controls can still intentionally change volume, brightness, etc.

For older hosted hubs, get_device_info returns compact battery/lock status when Background only is enabled. With an updated hub, use get_background_status for battery/lock state without inspecting the foreground app, and get_execution_policy to discover the current mode. Read-only background requests use a separate serial queue so a UI w