English | 简体中文

<img src="docs/assets/logo.svg" width="88" alt="" align="right">

mobile-dev-harness

Precise verification for coding agents on Android: a verdict with evidence from a real device, in fewer tokens than adb with screenshots or mobile-mcp.

When an agent changes a web app, it can open a browser, click around, read the console and see whether the change works. When it changes a mobile app, it usually can't: it edits code and hopes. mobile-dev-harness (command: mdh) gives agents eyes and hands on a real device or emulator, designed around how agents work:

• Compact screens. The current screen as a short tree of elements (~150 tokens instead of thousands for raw XML), each with a ref like e12 that stays the same for the whole session.
• Act, then see what changed. Every action waits until the UI has settled and reports only the difference.
• Crashes surface immediately. New errors, crashes, native crashes and ANRs come with every result, with the app's own stack frames and the steps that led there.
• Fast. A warm on-device helper reads the UI in ~10 ms (uiautomator takes ~2 s) and types any Unicode text.
• Knows what a change reaches. mdh impact reads the uncommitted change and the project's Kotlin, Java and resource files — no device, no build, ~150 ms on a 300-file app — and lists the screens it affects, how to reach each one, the call sites of changed signatures and what to verify.
• Verdicts, not impressions. mdh verify checks the app (enabled id=sign_in, screen .MessagesActivity, always no crash) and answers pass or fail with what it observed and a screenshot on disk; what the agent did can be saved as a flow and replayed from a clean start, with JUnit for CI.
• Performance with numbers, not hunches. mdh perf measures cold start, janky frames, memory and CPU over repeated runs against a baseline from the same device, fails only on changes larger than the noise, and says what got slow from a Perfetto trace (performCreate:MainActivity 509 ms → slowInit 451 ms).
• Compatibility from the change, not a device farm. mdh compat reads what the change touches — an SDK_INT branch, a raised targetSdk, a layout-sw600dp, saved state, background work — names the risks on other Android versions, device types, vendor ROMs and screen sizes, and verifies each on the fewest configurations that show it (a tablet or landscape on the same emulator first). What it can't verify here, such as a Xiaomi ROM, it says so instead of passing.
• Build errors an agent can act on. mdh run builds with Gradle, installs only what changed and restarts the app; compiler, resource, manifest and dependency failures come back as file:line with the offending line.
• One engine, two interfaces. A CLI for humans, scripts and shell-based agents, and an MCP server for agents such as Claude Code.

Status: early development. Android only for now; it already drives real apps end to end. See Status and roadmap.

Why agents get better at mobile with it

Models reason well; they perceive and time poorly. On mobile, an agent's mistakes are rarely in the code it writes and mostly in what it believes about the running app: it calls a clean compile "done", reads a screenshot taken mid-transition, misses the crash in logcat, or checks the button it changed but not the screen it broke. mdh replaces eyeballing a screen with structured facts and deterministic checks:

• Built to verify, not only to control: with adb and screenshots, or a device-control server such as mobile-mcp, an agent can look and tap, and whether the change works remains its impression. With mdh it knows every screen the change reaches, sees settled states, every side effect and every crash, and gets a pass or fail with what was observed: "it works" becomes something that was checked.
• Fewer tokens than those tools: a screen in ~150 tokens instead of a ~1,500-token screenshot or thousands of tokens of XML, and only the diff after each action — so verifying after every change is affordable.

What it looks like

$ mdh observe
screen dev.mdh.sample/.LoginActivity  1344x2992
[e69] textbox "Email" empty #email
[e70] textbox "Password" empty #password
[e71] checkbox "Remember me" unchecked #remember
[e72] button "SIGN IN" disabled #sign_in

$ mdh type "alice@example.com" --into Email
$ mdh type "correct-horse" --into Password
type •••• into e70 textbox "Password" → ok (783 ms)
screen dev.mdh.sample/.LoginActivity  1344x2992  keyboard
~ [e70] textbox "Password": value empty → ••••
~ [e72] button "SIGN IN": disabled → enabled

A toggle shows its side effects, not just itself:

$ mdh tap "Airplane mode"
tap e20 switch "Airplane mode" → ok (923 ms)
screen com.android.settings/.SubSettings  1344x2992
~ [e18] item "Internet": detail "AndroidWifi" → "Airplane mode is on"
~ [e19] item "SIMs": enabled → disabled
~ [e20] switch "Airplane mode": off → on

Before verifying, the agent learns what its change reaches — here, a signature change breaks a caller on another screen, and the login screen no longer goes to the messages list:

$ mdh impact
impact vs HEAD (b7197c3): 2 files changed · 2 declarations
changed
  ~ LoginActivity.onCreate  body  LoginActivity.kt:15
  ~ Checkout.pay            signature (cartId: String) → (cartId: String, retry: Boolean)  TroublesActivity.kt:57
before → after
  LoginActivity.onCreate: + SettingsActivity::class · - MessagesActivity::class · - finish()
affected screens
  LoginActivity     via LoginActivity.onCreate · reach: mdhsample://login | MainActivity ▸ "Log in" ▸ LoginActivity
  TroublesActivity  via Checkout.pay → TroublesActivity.onCreate · reach: MainActivity ▸ "Troubles" ▸ TroublesActivity
callers of changed signatures
  Checkout.pay: TroublesActivity.kt:21 in TroublesActivity.onCreate — 1 argument, needs 2
verify
  functional     LoginActivity, TroublesActivity
note: syntax only: reflection, dependency injection, generated code and routes built at run time are not followed

The harness judges, not the agent: checks come back with what was observed, and evidence stays on disk:

$ mdh verify 'screen .LoginActivity' 'enabled id=sign_in' 'not visible id=error'
verdict: FAIL · 3 of 4 checks passed · 3.7 s
  ✓ screen .LoginActivity
  ✗ enabled id=sign_in — disabled: [e72] button "SIGN IN" disabled #sign_in
  ✓ not visible id=error
  ✓ no crash
evidence: .mdh/runs/1790957616814-verify (screenshot.jpg, tree.txt, logs.txt)

What the agent did becomes a regression test, replayed from a clean start; a crash anywhere fails it:

$ mdh flow save login-success --check 'screen .MessagesActivity'
saved .mdh/flows/login-success.yaml (4 steps, 1 check); set MDH_PASSWORD before running it
$ MDH_PASSWORD=… mdh flow run login-success troubles-crash --junit report.xml
verdict login-success: PASS · 4 steps · 2 of 2 checks passed · 5.8 s
  ✓ screen .MessagesActivity
  ✓ no crash
evidence: .mdh/runs/1790958025069-flow-login-success (screenshot.jpg, tree.txt, logs.txt)

verdict troubles-crash: FAIL · stopped after 1 of 3 steps · 0 of 2 checks passed · 3.1 s
  ✗ step 2: tap "Crash (Java)" — the app crashed (report below)
  ✗ no crash — java.lang.IllegalStateException: Sample crash: could not pay for the cart
    !! CRASH dev.mdh.sample (pid 26651): java.lang.IllegalStateException: Sample crash: could not pay for the cart
         at dev.mdh.sample.Checkout.pay(TroublesActivity.kt:61)
         …
evidence: .mdh/runs/1790958184747-flow-troubles-crash (screenshot.jpg, tree.txt, logs.txt)

flows: 1 of 2 passed

And when the app crashes during an action, the agent knows right away (exit code 5):

$ mdh tap "Crash (Java)"
tap e114 button "CRASH (JAVA)" → ok (979 ms)
screen dev.mdh.sample/.MainActivity  1344x2992  overlay:android
!! CRASH dev.mdh.sample (pid 20272): java.lang.IllegalStateException: Sample crash: could not pay for the cart
     at dev.mdh.sample.Checkout.pay(TroublesActivity.kt:61)
     at dev.mdh.sample.TroublesActivity.onCreate$lambda$0(TroublesActivity.kt:21)
     … 12 more frames
   caused by: java.lang.IllegalArgumentException: cart id must not be blank
   after: key BACK → tap e7 button "TROUBLES" → tap e114 button "CRASH (JAVA)"
[e126] "mdh sample keeps stopping"
[e127] button "App info"
[e128] button "Close app"

Architecture

<picture> <source media="(prefers-color-scheme: dark)" srcset="docs/assets/architecture-dark.svg"> <img alt="Architecture: coding agents use mdh mcp and people, CI and scripts use the mdh CLI; both drive the same engine. Control (done) drives the app; the verification engine runs flows and pluggable checks — functional, UI consistency and performance (done) — and compatibility repeats them on the devices and configurations the change puts at risk (done); all on a shared foundation (observe, project, driver, core). On the Android device, the mdh helper keeps a UiAutomation connection warm and talks to the driver over adb forward in about 10 ms; crashes, ANRs and errors flow from logcat into observe." src="docs/assets/architecture-light.svg"> </picture>

• Two entry points, one engine. Agents connect over MCP, people, CI and scripts use the CLI; both get the same compact text.
• Control, then verification. Control (done) drives the app. On top of it, a verification engine runs flows and pluggable check kinds — functional, UI consistency, performance — into one verdict, and compatibility repeats it on the devices and configurations the change puts at risk (see Status and roadmap).
• A warm helper on the device. dev.mdh.helper keeps an accessibility connection open, so reading the screen and injecting input take milliseconds; logcat feeds crash and error reports back into every result.

The diagram is generated from code (scripts/diagram) in the style of Excalidraw.

Why Rust: verification is the bottleneck

With today's models, writing the code is rarely what slows an AI coding task down — checking it is. An agent edits, builds, runs, looks, fixes and repeats; one task can take dozens of verification rounds, and every round is spent waiting on the harness and on the model reading what it returned. So the part that verifies has to be fast in three ways:

1. Little time per call. mdh starts in about 5 ms — an empty Python or Node process alone takes 30–40 ms on the same machine, before loading anything — and does its device I/O concurrently: one observation reads the UI tree, the foreground activity and new logs in parallel. When an agent drives the CLI, every step is a new process, so startup is paid on every call.
2. Little time for the model to read. About 150 tokens per screen and only diffs after actions; the model's reading time and cost grow with every token.
3. Nothing in the way. One static binary (~6.5 MB) with the device helper embedded: no runtime, no dependencies to install, the same behavior in CI as on a laptop.

| Measured on an API 36 emulator | Time | |---|---| | Read the UI with uiautomator dump | ~2,000 ms | | Read the UI with the mdh helper | ~10 ms | | Start the mdh process | ~5 ms | | mdh observe, end to end (tree, activity, logs) | ~100 ms | | An action until the UI has settled | 0.8–1.4 s, mostly the app's own animations |

To be fair to other languages: the largest wins come from the design — a warm helper instead of uiautomator, diffs instead of full screens. Rust is what keeps the harness itself from adding anything on top, and keeps it that way as the performance, visual and compatibility checks do far more work per call.

Requirements

• macOS or Linux (Windows is untested)
• Android SDK platform-tools (adb); mdh finds the SDK through ANDROID_HOME, ANDROID_SDK_ROOT