android-use

<div align="center"> <h1><img src="docs/images/hero.png" alt="Android Use: Agent plans, optional Jev decides, Android Use validates and executes." width="100%" /></h1> <p><strong>Your agent. Now on Android.</strong><br/>Use it in Dim Agent. Follow the live screen. Add Jev when you need it.</p> <p> <a href="https://github.com/arcships/android-use/actions/workflows/test.yml"><img src="https://github.com/arcships/android-use/actions/workflows/test.yml/badge.svg" alt="Tests" /></a> <a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue" alt="MIT License" /></a> <a href="pyproject.toml"><img src="https://img.shields.io/badge/python-3.11%2B-blue" alt="Python 3.11 or newer" /></a> <a href="#mcp-client-support"><img src="https://img.shields.io/badge/MCP-stdio-3DDC84" alt="Standard stdio MCP" /></a> </p> <p><strong>English</strong> · <a href="README.zh-CN.md">简体中文</a></p> <p><a href="#quick-start"><strong>Use in Dim Agent →</strong></a> · <a href="https://github.com/arcships/android-use/releases/tag/v0.7.0">Download v0.7.0</a> · <a href="#enable-jev-from-the-screen">Set up Jev</a> · <a href="docs/guide.md">User guide</a> · <a href="https://github.com/arcships/android-use/issues">Report an issue</a></p> </div>

Android tasks, inside Dim Agent

Android Use brings your own Android phone or emulator into Dim Agent, with standard MCP support for other compatible clients. Read the screen, find a control, tap, type, scroll, and check the result. Follow the process in a live phone view.

For workflows that need repeated UI decisions, enable Jev. Your Agent keeps planning and final verification; Jev selects the next candidate; Android Use validates and executes it.

“Open Settings, find Battery, and read the current usage.”

“Fill in the profile with the text I provide, verify it, and save a local draft.”

No root, Appium, signing setup, or keyboard APK. The compiled UI is included. Use USB or already-authorized wireless ADB.

Choose an installation path

The standalone runtime is optional. The Agent inspects the environment and chooses the appropriate route; a working installation is reused.

| Environment | Installation choice | | --- | --- | | Existing healthy Android Use / MCP | Reuse it and verify the connection. | | Python/uv, ADB and ffmpeg already available | Use the source installation and existing tools. | | Dependencies missing, matching verified native assets available | Use the optional standalone runtime to avoid installing dependencies individually. | | Offline, unsupported native target, or no native release assets yet | Use a suitable local archive or source setup; report any missing prerequisites. |

Optional standalone runtime · v0.7.0

Python, ADB, ffmpeg, the screen UI and the control service in one native package. The lightweight Dim plugin installs it automatically on first use. Later MCP connections start the installed version immediately and check for updates in the background; verified updates take effect on the next connection. Failed updates retain the previous version; rollback rechecks it before switching.

macOS OCR is an optional add-on, omitted from the default package. Tree interaction, Agent screenshot inspection and Jev can be used without it. Users choosing this package do not need Python, uv, Git, Node.js, a JDK or Xcode tools; public-release bootstrap uses standard OS utilities. Private-release bootstrap additionally supports GitHub CLI with a private token; offline native installation avoids that requirement.

Available in v0.7.0 as separate source-plugin, lightweight Dim-plugin and native-runtime downloads. Choose the route for your environment; older v0.6.2 source ZIPs do not include the native runtime. Install, automatic updates, optional OCR and rollback →

Quick start

Use in Dim Agent

Start in Dim Agent Desktop on the computer connected to your Android phone. Install Android Use once, then describe tasks in the chat and follow the live phone view in the right panel. Jev is optional; your existing Agent can make decisions without an extra model key.

Install from the plugin marketplace. In Dim Agent, open Plugins (插件) → Plugin Marketplace (插件市场), search for AndroidUse, and click + to install. A checkmark means the plugin is already installed.

1. Connect your phone. Enable Developer options and USB debugging, connect a data cable, accept the debugging prompt, and keep the phone unlocked. ADB must be available on the computer where Dim executes commands. For a remote Dim session, connect the phone to that execution host.

2. Let Dim Agent finish setup. If installed from the marketplace, ask the Agent to reuse the installed AndroidUse plugin, check dependencies and the device connection, verify READY, and open the live screen. Keep the existing MCP registration.

For installation from GitHub, paste this into a chat with local terminal access:

Install Android Use for Dim Agent on this computer:
https://github.com/arcships/android-use

Read README.md, docs/guide.md and skills/android-use/SKILL.md.
Reuse existing installations and preserve other MCP servers and uncommitted work.
Inspect Python/uv, ADB, ffmpeg, platform, network and available release assets.
Choose source installation or the optional standalone package for this environment;
do not migrate a healthy installation just to use a runtime. Read docs/standalone.md.
For source setup use scripts/install.py install --source .; for native setup verify
the matching archive SHA256. Both routes must return a stable MCP launch command.
Register its returned mcp_config as Dim's local stdio server named android_use;
use Dim's MCP management or dim mcp add. Check for duplicate registrations first.
Discover devices, select my intended phone, call aua_setup(action="status"),
start the service only if absent, then verify aua_ready reports ready=true.
Call aua_screen to open the live phone view in Dim's right panel and check for
fresh video. If tools are not loaded, reload MCP or start a new chat and recheck.
Please perform the setup and report the results. I will handle phone authorization
and unlocking. Leave Jev disabled for now.

<details> <summary>Manual Dim setup — stable MCP entry</summary>

Choose the source route when its dependencies are available, or the optional standalone package when it better fits the environment. Both return a stable mcp_config. For source setup, prepare dependencies and clone the repository, then run from its root:

uv run --no-project --python 3.13 python scripts/install.py install --source .

The JSON result contains mcp_config.command and mcp_config.args. In Dim's MCP management, add a local stdio server named android_use, using that command and the arguments in their original order. The installer creates an independent runtime and verifies its MCP handshake before switching versions.

With the Dim CLI, register the returned command and all arguments in order. This example is the standalone macOS/Linux entry; replace the launcher path with your actual install result:

dim mcp add --scope user android_use -- /bin/sh "/absolute/path/runtime/launcher.sh" auto-serve
dim mcp test android_use --json

For Windows or the source installer, use their returned mcp_config instead of copying these arguments. See native package installation.

Check that the MCP test returns a non-empty tool list. Some Dim CLI versions can return success=true with tools=[]; that alone does not confirm tools are loaded. In a new Dim chat, actually call aua_setup and aua_ready. If Android Use is already registered, inspect that entry before adding another. Upgrade and rollback →

</details>

3. Give it a first task. Once the tools are loaded, send:

Use Android Use to check my phone is READY, open the live screen, then open Settings and report the battery level. Do not change any settings.

Dim Agent calls the Android tools and opens the screen through aua_screen. In Desktop, the view appears in the right panel; you can watch, tap or swipe, use Back/Home and copy a frame. If the host cannot embed the view, ask it to open the returned local preview URL in a browser on the same computer.

| Try in Dim Agent | Example request | | --- | --- | | Read and navigate | “Open Settings, find Battery, and read the current usage.” | | Fill a draft | “Enter the text I provide, read it back, and save a local draft.” | | Compare prices | “Compare the same drink in Meituan and Taobao for delivery, including delivery and packaging fees. Report the totals without ordering.” | | Use Agent + Jev | “Use Jev for short UI subgoals, keep planning and recovery with the Agent, and verify the final result yourself.” |

Optional: enable Jev. Click Jev in the phone view, enter a TypeSafe API Key, then click 验证并启用 (Verify & enable). New tasks use the setting immediately; no TEXT_MODEL_* configuration is needed. Dim Agent keeps planning, text composition and final verification. Jev settings and data sent →

A phone screen you can follow

Android Use in Dim Agent: describe a task in the chat and follow the live phone view beside it. Tap or swipe inside the screen, use Back/Home, or copy a PNG frame. Live-screen guide →

Agent plans. Jev decides. Android Use acts.

| Owner | Responsibility | | --- | --- | | Your Agent | Understand the request, plan subgoals, compose text, recover and verify outcomes. | | Jev · optional | Choose a code-owned candidate from the current UI. | | Android Use | Observe, check scope and freshness, execute native actions, enforce bounds and pause. |

Known paths use batch, action-plus-wait or scroll_until. Delegate short workflows whose next action depends on new UI to aua_execute_goal. Without a TypeSafe key it runs in host-decider mode: your Agent picks each code-owned candidate with aua_goal_decide, and Android Use applies the same freshness, effect and no-replay checks. On uncertainty, control returns to the Agent with a step ledger and safe context. Architecture and examples →

Enable Jev from the screen

1. Open the phone preview and click the Jev button.
2. Paste your TypeSafe API Key.
3. Click 验证并启用 (Verify & enable).

The server checks the connection and saves the key on this computer. New tasks pick up the setting immediately. You can test the connection, disable Jev, or remove the saved key from the same panel. Jev is off by default. Setup and advanced environment configuration →

<details> <summary>See the Jev settings panel</summary> <img src="docs/images/jev-settings.jpg" alt="Jev settings with a write-only API key field and Verify and enable button" width="340" />

Enabling Jev sends the subgoal, visible UI text and supplied input values to TypeSafe for decisions. The server never returns the saved key to the UI. Screenshots are sent only for visual location (aua_locate), after you separately turn on 「允许发送截图（视觉定位）」.

</details>

Built for real interaction

| Capability | What you get | | --- | --- | | Understand the screen | Accessibility trees, screenshots, selectors and bounded waits. | | Act precisely | Tap, long press, double tap, swipe, drag, launch apps and use system buttons. | | See what the tree misses | Opt-in Jev Box locates canvas, image and unlabeled targets; a visual tap rechecks pixels and taps once. | | Reduce round trips | Combine action, wait and observation. Search lists with bounde