kwin-mcp

Model Context Protocol server for Linux desktop GUI automation on KDE Plasma 6 Wayland

A Model Context Protocol (MCP) server that enables AI agents (Claude Code, Cursor, and other MCP clients) to launch, interact with, and observe any Wayland application in a fully isolated virtual KWin session -- without affecting the user's desktop. It also supports live desktop automation by connecting to an existing KWin session (real desktop or container) for collaborative workflows. With 30 MCP tools covering mouse, keyboard, touch, clipboard, accessibility tree inspection, screenshot capture, and window management, kwin-mcp provides everything needed for end-to-end GUI testing and desktop automation on Linux.

Table of Contents

• Why kwin-mcp?
• Use Cases
• Quick Start
• Configuration
• Available Tools
• How It Works
• System Requirements
• Installation
• Limitations
• Contributing
• License

Why kwin-mcp?

• Isolated sessions -- Each session runs in its own dbus-run-session + kwin_wayland --virtual sandbox. Your host desktop is never affected.
• Live session support -- Connect to a real KDE Plasma desktop or a KWin instance inside a container (e.g. systemd-nspawn) for collaborative "share my screen" workflows.
• No screenshots required for interaction -- The AT-SPI2 accessibility tree gives the AI agent structured widget data (roles, names, coordinates, states, available actions), so it can interact with UI elements without relying solely on vision.
• Zero authorization prompts -- Uses KWin's private EIS (Emulated Input Server) D-Bus interface directly, bypassing the XDG RemoteDesktop portal. No user confirmation dialogs.
• Works with any Wayland app -- Anything that runs on KDE Plasma 6 Wayland works: Qt, GTK, Electron, and more. Input is injected via the standard libei protocol.
• Full input coverage -- Mouse, keyboard, multi-touch, and clipboard -- all injected through the isolated session for complete desktop automation.

Use Cases

Automated GUI Testing

Run end-to-end GUI tests for KDE/Qt/GTK applications in headless isolated sessions. kwin-mcp launches each app in its own virtual KWin compositor, interacts via mouse, keyboard, and touch input, then verifies results through screenshots and the accessibility tree -- all without a physical display.

AI-Driven Desktop Automation

Let AI agents like Claude Code autonomously operate desktop applications. The agent reads the accessibility tree to understand the UI, performs actions through 30 MCP tools, and observes the results via screenshots -- creating a complete feedback loop for any Wayland application.

Live Desktop Collaboration

Connect to your real desktop session and let Claude observe and interact with what you see. Use session_connect or pass --default-live-session to make live mode the default. Also supports attaching to KWin running inside containers (e.g. systemd-nspawn) for isolated agent desktops.

Headless GUI Testing in CI/CD

Integrate Linux desktop GUI testing into CI/CD pipelines. kwin-mcp's virtual sessions require no X11 or physical display server, making it suitable for headless environments like GitHub Actions or GitLab CI runners on Linux.

Kiosk and Embedded Device Automation

Automate kiosk interfaces and embedded Linux desktops running KDE Plasma or a bare KWin Wayland compositor. Use session_start for isolated virtual testing of kiosk UIs, or session_connect to attach directly to a live kiosk or embedded device session for real-time automation and diagnostics.

Quick Start

Requires KDE Plasma 6 on Wayland. See System Requirements for details.

1. Install

# Using uv (recommended)
uv tool install kwin-mcp

# Or using pip
pip install kwin-mcp

2. Configure Claude Code

Add to your project's .mcp.json:

{
  "mcpServers": {
    "kwin-mcp": {
      "command": "uvx",
      "args": ["kwin-mcp"]
    }
  }
}

3. Use it

Ask Claude Code to launch and interact with any GUI application:

Start a KWin session, launch kcalc, and press the buttons to calculate 2 + 3.

Claude Code will autonomously start an isolated session, launch the app, read the accessibility tree to find buttons, click them, and take a screenshot to verify the result.

Configuration

Recommended: install as a plugin

The fastest way to wire kwin-mcp into your editor is to install one of the bundled plugins. Each plugin auto-registers the MCP server and ships the kwin-desktop-automation skill, which teaches the agent which tool to call when (session-mode selection, the observe → act → verify loop, US-QWERTY vs Unicode typing, AT-SPI2 surface-local coordinates, and other platform pitfalls).

Claude Code — install the plugin from the marketplace:

/plugin marketplace add isac322/kwin-mcp
/plugin install kwin-mcp@kwin-mcp

OpenCode — add the npm plugin to your opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@isac322/kwin-mcp-opencode"]
}

For the full integration guide (manual fallback, customising the skill, troubleshooting), see docs/ai-agent-integration.md.

Claude Code

Add to your project's .mcp.json:

{
  "mcpServers": {
    "kwin-mcp": {
      "command": "uvx",
      "args": ["kwin-mcp"]
    }
  }
}

Or if installed globally:

{
  "mcpServers": {
    "kwin-mcp": {
      "command": "kwin-mcp"
    }
  }
}

Claude Desktop

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "kwin-mcp": {
      "command": "uvx",
      "args": ["kwin-mcp"]
    }
  }
}

Running Directly

# As an installed script
kwin-mcp

# As a Python module
python -m kwin_mcp

# Interactive CLI (REPL for rapid testing)
kwin-mcp-cli

# Live session mode (default to real desktop instead of virtual)
kwin-mcp --default-live-session
kwin-mcp-cli --default-live-session

Available Tools

Session Management (3 tools)

| Tool | Parameters | Description | |------|-----------|-------------| | session_start | app_command? str, screen_width? int (1920), screen_height? int (1080), enable_clipboard? bool (false), keep_screenshots? bool (false), isolate_home? bool (false), keep_home? bool (false), env? dict | Start an isolated KWin Wayland session, optionally launching an app. Set enable_clipboard=true to enable clipboard tools (requires wl-clipboard). Set keep_screenshots=true to preserve screenshot files after session_stop. Set isolate_home=true to create a temporary HOME with isolated XDG directories (config, data, cache, state), preventing apps from reading/writing host user settings. Set keep_home=true to preserve the isolated home directory after session_stop. Pass extra environment variables via env. | | session_connect | dbus_address? str, wayland_display? str, keep_screenshots? bool (false) | Connect to an existing KWin session (real desktop or container). Defaults to $DBUS_SESSION_BUS_ADDRESS and $WAYLAND_DISPLAY. Clipboard is always enabled. session_stop only disconnects without killing KWin or pre-existing apps. | | session_stop | (none) | Stop the session and clean up. For virtual sessions: terminates KWin and all apps. For live sessions: disconnects without killing KWin or pre-existing apps. |

Observation (3 tools)

| Tool | Parameters | Description | |------|-----------|-------------| | screenshot | include_cursor? bool (false) | Capture a screenshot of the virtual display (saved as PNG, returns file path) | | accessibility_tree | app_name? str, max_depth? int (15), role? str | Get the AT-SPI2 widget tree with roles, names, states, and coordinates. Use role to filter to specific element types (e.g. "button", "check box"). Non-matching elements are hidden but their children are still traversed. | | find_ui_elements | query str, app_name? str, states? list[str] | Search for UI elements by name, role, or description (case-insensitive). Optionally filter by AT-SPI2 states (e.g. ["focused"], ["active", "visible"]). query can be empty when filtering by states only. |

Mouse Input (6 tools)

| Tool | Parameters | Description | |------|-----------|-------------| | mouse_click | x int, y int, button? str ("left"), double? bool, triple? bool, modifiers? list[str], hold_ms? int (0), screenshot_after_ms? list[int] | Click at coordinates. Supports left/right/middle, single/double/triple click, modifier keys (e.g. ["ctrl", "shift"]), and long-press via hold_ms. | | mouse_move | x int, y int, screenshot_after_ms? list[int] | Move the cursor (hover) to coordinates without clicking | | mouse_scroll | x int, y int, delta int, horizontal? bool, discrete? bool, steps? int (1) | Scroll at coordinates. delta positive = down/right, negative = up/left. Use discrete=true for wheel ticks, steps to split into smooth increments. | | mouse_drag | from_x int, from_y int, to_x int, to_y int, button? str ("left"), modifiers? list[str], waypoints? list[[x,y,dwell_ms]], screenshot_after_ms? list[int] | Drag from one point to another with smooth interpolation. Supports custom waypoints for complex drag paths. | | mouse_button_down | x int, y int, button? str ("left") | Press a mouse button at coordinates without releasing. Use with mouse_button_up for manual drag control. | | mouse_button_up | x int, y int, button? str ("left") | Release a previously pressed mouse button at coordinates |

Keyboard Input (5 tools)

| Tool | Parameters | Description | |------|-----------|-------------| | keyboard_type | text str, screenshot_after_ms? list[int] | Type a string of text character by character (US QWERTY layout) | | keyboard_type_unicode | text str, screenshot_after_ms? list[int] | Type arbitrary Unicode text (Korean, CJK, etc.) via wtype or clipboard fallback (wl-copy + Ctrl+V). Requires wtype or wl-clipboard installed. | | keyboard_key | key str, screenshot_after_ms? list[int] | Press a key or key combination (e.g., Return, ctrl+c, alt+F4, shift+Tab) | | keyboard_key_down | key str | Press and hold a key without releasing. Useful for holding modifiers across multiple actions (e.g., hold Ctrl while clicking items). | | keyboard_key_up | key str | Release a previously held key |

Touch Input (4 tools)

| Tool | Parameters | Description | |------|-----------|-------------| | touch_tap | x int, y int, hold_ms? int (0), screenshot_after_ms? list[int] | Tap at coordinates. Use hold_ms for long-press gestures. | | touch_swipe | from_x int, from_y int, to_x int, to_y int, duration_ms? int (300), screenshot_after_ms? list[int] | Swipe from one point to another with configurable duration | | touch_pinch | center_x int, center_y int, start_distance int, end_distance int, duration_ms? int (500), `screenshot_after_ms