<p align="right"><strong>English</strong> &nbsp;·&nbsp; <a href="README.zh-CN.md">简体中文</a> &nbsp;·&nbsp; <a href="README.ja.md">日本語</a> &nbsp;·&nbsp; <a href="README.es.md">Español</a></p>

<img src="docs/figures/tab-wave.webp" alt="Ruin Everything: a receipt surfs a wave of blue and lavender browser tabs" width="100%">

<p align="center"> <a href="https://github.com/yuki4266/ruin-everything/stargazers"><img src="docs/figures/badges/stars.svg" alt="GitHub stars"></a> <a href="#use-a-shared-local-chrome-window"><img src="docs/figures/badges/python.svg" alt="Requires Python 3.11 or later"></a> <a href="#connect-an-agent-over-mcp"><img src="docs/figures/badges/mcp.svg" alt="MCP over stdio"></a> <a href="LICENSE"><img src="docs/figures/badges/license.svg" alt="MIT license"></a> </p>

Pick up a browser task where the last agent left it. Ruin connects to Chrome on your machine. A new CLI run or MCP client can find the same open application tab, finish a supported verification step, and check what the site actually confirmed.

Set up Chrome · Connect an MCP client · See what has been tested · Try your first MCP call · Contribute

| In your browser | What Ruin can do | | --- | --- | | A Greenhouse application asks for an email code | Match a fresh Outlook message to the application and fill the code. | | An Ashby tab closed after submission | Look up a previously captured receipt by the exact job URL. | | A Workday tenant asks you to sign in | Use the local profile and follow a supported account activation back to the job. | | You have phone numbers from several countries | Choose a default in the offline identity editor. |

Institutional sign-in in the background

Run ruin watch to scan open and newly opened shared Chrome tabs every 500 ms. The watcher follows institutional search on IEEE Xplore, the ACM Digital Library and Springer Nature Link. It selects Duke University and follows the publisher’s redirect. At Duke SSO, Chrome can offer a saved login. An existing trusted-auth rule can disable the password dropdown. Duo remains in the normal browser flow.

On macOS, run ruin watch --install from a persistent installation to start the user service at login. Use ruin watch --uninstall to remove it. The service waits for shared Chrome and reconnects after a disconnect. ruin watch --site URL still inspects one application challenge.

<img src="docs/figures/academic-watch.svg" alt="Shared Chrome scans academic tabs and follows Duke institutional access." width="100%">

Live verification covered the IEEE institution lookup, the redirect to Duke, and a publisher PDF delivered with an existing Duke session. Isolated browser tests cover ACM and Springer discovery, newly opened tabs and duplicate suppression.

Choose a starting point

<img src="docs/figures/command-guide.svg" alt="Five terminal commands and when to use each, from starting Chrome to watching an existing application for its receipt." width="100%">

For a new task, start the shared browser and open the target site. For an interrupted application, choose the site-specific watcher and pass the original job URL. Generic click tasks open a new tab.

<img src="docs/figures/architecture.svg" alt="Independent clients reconnect to the original tab in shared Chrome." width="100%">

Use a shared local Chrome window

<img src="docs/figures/local-chrome.svg" alt="Two independent runner connections attach to a local Chrome profile and keep the same application tab open." width="100%">

Install from the repository directory with Python 3.11+ and Google Chrome:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -e .
ruin browser

Run ruin browser to start or reconnect to the project browser. Every CLI task uses Python Playwright over CDP. The default endpoint is http://127.0.0.1:9222. The runner launches Chrome with a dedicated profile at ~/.ruin-everything/canvas-browser when this port is free. The CDP connection preserves the existing browser context settings, including its download behavior. Chrome stays open after the command exits. Other local clients can attach to the same endpoint and locate the job tab by its full URL. Give one client ownership of submission for each application.

<img src="docs/figures/tab-lifetime.svg" alt="Two commands connect to the same Chrome process. A later run can recover a live tab; closing it can discard unsaved form state." width="100%">

Open Outlook Web in this profile and complete any required Microsoft sign-in before requesting a fresh email code. Closing the job tab, quitting Chrome or a site session expiring can still interrupt an application. A draft in a chat-specific browser must be recovered separately.

Run a browser click task

<img src="docs/figures/actions.svg" alt="The model receives a task, title and control labels. It returns an action ID to the browser. The next observation creates a fresh snapshot." width="100%">

Install the optional dependencies with python -m pip install -e '.[agent]' and set OPENAI_API_KEY. Give the model an HTTPS starting page and a specific task:

ruin "Open the help page" --site https://example.org --channel chrome

The runner sends the task, page title and visible action labels to the selected model. It clicks the chosen control directly and observes the page again. A task can use one tab and up to 20 steps.

<img src="docs/figures/action-example.svg" alt="An illustrative Help center button is assigned ID 1 in one snapshot. The model can request a click by that ID." width="100%">

For example, if a visible button is labeled Open guide, the model can select its current ID. A changed URL or an ID missing from the snapshot stops that step. A blocked model decision leaves the page open for inspection.

Local credential filling

<img src="docs/figures/autofill.svg" alt="At the selected HTTPS origin, the keyring can fill credentials and Chrome can offer saved sign-ins on the original form." width="100%">

Add --autofill to use the system keyring and Chrome’s sign-in form. For a form with one visible password field that submits to the selected HTTPS origin, the runner clicks the empty field so Chrome can offer credentials saved in that browser profile. Complete sign-in in the open tab. The runner waits up to three minutes for the credential fields to disappear, then resumes the task. Without trusted login options, it leaves account selection and form submission to the login UI. A Chrome fixture verifies same-tab continuation with a restrictive script policy.

Local account registration

<img src="docs/figures/signup.svg" alt="The selected form is checked for an explicit signup action. A matching form uses the local profile, and a subsequent code field can use a fresh domain-bound SMS." width="100%">

Run ruin identity to choose phone regions by name, save multiple numbers, select a default and reveal the password while editing. The offline editor uses the browser’s native select control and the phonenumbers library for dialing codes and international formatting. python-phonenumbers

For manual setup of a browser task with --site, create ~/.ruin/identity.json with owner-only permissions (chmod 600). Set auto_signup to true and supply string values for first_name, last_name, email, phone and default_password. This file stores the password in plain text. The local runner checks the selected HTTPS origin before each model decision until it submits a registration. It recognizes one visible Create Account, Create an Account, Register or Sign Up submit button on a same-origin POST form, fills recognized fields, and checks labeled Terms of Service, Terms of Use and Privacy Policy boxes. On an ordinary same-origin form it can also fill one empty, editable contact phone field. A visible matching signup field with a missing profile value leaves the tab open for completion.

For a subsequent single one-time-code field, the runner checks macOS Messages for up to 30 seconds for an incoming text received since the submission and ending in @hostname #code. Read access to the Messages database is required. MCP challenge and receipt results include a signup_policy field reporting the local setting. Chrome and SQLite fixtures exercise form submission, ordinary phone inputs, cross-origin actions, message age, direction and domain matching.

Feishu Jobs rotation puzzle

With the rotation puzzle open, pass the exact Feishu Jobs login URL to ruin_auto_register and enable auto_signup in the local profile. As shown below, the tool matches the edges of the two rings and drags the slider once. On the live Learnings recruiting site, the widget accepted the drag and the resend button showed a 30-second countdown. The tool returns sms_requested when that countdown is visible. An unresolved puzzle returns slider_pending.

<img src="docs/figures/feishu-slider.svg" alt="The original login tab shows the inner disc and outer ring. After the drag, the resend countdown confirms the SMS request." width="100%">

ZhongAn SMS login on Moka

When the ZhongAn campus job page shows the NetEase Yidun jigsaw, pass its full URL, including the #/job/… fragment, to ruin_auto_register with local auto_signup enabled. As shown below, the tool matches the piece outline to the gap and drags it in the original tab. On the live site, the SMS button changed to a resend countdown. The tool returns sms_requested for that countdown and slider_pending for an unconfirmed puzzle.

<img src="docs/figures/yidun-slider.svg" alt="The Yidun tile moves horizontally to the matching outline. The login dialog then shows a resend countdown." width="100%">

After a form action

Use ruin_action(site, button) for an authorized submission, or ruin action --site URL --button LABEL from the CLI. Give the exact open URL and visible button name. Ruin clicks once, starts scanning the original tab and visible frames at 1.5 seconds and watches for a late popup through six seconds. Calling the same tool with only the URL resumes an open challenge. A retry operates on the widget. Confirm submission from the site’s receipt.

The figure shows how a jigsaw moves from image to page coordinates. Ruin combines ddddocr and OpenCV, accounts for the initial piece position and checks the rendered position before releasing. Browser fixtures exercise a delayed cross-origin popup, an offset piece rendered at a different scale, canvas layers, a decorative profile banner and grid controls. Image grids and single-layer visual puzzles use a local Ollama vision model (default ollama pull qwen3-vl:2b-instruct). After three unresolved rounds the tool reports challenge_pending with the tab open. The live ZhongAn flow returned its official success page, and the application history confirmed initial screening.

<img src="docs/figures/post-action.svg" alt="The original submit click leads to visual inspection, a widget retry loop, and a separate site receipt." width="100%">

Workday accounts and signed-in sessions

<img src="docs/figures/account-session.svg" alt="The Workday branch checks sign-in before account creation and Outlook activation. The session branch periodically checks a separate status tab." width="100%">

On supported English Workday tenant forms, the CLI browser task tries the local email and password first. An explicit missing-account message opens registration. A generic sign-in failure leaves the original job open. After registration, Ruin checks the signed-in Outlook mailbox for a fresh activation message, follows a direct activation link