hive.worker-delegation

---

name: hive.worker-delegation description: Concrete patterns for breaking colony work into parallel worker jobs via run_playbook — when fan-out helps, how to model the goal as a tracker table, write the worker skill, author the playbook, pilot, and let convergence retry/resume the gap. metadata: author: hive type: default-skill visibility: [colony]

Operational Protocol: Worker Delegation

Applies when you're in COLONY mode and considering whether (and how) to fan out work to parallel workers via run_playbook. Read this before fan-out, not during.

Mental model: the tracker is the spine, the playbook is the controller

You don't coordinate workers by reading their reports and deciding what's next each turn. You model the goal as a tracker table where every unit of work is a row, and you write a playbook — a deterministic Python script — that drives that table to completion:

The playbook queries the rows that aren't done yet, dispatches one worker per undone row, and re-queries until none are left. Workers advance their own rows. Re-running the playbook resumes — done rows simply aren't in the work-list anymore.

This is a reconciliation loop. The tracker is the state; the playbook is the controller that converges it. Three artifacts, three jobs:

• Tracker table — the durable work-list and its state. The row's status column is the progress.
• Skill (write_skill) — the worker's operating procedure: schema, tool sequence, output format, quality bar. The risky part.
• Playbook (run_playbook) — the deterministic orchestration: which rows are undone, who runs them, rate limits, retry/convergence policy. The cheap part.

The worker's task string carries only the per-row slice; everything reusable lives in the skill, everything deterministic lives in the playbook.

The decision: should you fan out at all?

Fan-out helps when:

• The work has N independent units (rows, person on linkedin, files, accounts, segments) and each unit takes meaningful tool time (browser, API, file read, LLM call).
• The units are disjoint — no two workers need to write the same row at the same time.
• You can describe one unit's work in <100 words once shared playbook is in the skill.

Fan-out HURTS when:

• N=1 or N=2 with cheap units. Spawning has overhead (fresh AgentLoop, separate conversation, no shared context). Below ~3 units of meaningful work, do it yourself.
• The work is exploratory ("figure out X"). Workers are bad at open-ended scope. Decompose first, then fan out the bounded parts.

When the user explicitly asks for fan-out, do not reject the request from an untested architecture guess. If you are unsure whether a browser session, API cursor, login, or other shared resource can be used by workers, ask the user. Workers you spawn get their own separate Chrome tab groups within the SAME Chrome profile — their tabs won't interfere with yours or each other's, and they share cookies / logged-in sessions with you.

Pilot before fan-out (do the first one yourself)

You wrote the skill from your own walkthrough — but a walkthrough is not an execution. Selectors that worked when exploring can break under the exact tool sequence the skill prescribes; a page may paginate differently when fetched fresh; a field you eyeballed once might be intermittently null. Validate the skill yourself before paying N× to discover the bug.

The queen runs the pilot, not a worker. Pick ONE row from the tracker and execute the skill's protocol end-to-end with your own tools — the same hive-browser commands, tracker_*, web_scrape, etc. the workers would use. You see every tool result directly, with no [WORKER_REPORT] round-trip, and you can patch the skill mid-pilot as you discover gaps.

When to pilot (always, even when the user asks for "parallel"):

• You just wrote the skill from your own walkthrough, or you're recycling a skill across a UI/API you haven't driven this session.
• The work touches a UI surface that virtualizes, paginates, or has dynamic selectors (LinkedIn, Twitter, Notion, anything with virtual scroll or Shadow DOM).
• The per-unit work spans more than 2–3 tool calls.

How to pilot:

1. Pick ONE row — the most representative one, not the easiest.
2. Execute the skill yourself: run each tool in the prescribed order, advance the row to "done."
3. If you hit a snag — fix the skill in place before continuing. Capturing these patches is the whole point.
4. If the row finishes cleanly: the skill is validated. Run the playbook for the rest.
5. If you can't finish the row at all: the protocol is wrong (not one selector). Redesign before any worker touches it.

Skip the pilot only when the protocol is one you've already validated this session AND nothing about the target surface has changed.

The loop (always, in order)

1. Model the goal as a table — tracker_sql('CREATE TABLE …'). Every unit is a row. Include a done-predicate column (a status enum, or a *_at timestamp that is NULL until complete). The playbook's "what's left" query depends on it. Register the columns workers write with tracker_register_writable(...).
2. Write the worker protocol as a skill — write_skill(skill_name='<protocol>', skill_body='…'). Or write_skill(source_path='<root>') to lift an existing skill into this colony to pilot-patch it. The worker's last act is to advance its own row to done.
3. Pilot the first row yourself — execute the skill end-to-end against one row. Patch the skill in place. Don't run the playbook until this row finishes cleanly.
4. Author the playbook — a Python script (meta + async def run(args)) that calls converge(...) over the table. Set meta["concurrency"] to how many workers run at once (you own that number; the framework honors it, rejecting only if it's too high). The script runs in the colony's full Python env — import json / datetime etc. just work. This is the deterministic orchestration (next section).
5. Run it — run_playbook({playbook: '<script>'}). It saves the script to the colony library (playbooks/<meta-name>.play.py), returns immediately, and notifies you on completion. The convergence loop dispatches undone rows, retries the gap, and dead-letters terminal failures — without bouncing every worker report back to you. (If the script has a real error, you get it right away, not a false "started".)
6. Re-running is resuming — call run_playbook({playbook_name: '<meta name>'}) (no need to re-send the script) — or edit playbooks/<name>.play.py and re-run by name. It re-queries the undone rows; done rows are skipped. There is no manual "find the gap and re-dispatch" — the pending query is the gap.

Skipping step 1 means you have no done-predicate, so nothing can resume. Skipping step 2 means you pay N× tokens for duplicated protocol. Skipping step 3 means one bad skill becomes N failed workers.

GTM work: bookend the loop with the shared CRM (queen-only)

When the units are people / leads / accounts (cold outreach, enrichment, etc.), the shared team CRM — the hive-crm CLI (people/companies/opportunities), team-wide and cross-colony — is queen-owned: workers never touch it; they only fill the local tracker and report up. Wrap the loop with two CRM steps, and put both in your task plan so they're tracked deliverables, not afterthoughts:

• Before step 1 — CLAIM (dedup across colonies). Two colonies running outreach at once will double-touch the same prospect unless you claim first. hive-crm summary --json (load state), then hive-crm import --file leads.json --json to create/dedup the target people team-wide (returns their person_ids), then hive-crm claim <person_ids> --json to atomically lock them — you win only the unclaimed; ids that come back under skipped are owned by another colony, so drop them. Seed the local tracker with only the people you won. Do not list-then-decide — that races the other colony.
• After step 6 — PROMOTE. On [PLAYBOOK_COMPLETE], read the completed local rows, hive-crm import the finished people to update the shared record, then hive-crm release <person_ids> --json to hand them off. The playbook stays local; you do the promote.

(Recording outreach outcomes on a person — advancing stage, logging calls/emails/replies — is rolling out; for now import keeps the shared people record current.)

What goes in the playbook

The playbook is plain deterministic code — pull everything OUT of the worker prompt that doesn't need judgment:

• Concurrency — meta["concurrency"]: how many workers run at once. You set it; the framework honors it.
• Decomposition — the pending query: the rows not yet done (WHERE researched_at IS NULL). Derived from tracker state every run.
• Routing — which profile (account binding) and which lane (rate limit) each row goes to.
• Convergence policy — max_rounds (how many times to retry the gap), circuit_breaker (abort a round if too many fail). (chunk defaults to meta["concurrency"]; only set it to override per-round in-flight count.)
• Contract — the receipt schema each worker must return, and the row transition that counts as done.
• Reduce — the final summary you hand back (counts, dead-letter list).

The API contract — get these right or you dispatch 0 workers:

• tracker_query(sql) / tracker_count(sql) are synchronous (no await). tracker_query returns a list of row dicts ([{'id':'a', ...}, ...]) — index a row's column (row['id']), never the list. A SELECT COUNT(*) returns ONE row [{'cnt': N}] — that's for counting, not the pending list.
• converge(...) and worker(...) are async: await converge(...), and inside it dispatch=lambda row, i: worker(...) (converge awaits each worker for you). Never call worker() in a bare loop without await — the coroutine won't run and you dispatch nothing. And the inverse trap: for row in rows: await worker(...) runs SERIALLY — each await blocks until that worker reports, so meta["concurrency"] does nothing. Parallel dispatch happens ONLY through converge — hand rows in via pending and let dispatch build the worker coroutine.
• pending must SELECT the undone rows, not a COUNT.

Hooks available in the script: converge, worker, tracker_query / tracker_count, lane, deadletter, log, phase. There is no mid-run escalation: a worker unsure about a row records that in the row (e.g. a needs_review status) and moves on; you review those rows — and the dead-letter — after the run completes. Because re-running is resume, the completion boundary is the decision point: stop, decide, edit, re-run (done rows are skipped).

Decomposition patterns

Pick ONE decomposition axis per playbook.

Per-row — one worker per undone row (the default). pending selects rows; dispatch runs one worker each. For very cheap units, group a few rows per worker by selecting in batches. Use when each row is independently researchable / fillable.

Per-segment — workers DISCOVER rows within a slice (alphabetical, geographic, time window) rather than filling seeded rows. Use a UNIQUE INDEX on the natural key so two segments finding the same entity don't duplicate (INSERT OR IGNORE in worker upserts). The done-predicate is "segment marked swept."

Per-stage (state machine) — a multi-phase pipeline becomes states on the row: new → enriched → notified → done. Each converge pass (or each playbook) advances rows at one state to the next. The row's status column carries the progress; stage-2 workers read what stage-1 wrote.

Routing, lanes, and rate limits

Workers bound to the same external account must not hammer it in parallel. Two levers, both keyed off the row index:

ACCOUNTS = ["li-work-1", "li-work-2", "li-work-3"]   # worker profiles, one per account
for a in ACCOUNTS:
    lane(a, concurrency=3, ra