Queen / Colony Debug Skill

SOP for live debugging of queen sessions, colony forks, worker spawns, and tracker DB plumbing without touching the user's production Hive Desktop. Use this when something is wrong in the create_colony → tracker → run_parallel_workers → worker pipeline.

Trigger

User asks you to debug, reproduce, or verify behavior in:

• Queen DM sessions, colony sessions, fork_session_into_colony
• ColonyBinding propagation (queen exec context, worker input_data)
• tracker_sql / tracker_register_writable / tracker_upsert / tracker_query
• run_parallel_workers preflight
• Phantom colonies/session_<uuid>/ shadow folders (the original split-brain bug)
• Session resume from disk, queen phase transitions (independent → incubating → colony)

Examples: "queen says no such table", "workers can't see what queen wrote", "phantom colony folder appeared", "verify my colony refactor didn't break anything".

Hard rules

1. Never run against the user's real Hive Desktop runtime by default. Use an isolated HIVE_HOME=/tmp/hive_e2e first. Only switch to the real HIVE_HOME (~/Library/Application Support/Hive/users/<hash>) when the user has explicitly asked for live LLM verification or when an offline repro is impossible.
2. Never read the real secrets/, credentials/, or configuration.json directories. The auto-mode classifier will block credential exploration. You don't need their contents — the server reads them itself.
3. Pick a non-default port (--port 8901/8902/8903) so you don't collide with a running Hive Desktop on 8787.
4. Background the server, don't foreground it. & redirects the log to a file you can tail/grep while you make HTTP calls in parallel.
5. For "wait for thing X" patterns: use Bash run_in_background:true with an until grep -q ... loop — never chain sleep N. The harness blocks long leading sleeps.
6. LLM-driven turns cost real credits. Budget your queen prompts: prefer terse, deterministic instructions ("just call create_colony with these exact args") over open-ended questions.

What "correct" looks like (key invariants)

These are the invariants the refactor enforces; verifying them is most of the job:

• A DM queen session must not create colonies/<session_uuid>/ (the phantom-folder bug). Only on-disk colony names live under colonies/.
• worker.json input_data carries exactly one key: {"binding": {"name", "dir", "tracker_db"}}. No tracker_db_path, no colony_id (those are legacy and get stripped by _patch_worker_configs on every server boot).
• The queen and her workers in a given colony share one tracker.db — the one inside colonies/<name>/data/.
• Tools refuse with "no colony context — this tool only works inside a colony" when called without a binding. They never synthesize paths.
• run_parallel_workers emits the log line run_parallel_workers: attached binding to N spawn(s) (colony=<name>). If that line is missing, the binding plumbing is broken.

Authoritative source for the binding model: core/framework/host/colony_binding.py.

SOP

Step 1 — Pick a runtime

Default to isolated:

mkdir -p /tmp/hive_e2e/colonies /tmp/hive_e2e/agents/queens
PORT=8901
HIVE_HOME=/tmp/hive_e2e uv run hive serve --port $PORT --verbose 2>&1 > /tmp/hive_e2e/server.log &
echo "pid: $!"

Confirm it's up:

until curl -sf http://127.0.0.1:$PORT/api/health >/dev/null 2>&1; do sleep 1; done
curl -s http://127.0.0.1:$PORT/api/health

For real-runtime verification (only when explicitly requested):

REAL="/Users/aden/Library/Application Support/Hive/users/<the-user-hash>"  # find via: ls ~/Library/Application\ Support/Hive/users/
HIVE_HOME="$REAL" uv run hive serve --port 8903 --verbose 2>&1 > /tmp/hive_real.log &

Verify Commercial extensions loaded appears in the startup log; that's the green light.

Step 2 — Snapshot the starting state

echo "=== colonies dir ==="; ls "$HIVE_HOME/colonies/"
echo "=== queens ==="; ls "$HIVE_HOME/agents/queens/" 2>&1 | head -10
echo "=== existing sessions ==="; curl -s http://127.0.0.1:$PORT/api/sessions | uv run python -m json.tool

Anything session_* under colonies/ BEFORE you do anything is an existing phantom-folder issue.

Step 3 — Drive the failing flow

(a) Create a DM session (queen-only, no LLM-side actions)

RESP=$(curl -s -X POST http://127.0.0.1:$PORT/api/sessions -H 'Content-Type: application/json' \
  -d '{"queen_name": "queen_technology"}')
SESSION_ID=$(echo "$RESP" | uv run python -c "import json,sys; print(json.load(sys.stdin)['session_id'])")
echo "$SESSION_ID"

Invariant check: colonies/ should still be empty. If colonies/session_$SESSION_ID/ appeared, the phantom-folder bug is back. Suspect: ColonyRuntime.__init__ re-introduced an unconditional ensure_task_list(colony:<colony_id>) call.

(b) Fork DM into a colony — non-LLM path

This drives fork_session_into_colony without burning credits on a queen turn:

curl -s -X POST "http://127.0.0.1:$PORT/api/sessions/$SESSION_ID/colony-spawn" \
  -H 'Content-Type: application/json' \
  -d '{"colony_name":"debug_test","task":"debug"}' | uv run python -m json.tool

Expected response shape: {colony_path, colony_name, queen_session_id, is_new, compaction_status}. No tracker_db_path field — if it's there, the cleanup regressed.

Then verify the on-disk binding:

uv run python -c "
import json
cfg = json.load(open('$HIVE_HOME/colonies/debug_test/worker.json'))
print(json.dumps(cfg.get('input_data'), indent=2))
"

Expected:

{
  "binding": {
    "name": "debug_test",
    "dir": "/.../colonies/debug_test",
    "tracker_db": "/.../colonies/debug_test/data/tracker.db"
  }
}

If you see tracker_db_path or colony_id keys here, worker_definition.build_input_data or routes_execution.fork_session_into_colony is writing the legacy shape.

(c) LLM-driven path (talk to the queen)

Only use when (b) isn't enough. Send a tight, deterministic prompt:

curl -s -X POST "http://127.0.0.1:$PORT/api/sessions/$SESSION_ID/chat" \
  -H 'Content-Type: application/json' \
  -d '{"message": "Just call create_colony(colony_name=\"debug_e2e\", task=\"debug\"). Do nothing else."}'

Then watch for the actual tool call (this is the right way to wait — no sleep chains):

# In Bash with run_in_background:true
until grep -qE "tool_call: create_colony|Forked queen to colony|colony fork failed" /tmp/hive_e2e/server.log; do sleep 5; done

When background command exits, grep the log for what actually happened:

grep -E "tool_call: create_colony|Forked queen to colony|colony fork failed|fork_session" /tmp/hive_e2e/server.log | tail -10

(d) Activate the colony's queen session (post-fork, for tracker work)

fork_session_into_colony creates a separate colony-queen session on disk (returned as queen_session_id) that isn't loaded into the SessionManager until you ask. To talk to it:

COLONY_SESSION="<queen_session_id from fork response>"
COLONY_PATH="$HIVE_HOME/colonies/debug_e2e"
curl -s -X POST http://127.0.0.1:$PORT/api/sessions \
  -H 'Content-Type: application/json' \
  -d "{\"agent_path\":\"$COLONY_PATH\", \"queen_resume_from\":\"$COLONY_SESSION\", \"queen_name\":\"queen_technology\"}" \
  | uv run python -m json.tool

Verify queen_phase: "colony" in the response — that's when tracker tools are exposed. If she's still "independent", she lost her binding on resume; check queen_orchestrator.py:_queen_loop — it should call ColonyBinding.for_name(session.colony_name) and stamp the exec context.

(e) Drive the queen→worker tracker flow

curl -s -X POST "http://127.0.0.1:$PORT/api/sessions/$COLONY_SESSION/chat" \
  -H 'Content-Type: application/json' \
  -d '{"message": "Run tracker_sql to CREATE TABLE x (id INTEGER PRIMARY KEY, body TEXT). Then CREATE UNIQUE INDEX x_id ON x(id). Then tracker_register_writable(table=x, write_columns=[body], key_columns=[id]). Then INSERT INTO x VALUES (1, \"seed\"). Then run_parallel_workers with one task that calls tracker_upsert to set body=\"worker wrote\" on id=1, then report_to_parent."}'

(Note: INTEGER PRIMARY KEY does NOT register as a unique index in SQLite's PRAGMA index_list — you need an explicit CREATE UNIQUE INDEX. This is pre-existing validator behavior in tracker_tools, not a refactor regression.)

Then wait for the marker line:

# Bash run_in_background:true
until grep -q "attached binding to" /tmp/hive_e2e/server.log; do sleep 3; done
grep "attached binding to" /tmp/hive_e2e/server.log

Expected line: run_parallel_workers: attached binding to N spawn(s) (colony=debug_e2e). Missing → binding plumbing broken in queen_lifecycle_tools.py::run_parallel_workers.

Step 4 — Verify with sqlite3 directly

Cut out the HTTP layer and look at the raw tracker DB:

sqlite3 "$HIVE_HOME/colonies/debug_e2e/data/tracker.db" ".schema"
sqlite3 "$HIVE_HOME/colonies/debug_e2e/data/tracker.db" "SELECT * FROM _tracker_registry"
sqlite3 "$HIVE_HOME/colonies/debug_e2e/data/tracker.db" "SELECT * FROM x"  # or whatever table

Confirm there is no second DB at a session-id-named path:

ls "$HIVE_HOME/colonies/" | grep "^session_" && echo "PHANTOM FOLDER PRESENT (BUG)" || echo "clean ✓"
find "$HIVE_HOME/colonies" -name "tracker.db" -type f

There should be exactly one tracker.db per real colony. Multiple means split-brain.

Step 5 — Test refusal behavior without an LLM

Cheap Python repro of "tools refuse without binding":

HIVE_HOME=/tmp/hive_e2e uv run python -c "
import asyncio
from framework.tools.tracker_tools import _make_tracker_sql_executor, _make_tracker_query_executor, _make_tracker_upsert_executor, _make_tracker_register_executor

async def main():
    for name, mk in [
        ('tracker_sql', _make_tracker_sql_executor),
        ('tracker_query', _make_tracker_query_executor),
        ('tracker_upsert', _make_tracker_upsert_executor),
        ('tracker_register', _make_tracker_register_executor),
    ]:
        r = await mk()({'sql': 'SELECT 1', 'table': 'x', 'row': {'a': 1}, 'write_columns': ['a'], 'key_columns': ['a']})
        ok = r.get('success') is False and 'no colony context' in r.get('error', '')
        print(f'  {name}: {\"REFUSED ✓\" if ok else \"UNEXPECTED ✗\"} → {r}')

asyncio.run(main())
"

Step 6 — Test queen↔worker tracker sharing (offline)

Exercises the entire binding flow in-process, no LLM cost:

HIVE_HOME=/tmp/hive_e2e uv run python -c "
import asyncio
from framework.host.colony_binding import ColonyBinding
from framework.host.tracker_db import ensure_tracker_db
from framework.loader.tool_registry import ToolRegistry
from framework.tools.tracker_tools import (
    _make_tracker_sql_executor,
    _make_tracker_register_executor,
    _make_tracker_upsert_executor,
    _make_tracker_query_executor,
)

async def main():
    binding = ColonyBinding.for_name('offline_test')
    ensure_tracker_db(binding.dir)

    # Queen: DDL + register + seed
    tok = ToolRegistry.set_execution_context(binding=binding)
    try:
        sql = _make_tracker_sql_executor()
        await sql({'sql': 'CREATE TABLE t (k TEXT PRIMARY KEY, v TEXT)'})
        await sql({'sql': 'CREATE UNIQUE INDEX t_k ON t(k)'})
        await _make_tracker_register_executor()({'table': 't', 'write_columns': ['v'], 'key_columns': ['k']})
        await sql({'sql': \"INSERT INTO t VALUES ('a', 'queen')\"})
    finally:
        ToolRegistry.reset_execution_context(tok)

    # Worker: read + upsert from a SEPARATE exec context (binding comes via input_data)
    binding_f