Local AI Coding Agent for AMD Strix Halo (Ryzen AI Max+ 395)

Qwen3.8 125B on the Radeon 8060S iGPU, four small models on the NPU, cloud optional — a self-hosted pi coding-agent harness built on llama.cpp.

<div align="center">

What you get • The stack • Evidence • Install • Model files • Quick Start • The NPU • Crash recovery • Upgrading halogen • Documentation • License

</div>

A coding-agent harness for the AMD Strix Halo (Ryzen AI Max+ 395 / gfx1151): the pi coding agent driving Qwen3.8 Flash-Next (125B) on the iGPU, with the NPU running the small side jobs (prompt-injection screening, fast routing decisions, embedding dedup) and a llama.cpp sidecar writing commits and compaction summaries. One-command setup, crash recovery that survives a killed server, and the 262k ctx window the engine is capable of.

It runs with no API key and no outbound calls by default, and switching to a cloud model mid session is a supported shape rather than a fallback: the NPU tools and the whole sidecar suite stay on-device either way, so screening, search and compaction never follow your code off the box — the hybrid, measured.

The model was never the hard part. This repo is everything around the model: the launch scripts, the extension suite, the allocator pre-flight, the speed tooling, and the measured numbers behind every claim. Every benchmark here is logged — see BENCHMARKS.md and the local vs cloud head-to-head, seven runs of one real debugging task rated against verified ground truth.

[!TIP] Just want the agent running? ./setup.sh --halogen auto, then ./start.sh ~/my-project. The two things you must supply are docker and the model files (~124 GB, local). Everything else is optional, and ./setup.sh --doctor tells you what is live.

Measured on an ASUS ROG Flow Z13 (Ryzen AI Max+ 395, Radeon 8060S, 128 GB) at 70 W sustained.

What you get

• One-command setup — ./setup.sh is idempotent: extensions, providers, sidecar download, keep-alive units, halogen launch, memory check. It touches nothing you configured yourself and probes every piece instead of trusting it — the NPU guard answers a real /v1/moderations probe, the vision tower shows enabled: true on /health, and the smoke test checks its own file artifact rather than the model's report.
• Crash recovery — progress-tracker checkpoints task and partial state; relaunch and say "continue". Verified across six scenarios, including kill -9 at response 1, server death mid-task, and 15-deep compaction chains.
• NPU side jobs — the npu-retrieval extension ships triage (a constrained routing decision with per-option probabilities in ~120 ms) and dedup_scan (near-duplicate files by embedding similarity); auto-guard screens every prompt on the guard model as a fail-open tripwire (42 % recall / 0 % false positives on the 30-prompt canary — not a security boundary). See The NPU.
• Semantic code search — the pinned semble MCP server (search + find_related), wired into mcp.json by setup. A/B on 18 ground-truth queries against the NPU RAG pipeline it replaced: 17/18 vs 13/18 top-3 hits, 2-3x faster queries, ~40x faster indexing. It needs no NPU and indexes any local path or remote git URL, on first query.
• Sidecar suite off the main context — ling-tiny-compaction (non-blocking, thinking off, 10/10 rule retention on the canary), ling-tiny-commit, ling-tiny-branch-summary, ling-tiny-repomap. Diffs and transcripts never touch halogen's prompt cache, so a side call can never evict your session prefix.
• Speed tooling — scripts/halogen-top.sh (live prefill/decode dashboard) and scripts/session-audit.sh --perf (per-session throughput from the server logs).
• bench/ and config/ — harness-ab.sh (baseline vs kill+resume A/B, one command), report-accuracy.py (grades every file:line claim in a report against the tree), the guard-canary.py / compaction-retention.py canaries, check-sidecar.sh + check-fixes.sh (offline regression checks), plus models.json.example and the compaction snippet. Two numbers there must be understood, not copied: reserveTokens is per-model, and on halogen maxTokens must stay <= 8192 or the server 400s past ~32.7k ctx.
• Also shipped — harness-tune (/tune live knobs), turn-timer, subagent/ (worker/scout/reviewer/planner, session-model inheritance), /gauntlet and /speedtest prompt templates, and opt-in voice (./setup.sh --voice: privateer-speak on a local Lemonade server — whisper STT 0.43 s, kokoro TTS 0.3 s, both on CPU). Measured no-ops were deleted; receipts in extensions/README.md.

The stack

One box, four model servers (five with voice), three pieces of silicon:

• halogen (:8731, iGPU) — Qwen3.8 Flash-Next 125B from a local checkpoint: the agent's main model, 262,144 ctx x 4 slots, prompt cache on.
• NPU sidecars (:8731, NPU) — four 0.6-0.8B models on the same server: guard, decider, embedder (and one idle reranker). Detailed in The NPU.
• ling3.0-tiny (:8090, iGPU, llama.cpp) — the sidecar for commit messages, compaction summaries and repo maps.
• semble (child process, CPU) — semantic code search over uvx; no model server.
• Lemonade (:13305, CPU, opt-in --voice) — whisper STT and kokoro TTS.

Flow: every prompt is screened by the NPU guard before the main model sees it; the main model generates on the iGPU with its prompt cache warm; side jobs that need a judgement in milliseconds (is this injection, which of these three is it, which files are near-duplicates) go to the NPU instead of the 125B model; sidecars read diffs and transcripts directly so they never evict the cached prefix; and search runs on CPU against an index built at first query. Halogen is the only engine this harness provisions — what breaks on another engine is measured, too.

Halogen vs gufo (the best open engine)

gufo (MIT) is the open alternative to closed-source halogen. Matched A/B on this box, same session and client, fabric clock held:

| | halogen 0.18.1 (default) | gufo 0.12.0 | |---|---|---| | decode, 20-prompt battery (median) | 62 t/s | 40 t/s | | decode, 4k to 131k context | 45-50 t/s | 29-32 t/s | | cold prefill, 131k prompt | 79 s | 94 s | | warm cache replay | 0.01-0.05 s | ~0.1 s | | NPU tools (guard, triage, dedup) | yes | no | | license | closed | MIT |

Halogen stays the default; gufo is the open fallback. To run it, stop halogen first (they share one memory heap), then server/start-gufo-flashnext.sh (UD-Q4_K_XL + MTP, :8080). Full numbers and the NPU-prefill fork status: ENGINES.md.

Evidence

<table> <tr> <td align="center"><strong>Local vs cloud: TTFT and decode by tier</strong><br><img src="docs/charts/local-vs-cloud.png" alt="Flash-Next vs GLM 5.3 tiers"></td> <td align="center"><strong>Wall-clock per run, seven runs</strong><br><img src="docs/charts/wallclock-by-run.png" alt="Wall clock by run"></td> </tr> <tr> <td align="center"><strong>halogen 0.16.2 to 0.17.1: NPU latency 4000ms to 130ms, decode 44 to 64 tok/s</strong><br><img src="docs/charts/halogen-0171-upgrade.png" alt="halogen upgrade delta"></td> <td align="center"><strong>Every number in this README</strong><br>logged in <a href="docs/BENCHMARKS.md">docs/BENCHMARKS.md</a></td> </tr> </table>

Install

# one-time install: extensions, providers, sidecar, pi defaults
./setup.sh

# launch halogen from your local checkpoint (zero downloads)
./setup.sh --halogen auto

# status check any time, ends with a live model ping
./setup.sh --doctor

--halogen auto finds *.hgn in ., /models, ~/models, ~/Downloads, or take a path explicitly; restart later with ./start-halogen.sh (written on first launch); --no-halogen installs pi without container management (laptop pi against your own providers, or against a remote Strix Halo appliance via baseUrl + PI_NPU_BASE). Other flags: --voice, --halogen-upgrade [tag], --uninstall. Prerequisites: docker (setup will not sudo-install it), pi (setup installs it via npm if missing), uvx from uv for semantic search, and an IOMMU-enabled gfx1151 box for the NPU driver. The launch is pinned to local files (no HALOGEN_DOWNLOAD), so the engine makes no outbound connections.

[!NOTE] Do not clone the whole model repo — it carries ~190 GB of optional variants and the minimum set is ~124 GB. ./setup.sh --halogen auto offers to fetch exactly the right files and resumes where it stopped if interrupted.

Plain pi also works after setup: startup defaults point at halogen, and existing overrides are never touched.

Model files

From huggingface.co/peonist-ai/halogen-qwen3.8-flash-next, plus four separate NPU repos:

| file | size | why | |---|---|---| | qwen38-flash-next-v2.hgn | 66.7 GB | the main checkpoint (MTP drafter head included) | | qwen38-flash-next-ngram.hgn | 51.2 GB | required — the checkpoint's weight lookup table (read through the page cache; not an n-gram drafter despite the name) | | qwen38-flash-next-vision.hgn | 0.9 GB | vision tower — image input is enabled by default | | tokenizer/ | small | chat template the engine loads | | npu/decider-0.8b/ | 2.3 GB | triage — constrained decisions (~120 ms) | | npu/qwen3-embedding-0.6b/ | 848 MB | dedup_scan embeddings | | npu/qwen3guard-gen-0.6b/ | 787 MB | injection screening (auto-guard) | | npu/qwen3-reranker-0.6b/ | 821 MB | unused since v1.1 — loaded only because it is in the default HALOGEN_NPU_MODELS; drop it (see The NPU) |

One directory (e.g. ~/models/halogen-qwen3.8-flash-next/), NPU models in a lowercase npu/ subdirectory; the launcher mounts it read-only at /models inside the container, so the engine reads its NPU models from /models/npu/.

pip install -U "huggingface_hub[cli]"   # provides the hf command
hf download peonist-ai/halogen-qwen3.8-flash-next \
  qwen38-flash-next-v2.hgn qwen38-flash-next-ngram.hgn qwen38-flash-next-vision.hgn tokenizer/ \
  --local-dir ~/models/halogen-qwen3.8-flash-next
for n in decider-0.8b qwen3-embedding-0.6b qwen3guard-gen-0.6b qwen3-reranker-0.6b; do
  hf download peonist-ai/halogen-npu-$n --local-dir ~/models/halogen-qwen3.8-flash-next/npu/$n
done

Skip the rest (w4b.hgn, ht43.hgn, MTP.hgn, w4b.overlay*.hgn, qwen3.5-2b) — the checkpoint carries its own draft head. Upstream also ships an optional smaller/faster ht43.hgn checkpoint; the default stays v2 — pass its path explicitly to use it. First cold boot pins the 47.7 GiB lookup table from disk, so a few minutes of startup is normal.