Home Keeper — testing & workflow conventions

Git & PR workflow

• Never push directly to main. Work on a feature branch and open a PR; squash merge.
• Update CHANGELOG.md for every user-facing change before a release.
• Post screenshots to the PR for any change that adds/changes/fixes UI (capture via tests/e2e/screenshots.capture.ts, commit under docs/images/, embed via a raw.githubusercontent.com/.../<commit-sha>/docs/images/<file>.png URL).
• The video walkthrough is a CI build artifact, never committed — for a PR that adds a new user-facing UI feature, CI keeps it current (bug-fix/styling PRs need only screenshots). walkthrough-preview.yml runs the capture harness (tests/e2e/walkthrough.capture.ts → walkthrough.config.ts, wrapped by ci/capture-video.sh) on every PR, transcodes to gif+mp4, publishes them to the gh-pages pr-preview-media/pr-<n>/ umbrella (GitHub Pages), and posts a sticky PR comment embedding the gif with an mp4 link. docs/videos/ is gitignored, so there's zero git bloat. The author's gate is editing the tour: extend walkthrough.capture.ts for a new surface in the same PR and confirm the regenerated comment shows it; capture is a soft gate (a flaky run posts a failure note, doesn't block). Run ci/capture-video.sh locally only to debug the tour.
• Document new major features in README.md in the same change — add a brief section covering the use cases (what problem it solves) and a little about how it's used, with screenshot(s) (capture via the Playwright harness, commit under docs/images/, embed in the README with a relative docs/images/… path). A new headline feature isn't "done" until the README shows it. (The moving walkthrough is not committed to the README — it's the per-PR CI comment above.)
• User-facing prose is linted for AI-tell phrasing. lint.yml's vale job runs the vale-ai-tells style (pinned version in .vale.ini) over README.md, CHANGELOG.md, the canonical docs/*.md (excludes *_PLAN.md/research scratch docs), website/docs/intro.md, strings.json, services.yaml, and locales/en.json (not the other locales or translations/, since the rules are English-phrase regexes). It's diff-scoped (filter_mode: added), so only new/changed lines can fail CI. The existing corpus is cleaned up separately. Run locally with vale sync && vale <paths>. Disable an accepted false positive per-file in .vale.ini (ai-tells.RuleName = NO) or inline with <!-- vale ai-tells.RuleName = NO --> / ... = YES -->. For example, services.yaml disables ColonUsage, which otherwise fires on every unquoted YAML key: Value line. Diff-scoping misses pre-existing hits on lines a full-file prose rewrite happens to move, so run vale <file> yourself first for that case. The pinned ai-tells.zip version has no bump automation (Dependabot/Renovate don't track raw release URLs), so bump it by hand periodically.

Tests (run locally before pushing — never use CI as the test runner)

• The recurrence engine and model are the correctness core: keep them HA-free and thoroughly unit-tested. pytest tests/unit must run without the HA harness. tests/conftest.py executes the pure modules under their real dotted name (custom_components.home_keeper.<mod>, with stub parent packages so the HA-importing __init__.py never runs) and registers hk.<mod> / hk_<mod> as aliases. Two invariants there: mutmut matches a mutant's path-derived key against the function's __module__, so executing them as hk.<mod> would make every mutant look untested; and hk must stay a distinct package object, not an alias of custom_components.home_keeper, because from . import x resolves through the parent's __name__ — aliasing them makes the modules test_coordinator_purge.py / test_calendar.py load as hk.coordinator pull in the real HA-importing siblings instead of their fakes.
• Layers: tests/unit (pytest, pure logic), tests/frontend + frontend/test (vitest), tests/integration (Docker HA), tests/e2e (Playwright), tests/upgrade (two-phase HA version upgrade). Run e2e/integration with bash ci/e2e-up.sh / ci/test-python-integration.sh; stage the upgrade suite's fixtures with bash ci/fetch-glues.sh first.
• A panel assertion is not coverage for a native entity. The panel and the todo/calendar entities are separate projections of the same store, so the panel being right proves nothing about them. #221 shipped with a passing e2e test that created a one-off, completed it, and asserted the panel filed it under Completed — while the to-do entity went on offering it as needs_action forever. A state change that should be visible on a native surface needs an assertion on that surface.
• Assert disappearance, not just appearance. Presence gets asserted by accident; absence has to be asked for, and the interesting bugs are absences that didn't happen. Test a state transition from both ends — present before, gone after — via expectAbsentFromActiveSurfaces / expectOnTodoList in tests/e2e/tests/helpers.ts. Asserting only the post-state also passes for a task that was never listed at all.
• A screenshot is documentation, not verification. The capture harness wrote docs/images/4-usage-todo-and-calendar.png showing #221 in plain sight — stale to-do items beside panel columns marking those same tasks Completed — for months. Capturing a surface is not covering it; if a screenshot shows a surface, something should be asserting on it too.
• An e2e spec owns what it creates. The container's task store is the committed seed fixture (tests/integration/ha_config/.storage/home_keeper), so anything a spec leaves behind is a permanent addition to it. Register created ids and delete them in afterEach (createTask/deleteTask in helpers.ts), and give fixtures stable names — a Date.now() suffix makes each leak look like a new record instead of the same spec failing to clean up, which is how eight of them reached git.
• Anything that rests on an HA framework contract — device registry, entity registry, device automation — needs an integration-level assertion. Unit tests mock the framework away and cannot see the contract change. #183 (devices split per config entry in HA 2026.8) shipped because the only device-attachment coverage was for the self-owned case, never the foreign-device one.
• Cross-version behaviour needs an upgrade test, not just a fresh-boot test. tests/upgrade boots a frozen pre-split HA, seeds every scenario into one config dir, then boots the current HA against that same dir so HA runs its own migration in between — two cold starts for the whole suite. The pre-split tag is a frozen pin: it defines "the world users upgrade from", so bumping it changes the meaning of the test.
• A test must exercise the shipped function, never a copy of it. Re-implementing the logic under test inside the test file (to dodge an import) proves nothing: the production code keeps zero coverage and every later edit to it stays green. An HA-importing module is still unit-testable — test_calendar.py, test_coordinator_purge.py and test_device_heal.py stub the HA symbols the module imports, register fakes for its HA-aware siblings, load the real file under hk.<mod>, then inject fakes by patching the loaded module's bindings. Follow that pattern instead of duplicating the source.
• Check that a new test can fail. Mutate the line it covers and confirm it goes red before relying on it. A test whose fake can only produce the passing case (e.g. a mock registry that returns one candidate, "verifying" a preference between several) is worse than no test: it reports coverage the code does not have.
• Never commit a real .storage dump as a fixture. Production snapshots carry serial numbers, MAC addresses, document links and other household data, and they live forever in git history. Build fixtures from synthetic data, and wire every fixture into a test — an unreferenced fixture is only a leak with no upside.
• Known-broken contracts get xfail(strict=True), never a weakened assertion. The test then documents the breakage without going red, and becomes a hard failure the moment a fix lands, forcing the marker off.
• HA versions: PRs run stable (HA_TAG in tests/integration/docker-compose.yml); ha-beta.yml runs beta nightly as an early warning and gates nothing.
• After running the Docker HA container locally, restore the seeded fixtures (tests/integration/ha_config/.storage/{home_keeper,core.config_entries}); don't commit runtime-mutated state.
• A second delivery path needs a test that deletes the first one. #228 was invisible to a suite where every dashboard test loaded a freshly-rendered app shell that happened to carry the card's import. Don't wait for a stale cache — reproduce what one is: tests/e2e/tests/card-registration.spec.ts intercepts the dashboard navigation with page.route, strips the card's import(...) out of the HTML, and asserts the card still renders. Two things make it honest. It sets test.use({ serviceWorkers: 'block' }), because a service worker answers navigations before page.route sees them, and HA registers one on first load — so without it the reload that follows is served the original shell and the test passes for the wrong reason. And it asserts the unstripped HTML did contain the import, so the test cannot quietly go vacuous if HA changes how it delivers extra modules. It deliberately does not use openCardDashboard: that helper reloads up to 3x to absorb cold-frontend flake, which here would only re-serve the stripped shell while turning a precise failure into an opaque timeout.
• Verify a browser-sensitive e2e spec with the browser CI actually uses. e2e.yml runs npx playwright install chromium and no CHROMIUM_EXEC, so CI drives Playwright's headless shell; the CHROMIUM_EXEC override documented in AGENTS.md for the Claude Code remote environment points at a different, older full Chromium. card-registration.spec.ts passed locally and failed on CI three times for exactly that reason. Re-run a spec with CHROMIUM_EXEC unset (CI=true npx playwright test <spec>) before trusting it.
• A spec that rewrites a document needs the Local Network Access flag. Chrome classifies a response synthesized by route.fulfill as coming from a public address space, then blocks the page's own ws://localhost:8123/api/websocket as a local-network request (net::ERR_BLOCKED_BY_LOCAL_NETWORK_ACCESS_CHECKS). The frontend never connects, so nothing websocket-delivered — Lovelace resources included — ever loads, and the failure looks like the feature under test is broken. Pass --disable-features=LocalNetworkAccessChecks in that spec's own test.use({ launchOptions }), not in playwright.config.ts: only a spec that rewrites a document needs it, and every other spec should keep the check so a future test of network or CORS behaviour still gets it. Note launchOptions replaces the config's copy rather than merging, so the spec has to re-plumb CHROMIUM_EXEC itself — see tests/e2e/tests/card-registration.spec.ts.
• Give an e2e assertion that depends on browser plumbing a failure message that names what it saw. The above took a CI round-trip per guess until the spec captured console errors and whether the bundle was requested at all; "the bundle was never requested" is the line that ended it. A bare waitFor timeout says only that something, somewhere, did not happen.

Mutation testing (a PR gate)

Coverage proves a line ran; mutation testing proves a test woul