The enumerated single-retry policy (#1419) has fired zero times since it
landed on 2026-07-29: 0 of 234 sampled Coverage-job lane envelopes
(2026-08-11 to 2026-08-18) have retryCount > 0, and none of 17 recent
failed runs was retried (5 refused "outside the enumerated retry list",
4 refused "unhandled error"). All three trackers its entries pointed at
(#1098, #1414, #1419) are closed. It cost ~1,454 LOC, a per-run secret
marker threaded through a setup file on every Vitest project, and a
standing obligation for every future gate reporter to call the blocker
bus.
Delete the scripts, tests and fixtures, the check:contention-retry
script and gate, the envelope artifact upload, and the runner-timeout
setup file; test:coverage:ci is a plain `vitest run --coverage` again.
lane-envelope.ts stays: the mutation, fuzz and concurrency-torture lanes
build their envelopes from it. run-blocker-bus.ts goes: its only
consumer was the retry's failure sink, and its only publisher already
fails the run by setting process.exitCode.
Keep the subprocess-stub project for the three files that really spawn
(client-metro, fuzz harness, fuzz corpus-replay) and drop the three that
run in 31/212/277ms in CI, which cannot contend for anything. The list
is now a plain array in vitest.config.ts with the reason at each entry.
Membership and the project's kill criterion live in #1823.
Because test:coverage:ci is a bare vitest run, the gate manifest reads
its projects directly, so OPAQUE_RUNNERS no longer needs it and an
unrun Vitest project becomes unrepresentable rather than detected; the
audit test now constructs that state by project-scoping the script.
* test(ci): single-retry policy for enumerated contention-flaky files
* fix: satisfy fallow
* test(ci): read failures through a lane reporter so timeouts stay distinguishable
* test(ci): cover the lane reporter and drop its duplicated boilerplate
* chore(fallow): own the retry lane's tool-loaded export seams
* test(ci): block retries on non-test failures and classify timeouts structurally
* test(ci): decide retry eligibility from runner metadata and route gate verdicts through blockers
* refactor(ci): name the retry policy's rules in code instead of comments
* test(ci): mark runner-aborted timeouts inside the runner instead of inferring them
* test(ci): make timeout provenance a per-run secret, not a writable flag
Cover direct task.meta mutation in the real child-Vitest fixture gate.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(ci): retry the failed files in the first run's project and coverage modes
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* fix: drop deleted repo-health file from the retry list after #1480
Rebase onto main post-#1480: the SkillGym/repo-health descope deleted
scripts/repo-health/run.test.ts, whose CONTENTION_RETRY_FILES entry
would now fail this PR's own missing-file check, and inlined the
slow-test budgets into the reporter, resolving the budgets-module
import. Envelope comments now point at scripts/lib/lane-envelope.ts
instead of the closed#1430.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
* chore: drop SkillGym and the repo-health aggregator (#1412 descope)
Remove the SkillGym harness (test/skillgym/), its check-affected lane,
package scripts, and devDependency — the help-conformance bench is now
the single non-gating small-model oracle. skills/ markdown classifies
as docs in the affected-check selector instead of failing open.
Remove scripts/repo-health: its only gating assertion duplicated the
Layering Guard job, its case-count metric imported the deleted SkillGym
suite, and its sole planned consumer (#1424 / PR #1477) was closed with
the Track C descope on #1412.
Verified: check-affected node --test suites, oxfmt, oxlint, tsc,
check:layering, fallow audit vs origin/main, and the full unit suite
(unit-core + subprocess-stub) all pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
* fix(scripts): fold slow-test budgets into the reporter for production-exports
The Fallow production-exports gate flagged all three budget exports:
their in-file consumer (SLOW_TEST_RATCHET) and the repo-health entry
point that kept the module reachable were both removed in the descope,
leaving the config-loaded reporter as the only consumer — invisible to
--production analysis. The data-only module's second consumer is gone,
so per the boundaries-are-earned norm the constants move into the
reporter instead of gaining a suppression.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
* docs: align skills/ format policy and purge last SkillGym mention
Address both P2 review findings on #1480: the testing-matrix row and
the selector's formatGate both still claimed oxfmt covers skills/,
while selectChecks classifies skills/*.md docs-only (oxfmt ignores
**/*.md, so the claim was a no-op even before). The matrix now states
the docs-only policy and formatGate drops the dead underSkills fact.
The merged examples/README.md index (from #1469) loses its skillgym
mention.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep
---------
Co-authored-by: Claude <noreply@anthropic.com>
* obs: add repo-health snapshot command aggregating existing analyzers into one JSON
Adds `pnpm repo-health [--json]`, an offline command that aggregates the
signals this repo already computes into one deterministic JSON snapshot by
reusing each analyzer (depgraph, layering ratchets, coverage json-summary,
size-report, fallow baselines, slow-test ratchet, bench/skillgym registries)
rather than reimplementing any metric.
Carries mandatory v1 provenance (schemaVersion, commit, per-analyzer content
hashes, input provenance) so #1424 can persist history. Component metrics
(instability/abstractness/main-sequence distance) are observatory-only. The
only gating behaviour is the depgraph-vs-layering R6 consistency assertion,
already wired into the Layering Guard job via scripts/depgraph/model.test.ts.
Closes#1423
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* repo-health: hash read artifacts and flag staleness in provenance
Coverage and size are artifacts repo-health reads but does not produce, so
they carry no producing-commit stamp. Previously provenance recorded only
their paths and the current HEAD, so a snapshot taken after a source edit
without rerunning those producers would pair prior metrics with the new SHA —
#1424 would persist a false commit-indexed history entry.
Now each read artifact is bound via artifactProvenance() to its content hash
(so history keys on the bytes the metrics came from, not a commit they may
predate) and an explicit `stale` flag (true when a production source file is
newer than the artifact). The coverage/size analyzer-config hashes are added
to provenance.tool, and the human summary marks stale artifacts. Adds a pure
stale-artifact regression test.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* repo-health: bind artifact freshness to the whole producer input set
The stale check derived freshness only from listSourceFiles() (src/*.ts) and
applied it to both coverage and size. Coverage also depends on tests and vitest
config; size-report.mjs reads package.json and runs npm pack — so a change to a
non-src producer input could leave an old artifact marked stale:false while
provenance recorded the current HEAD, a commit the metrics predate.
Each read artifact now declares its full producer input set (PRODUCER_INPUTS)
and is flagged stale when ANY tracked input in that set is newer than the
artifact (via git ls-files mtimes). Freshness that cannot be proven — an empty
observable input set — is reported stale, never falsely fresh. The input set is
recorded as provenance.inputs.*.producerInputs. Adds isArtifactStale() with a
regression covering a non-src input newer than the artifact and the unprovable
case.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* repo-health: prove artifact freshness from a stamped producer commit; route git through runCmdSync
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): add Renovate config and enforce packageManager pnpm version in CI
Refs #1422
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: bump pnpm to 11.17.0 and format the whole repo with oxfmt
format/format:check drop their hand-maintained path list: oxfmt already skips
node_modules and honors .gitignore, so the only exclusion list is
.oxfmtrc.json ignorePatterns.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mutation): accept either quote style in the affected-lane path filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): keep fixture-app runtime deps as individual Renovate PRs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test: split the Android platform test aggregation and share the scripted adb stub
AGENTS.md names the platform index.test.ts aggregations as offenders to
shrink opportunistically; this splits the 2,735-line Android one along
its (already well-factored) source modules, every test moved verbatim
(92 tests before and after):
- ui-hierarchy.test.ts (22): parseUiHierarchy/androidUiNodes
- app-lifecycle-install.test.ts (13): install/resolve/infer/launch
component parsing
- app-lifecycle-open.test.ts (19): open/close, deep links, launch args,
TV category, fallback resolve-activity
- input-actions.test.ts (11): type/fill/swipe/scroll/rotate
- settings.test.ts (14): appearance/clear-app-state/fingerprint/
permissions
- notifications.test.ts (2), app-parsers.test.ts (1)
- keyboard state/dismiss tests (10) appended to the existing
device-input-state.test.ts
Consistency fix folded in: the file carried a local withMockedAdb fork
because it needs scripted per-subcommand adb responses, which the shared
arg-recorder helper cannot express. The fork now lives in
src/__tests__/test-utils/mocked-binaries.ts as withScriptedAdb next to
withMockedAdb, and hands each call a fresh copy of the shared
ANDROID_EMULATOR fixture.
The copy matters: the Android TV test mutated the callback's device
(device.target = 'tv'), which the old per-call object literal absorbed
silently. With a shared fixture that mutation leaked into the next test
and flipped its launch to LEANBACK. The helper now clones per call and
the TV test builds { ...device, target: 'tv' } instead of mutating.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS
* test: serialize the scripted-adb group and repoint its slow-test pins
Review follow-up for the android index.test.ts split: the monolith
implicitly serialized the env-mutating adb-stub tests (PATH,
AGENT_DEVICE_TEST_ARGS_FILE) in one worker, and the split let vitest
run them across parallel files. Make the contract explicit:
- new android-adb vitest project runs the six scripted-adb test files
in a single fork (singleFork), keeping the pre-split execution
semantics; ui-hierarchy and app-parsers stay in the parallel unit
project (pure parsing, no env mutation)
- test/test:unit scripts run both projects
- the five slow-test ratchet pins that referenced index.test.ts keys
now point at the split file names, so the pinned real-time offenders
keep their exemption instead of failing at 2x budget under load; the
reporter's own pinned-key fixture updated to match
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FqeW8sA2ZnvnftdvpCqFMS
* test: use vitest 4 android adb serialization
* docs: update unit project readiness guidance
---------
Co-authored-by: Claude <noreply@anthropic.com>
* test: slow-test ratchet, budget-derived emulator poll, speed guidance from experiments
Measured (2026-07-04, full unit suite: 340 files / 3,210 tests / 48s wall):
wall clock was bounded by the slowest FILE (44.6s android monolith at ~7x
file-level parallelism), and the slowest tests were sleeping through real
production budgets (10.8s proving 'times out' by waiting the constant out,
8s emulator polls at 1Hz, real retry backoff). Two config experiments
rejected with data: --no-isolate exploded the suite to 205s (module state
thrashes across files sharing workers) and --pool=threads changed nothing.
- scripts/vitest-slow-test-reporter.ts: the slow-test ratchet. Unit budget
2.5s / integration 15s; failure at 2x budget (the band between reports
without failing so host-load variance cannot make the gate cry wolf);
36 pinned offenders, exact keys, ratchet-only pin (tracking #1098).
- waitForAndroidEmulatorByAvdName: poll cadence derives from the caller's
budget (min 1s, floor 50ms, ~timeout/20) — devices.test.ts 25.6s -> 2.8s
(9x) in isolation, and short-budget production calls stop sampling at
1Hz against small budgets.
- vitest.config: slowTestThreshold 500 for local visibility; reporter
wired; isolation/pool decisions documented with the measurements.
- docs/agents/testing.md 'Speed rules' + AGENTS.md testing bullet: the
three conversion patterns in preference order (budget-derived cadence,
budget-wiring assertion, fake clocks), the no-seam constraint, and the
file-granularity Amdahl argument that makes the monolith test split a
wall-clock fix, not just navigation.
* fix: fallow findings on the slow-test gate — import edge, factory reporter, unit tests
The string-path reporter wiring read as a dead file (fallow cannot see
vitest's reporter loading); the config now imports the factory, making
the edge real and type-checked. The class shape tripped the
unused-class-members rule (framework callbacks are invisible to
reference analysis) — converted to a factory returning the Reporter
object, with the classification and rendering logic extracted as pure
exported functions. Those functions now carry their own unit tests
(budget bands, integration budgets, pin matching, warn-vs-fail
rendering), which also grounds the CRAP estimate in real references.
Canary re-verified: unpinned 5.2s sleeper fails the run with exit 1;
clean runs exit 0.
Michał Pierzchała