Commit Graph

5 Commits

Author SHA1 Message Date
Michał Pierzchała 80feff42d6 build: verify the published tarball instead of grepping the bundle (#1578)
* build: verify the published tarball instead of grepping the bundle

Replaces the bundle-dependency grep with one gate that packs the tarball npm
would publish and proves it sound from a clean consumer install: publint and
attw on the tarball, a two-way dependency-closure audit, an import of every
`exports` subpath, and the CLI smoke run — all from outside the workspace,
where no pnpm link can mask an unresolvable specifier.

Also stops the build from emitting a publishable bundle in the first place: a
missing workspace link now fails `pnpm build` instead of warning and exiting 0,
which is how 0.20.4 shipped an unresolvable `@agent-device/ad-script` import.

publint found 12 real defects in the current package — every `exports` entry
listed `types` after `import`, so TypeScript resolved declarations by accident
rather than by condition. The dependency audit found `pngjs` declared as a
runtime dependency while tsdown inlines it, an install every user paid for and
no shipped code reached; it moves to devDependencies.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD

* fix(ci): run the package gate without pnpm on the Node floor

pnpm 11.17 requires Node >= 22.13, so `pnpm check:package` could not start on
the 22.12 floor the Packaged CLI job exists to cover. The gate needs only `node`
and `npm`, so the job invokes the script directly.

Splits the dependency-closure audit into a collector and a message builder to
clear Fallow's complexity threshold, and classifies both packaging linters in
ignoreDependencies: they are subprocess CLIs with no importable API here, which
dependency analysis cannot follow to an import.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD

* fix(publishing): read every literal resolution form in the closure audit

The dependency-closure audit derived shipped imports from the ESM module
record alone, so it could not see a package resolved through `require` or a
`createRequire` result: neither produces a module-record entry. A lazy
`createRequire('@agent-device/…')` would therefore clear the audit, the
all-export probe and the exercised CLI paths, reintroducing the 0.20.4
published-install failure class for another command.

Measuring the built bundle turned up a second, larger hole in the same
reader. The shipped files are minified, and the minifier rewrites every
string literal to a no-substitution template literal, so the dynamic-import
extraction — which accepted quoted strings only — matched 0 of the 99
dynamic imports the bundle contains. The lazy `import()` path that broke
0.20.4 was reported as covered while checking nothing.

Specifiers now come from the module record plus an AST walk over every
literal runtime-resolution form: `import()`, `require()`,
`require.resolve()`, an immediately-invoked `createRequire(...)`, and calls
through a `createRequire` result under any import or minified alias. Both
spellings of a string literal count everywhere, and `.cjs` joins the
scanned extensions.

Computed specifiers stay explicitly out of scope, and are pinned as such.
Rejecting them is not available: minifiers reuse short identifiers across
scopes, and the packed bundle really does contain an unrelated
`a(h[t],f,g,l,e,m)` that no name-based match can distinguish from a require
call. Those are covered by the gate's runtime half instead, which resolves
them for real. Bare-identifier calls need the one-string-argument shape for
the same reason.

The audit moves to scripts/lib/shipped-imports.ts so fixture packages can
exercise it. The gate needs a real `npm pack` behind minutes of Swift and
Android builds, so every check that runs it can only watch a healthy
package pass — which is how a reader that matched nothing looked covered.
The new fixtures assert the failure direction per resolution form: 16 of
the 22 fail against the previous reader, and the 6 that pass are the
quoted-spelling and pinned-limitation cases. A wiring assertion keeps the
audit and both runtime probes attached to the gate, since fixtures alone
would stay green if the call were deleted.

Verified against the real built bundle: the closure resolves to exactly the
two declared dependencies, so the stricter reader adds no false positives.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NamFJUgn9DGHrT2za11JbD

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-08-04 12:28:05 +02:00
devin-ai-integration[bot] 885c1486bb test(ci): single-retry policy for enumerated contention-flaky files (timeouts only) (#1448)
* test(ci): single-retry policy for enumerated contention-flaky files

* fix: satisfy fallow

* test(ci): read failures through a lane reporter so timeouts stay distinguishable

* test(ci): cover the lane reporter and drop its duplicated boilerplate

* chore(fallow): own the retry lane's tool-loaded export seams

* test(ci): block retries on non-test failures and classify timeouts structurally

* test(ci): decide retry eligibility from runner metadata and route gate verdicts through blockers

* refactor(ci): name the retry policy's rules in code instead of comments

* test(ci): mark runner-aborted timeouts inside the runner instead of inferring them

* test(ci): make timeout provenance a per-run secret, not a writable flag

Cover direct task.meta mutation in the real child-Vitest fixture gate.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(ci): retry the failed files in the first run's project and coverage modes

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: drop deleted repo-health file from the retry list after #1480

Rebase onto main post-#1480: the SkillGym/repo-health descope deleted
scripts/repo-health/run.test.ts, whose CONTENTION_RETRY_FILES entry
would now fail this PR's own missing-file check, and inlined the
slow-test budgets into the reporter, resolving the budgets-module
import. Envelope comments now point at scripts/lib/lane-envelope.ts
instead of the closed #1430.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FUv7bvbWNryuXgSBuqTtep

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-07-29 12:53:33 +02:00
devin-ai-integration[bot] 6544e9a0c5 obs: repo-health snapshot command aggregating existing analyzers into one JSON (#1471)
* obs: add repo-health snapshot command aggregating existing analyzers into one JSON

Adds `pnpm repo-health [--json]`, an offline command that aggregates the
signals this repo already computes into one deterministic JSON snapshot by
reusing each analyzer (depgraph, layering ratchets, coverage json-summary,
size-report, fallow baselines, slow-test ratchet, bench/skillgym registries)
rather than reimplementing any metric.

Carries mandatory v1 provenance (schemaVersion, commit, per-analyzer content
hashes, input provenance) so #1424 can persist history. Component metrics
(instability/abstractness/main-sequence distance) are observatory-only. The
only gating behaviour is the depgraph-vs-layering R6 consistency assertion,
already wired into the Layering Guard job via scripts/depgraph/model.test.ts.

Closes #1423

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* repo-health: hash read artifacts and flag staleness in provenance

Coverage and size are artifacts repo-health reads but does not produce, so
they carry no producing-commit stamp. Previously provenance recorded only
their paths and the current HEAD, so a snapshot taken after a source edit
without rerunning those producers would pair prior metrics with the new SHA —
#1424 would persist a false commit-indexed history entry.

Now each read artifact is bound via artifactProvenance() to its content hash
(so history keys on the bytes the metrics came from, not a commit they may
predate) and an explicit `stale` flag (true when a production source file is
newer than the artifact). The coverage/size analyzer-config hashes are added
to provenance.tool, and the human summary marks stale artifacts. Adds a pure
stale-artifact regression test.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* repo-health: bind artifact freshness to the whole producer input set

The stale check derived freshness only from listSourceFiles() (src/*.ts) and
applied it to both coverage and size. Coverage also depends on tests and vitest
config; size-report.mjs reads package.json and runs npm pack — so a change to a
non-src producer input could leave an old artifact marked stale:false while
provenance recorded the current HEAD, a commit the metrics predate.

Each read artifact now declares its full producer input set (PRODUCER_INPUTS)
and is flagged stale when ANY tracked input in that set is newer than the
artifact (via git ls-files mtimes). Freshness that cannot be proven — an empty
observable input set — is reported stale, never falsely fresh. The input set is
recorded as provenance.inputs.*.producerInputs. Adds isArtifactStale() with a
regression covering a non-src input newer than the artifact and the unprovable
case.

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* repo-health: prove artifact freshness from a stamped producer commit; route git through runCmdSync

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-28 20:07:45 +02:00
devin-ai-integration[bot] 8cce0ef6b8 test: ratchet mutation score over enumerated decision kernels (#1441)
* test: ratchet mutation score over enumerated decision kernels

Adds a Stryker (vitest runner) mutation lane scoped to the decision kernels,
a per-module baseline with tool/config provenance, and a ratchet that only
lets scores rise. Non-gating until two consecutive stable weekly sweeps.

Refs #1415

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* chore: declare the mutation test-scope seam for production-export analysis

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: own kernel tests in the mutation registry and ship the #1430 lane envelope

- restore bench:help-conformance, broken by a formatting-path edit
- kernel test files select their module on PRs (registry `tests` + workflow paths),
  asserted to reach the kernel through the import graph
- every mutation run writes the standard scheduled-lane artifact envelope
- move src/utils/__tests__/errors.test.ts beside its source per the mirror rule

Refs #1415, #1430

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test: derive kernel test ownership and land the scheduled-lane health monitor

Ownership of a kernel's tests is now computed from the static import graph
(scripts/mutation/ownership.ts) instead of a hand-listed set, so a test that
reaches a kernel indirectly -- src/__tests__/daemon-error.test.ts through
src/daemon.ts -- selects that kernel on a PR. The PR lane triggers on every src
test and shards the derived modules, keeping wall clock at one module.

The lane envelope (#1430) is now written on every exit path with the stage it
reached, so a crash before any mutant runs is distinguishable from a lane that
never ran. Adds the derived cadence monitor (scripts/lane-health, daily
workflow): scheduled lanes are enumerated from .github/workflows/ and reported
dark, failing, or never-run against their own cron cadence.

* fix: merge only Stryker reports from a shard directory

The shard artifacts now carry the lane envelope beside mutation.json, and the
merge globbed every .json under the download path, so the ratchet job fed the
envelope to the report parser and died after the mutants had already run.

* fix(mutation): fail on incomplete shard sets and envelope pre-run failures

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mutation): downgrade a passing envelope when a later lane step fails

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* refactor(mutation): shard by registry, defer the PR lane, drop the bundled watcher

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs: describe registry sharding and the deferred PR mutation lane

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix(mutation): make the pre-graduation tooling exception select real mutants

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* test(mutation): give the worktree fixture commits their own identity

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
2026-07-28 10:07:14 +02:00
Michał Pierzchała d237bc555d chore: remove verified dead code and migration scaffolding (~700 LOC) (#1367)
* chore: remove verified dead code and migration scaffolding

Multi-agent audit of accumulated waste, every finding adversarially
verified against call sites, git history, and the published surface
before removal. Net -710 lines.

- delete src/core/platform-descriptor/ (superseded ADR-0009 migration
  scaffold; parity tests now assert an inline table)
- remove test-only seams: registry introspection exports,
  CommandFacet.extraDaemonWriters, MaestroEngineOptions.timing
- remove dead flexibility: backend capability allow-list,
  screenshot-diff maxRegions, CloudWebDriverSupportLevel 'partial',
  clearFirst on the TS+Swift runner wire contract
- remove dead deprecated surface: --session-locked /
  --session-lock-conflicts aliases (hard migration error now points at
  --session-lock), replay export --format single-value enum,
  unused Lease*Payload contract types, runtime-layer rotate duplicate
- collapse pass-throughs/duplication: withRetry adapter,
  default-cloud-artifact-provider, connect-profile client-id hashing
  (3x sha256 impls -> one helper, byte-identical output), shared
  scripts walker, cloneValue -> structuredClone, fill-diagnostics
  moved into android/

BREAKING CHANGE: --session-locked and --session-lock-conflicts now fail
with a migration error pointing at --session-lock; replay export
--format is removed (Maestro was the only value); Lease*Payload types
are dropped from the ./contracts subpath.

* chore: satisfy fallow gates tightened by #1363/#1364 after rebase

- drop the consumer-less AndroidFillVerificationNode re-export
- reuse requireSnapshotSession in resolveSnapshotForRef instead of
  inlining the same authorized-frame resolution (fallow clone group);
  the helper's return type now guarantees the session it already
  throws for

* chore: address review — keep cloud-webdriver partial capability metadata

The partial/supported/unsupported levels and their notes are part of the
lease-response capability contract for genuinely limited operations
(Appium page-source snapshots, upload-then-install), not dead
scaffolding. Restore them and the asserting tests unchanged from main.

Also add the missing CHANGELOG entry for the Lease*Payload type removal
from agent-device/contracts.
2026-07-23 07:36:52 +02:00