Files
callstack__agent-device/scripts/integration-progress-model.ts
T

686 lines
24 KiB
TypeScript
Raw Normal View History

refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
import fs from 'node:fs';
import path from 'node:path';
import { PUBLIC_COMMANDS } from '../src/command-catalog.ts';
import { listCommandMetadata } from '../src/commands/command-metadata.ts';
import { getFlagDefinitions } from '../src/commands/cli-grammar/flag-registry.ts';
chore: remove verified dead code and migration scaffolding (~700 LOC) (#1367) * chore: remove verified dead code and migration scaffolding Multi-agent audit of accumulated waste, every finding adversarially verified against call sites, git history, and the published surface before removal. Net -710 lines. - delete src/core/platform-descriptor/ (superseded ADR-0009 migration scaffold; parity tests now assert an inline table) - remove test-only seams: registry introspection exports, CommandFacet.extraDaemonWriters, MaestroEngineOptions.timing - remove dead flexibility: backend capability allow-list, screenshot-diff maxRegions, CloudWebDriverSupportLevel 'partial', clearFirst on the TS+Swift runner wire contract - remove dead deprecated surface: --session-locked / --session-lock-conflicts aliases (hard migration error now points at --session-lock), replay export --format single-value enum, unused Lease*Payload contract types, runtime-layer rotate duplicate - collapse pass-throughs/duplication: withRetry adapter, default-cloud-artifact-provider, connect-profile client-id hashing (3x sha256 impls -> one helper, byte-identical output), shared scripts walker, cloneValue -> structuredClone, fill-diagnostics moved into android/ BREAKING CHANGE: --session-locked and --session-lock-conflicts now fail with a migration error pointing at --session-lock; replay export --format is removed (Maestro was the only value); Lease*Payload types are dropped from the ./contracts subpath. * chore: satisfy fallow gates tightened by #1363/#1364 after rebase - drop the consumer-less AndroidFillVerificationNode re-export - reuse requireSnapshotSession in resolveSnapshotForRef instead of inlining the same authorized-frame resolution (fallow clone group); the helper's return type now guarantees the session it already throws for * chore: address review — keep cloud-webdriver partial capability metadata The partial/supported/unsupported levels and their notes are part of the lease-response capability contract for genuinely limited operations (Appium page-source snapshots, upload-then-install), not dead scaffolding. Restore them and the asserting tests unchanged from main. Also add the missing CHANGELOG entry for the Lease*Payload type removal from agent-device/contracts.
2026-07-23 07:36:52 +02:00
import { walkFiles } from './lib/walk-files.ts';
const EMPTY_COVERAGE_METRIC = { pct: 0 };
const EMPTY_STATEMENT_COVERAGE = { covered: 0, pct: 0, total: 0 };
export function buildIntegrationProgressModel({ root = process.cwd() } = {}) {
const coverageSummary = path.join(root, 'coverage/coverage-summary.json');
const handlerTestDir = path.join(root, 'src/daemon/handlers/__tests__');
const providerScenarioDir = path.join(root, 'test/integration/provider-scenarios');
chore: remove verified dead code and migration scaffolding (~700 LOC) (#1367) * chore: remove verified dead code and migration scaffolding Multi-agent audit of accumulated waste, every finding adversarially verified against call sites, git history, and the published surface before removal. Net -710 lines. - delete src/core/platform-descriptor/ (superseded ADR-0009 migration scaffold; parity tests now assert an inline table) - remove test-only seams: registry introspection exports, CommandFacet.extraDaemonWriters, MaestroEngineOptions.timing - remove dead flexibility: backend capability allow-list, screenshot-diff maxRegions, CloudWebDriverSupportLevel 'partial', clearFirst on the TS+Swift runner wire contract - remove dead deprecated surface: --session-locked / --session-lock-conflicts aliases (hard migration error now points at --session-lock), replay export --format single-value enum, unused Lease*Payload contract types, runtime-layer rotate duplicate - collapse pass-throughs/duplication: withRetry adapter, default-cloud-artifact-provider, connect-profile client-id hashing (3x sha256 impls -> one helper, byte-identical output), shared scripts walker, cloneValue -> structuredClone, fill-diagnostics moved into android/ BREAKING CHANGE: --session-locked and --session-lock-conflicts now fail with a migration error pointing at --session-lock; replay export --format is removed (Maestro was the only value); Lease*Payload types are dropped from the ./contracts subpath. * chore: satisfy fallow gates tightened by #1363/#1364 after rebase - drop the consumer-less AndroidFillVerificationNode re-export - reuse requireSnapshotSession in resolveSnapshotForRef instead of inlining the same authorized-frame resolution (fallow clone group); the helper's return type now guarantees the session it already throws for * chore: address review — keep cloud-webdriver partial capability metadata The partial/supported/unsupported levels and their notes are part of the lease-response capability contract for genuinely limited operations (Appium page-source snapshots, upload-then-install), not dead scaffolding. Restore them and the asserting tests unchanged from main. Also add the missing CHANGELOG entry for the Lease*Payload type removal from agent-device/contracts.
2026-07-23 07:36:52 +02:00
const commandContractFiles = walkFiles(path.join(root, 'src/commands'), (file) =>
isCommandContractSource(file),
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
);
const clientCommandMethods = readClientCommandMethods(commandContractFiles);
chore: remove verified dead code and migration scaffolding (~700 LOC) (#1367) * chore: remove verified dead code and migration scaffolding Multi-agent audit of accumulated waste, every finding adversarially verified against call sites, git history, and the published surface before removal. Net -710 lines. - delete src/core/platform-descriptor/ (superseded ADR-0009 migration scaffold; parity tests now assert an inline table) - remove test-only seams: registry introspection exports, CommandFacet.extraDaemonWriters, MaestroEngineOptions.timing - remove dead flexibility: backend capability allow-list, screenshot-diff maxRegions, CloudWebDriverSupportLevel 'partial', clearFirst on the TS+Swift runner wire contract - remove dead deprecated surface: --session-locked / --session-lock-conflicts aliases (hard migration error now points at --session-lock), replay export --format single-value enum, unused Lease*Payload contract types, runtime-layer rotate duplicate - collapse pass-throughs/duplication: withRetry adapter, default-cloud-artifact-provider, connect-profile client-id hashing (3x sha256 impls -> one helper, byte-identical output), shared scripts walker, cloneValue -> structuredClone, fill-diagnostics moved into android/ BREAKING CHANGE: --session-locked and --session-lock-conflicts now fail with a migration error pointing at --session-lock; replay export --format is removed (Maestro was the only value); Lease*Payload types are dropped from the ./contracts subpath. * chore: satisfy fallow gates tightened by #1363/#1364 after rebase - drop the consumer-less AndroidFillVerificationNode re-export - reuse requireSnapshotSession in resolveSnapshotForRef instead of inlining the same authorized-frame resolution (fallow clone group); the helper's return type now guarantees the session it already throws for * chore: address review — keep cloud-webdriver partial capability metadata The partial/supported/unsupported levels and their notes are part of the lease-response capability contract for genuinely limited operations (Appium page-source snapshots, upload-then-install), not dead scaffolding. Restore them and the asserting tests unchanged from main. Also add the missing CHANGELOG entry for the Lease*Payload type removal from agent-device/contracts.
2026-07-23 07:36:52 +02:00
const handlerTests = walkFiles(handlerTestDir, (file) => file.endsWith('.test.ts'));
const providerScenarioTests = walkFiles(providerScenarioDir, (file) => file.endsWith('.test.ts'));
const providerScenarioSources = walkFiles(providerScenarioDir, (file) => file.endsWith('.ts'));
const providerScenarioSupportSources = providerScenarioSources.filter(
(file) => !file.endsWith('.test.ts'),
);
const handlerStats = summarizeFiles(handlerTests);
const providerScenarioStats = summarizeFiles(providerScenarioTests);
const providerScenarioSupportStats = summarizeFiles(providerScenarioSupportSources);
const mockHeavyHandlerFiles = handlerTests.filter((file) =>
fs.readFileSync(file, 'utf8').includes('vi.mock('),
);
const mockHeavyHandlerRows = summarizeMockHeavyHandlerFiles(root, mockHeavyHandlerFiles);
const providerPressureRows = summarizeProviderPressure(providerScenarioSources);
const publicCommandRows = summarizePublicCommandCoverage(
providerScenarioTests,
clientCommandMethods,
);
const missingPublicCommands = publicCommandRows.filter((command) => command.references === 0);
const flagCoverageRows = summarizeProviderScenarioFlagCoverage(providerScenarioTests);
const missingFlagRows = flagCoverageRows.filter((flag) => flag.references === 0);
const excludedFlagRows = summarizeProviderScenarioFlagExclusions();
const publicCliFlagKeys = readPublicCliFlagKeys();
const classifiedFlagKeys = new Set([
...flagCoverageRows.map((flag) => flag.key),
...excludedFlagRows.flatMap((group) => group.keys),
]);
const unclassifiedFlagKeys = [...publicCliFlagKeys].filter((key) => !classifiedFlagKeys.has(key));
const coverage = readCoverageSummary(coverageSummary);
const lowCoverageFiles = readLowCoverageFiles(root, coverageSummary);
const summaryRows = [
['Handler unit test files', String(handlerStats.files)],
['Handler unit test LOC', String(handlerStats.lines)],
['Handler unit tests', String(handlerStats.tests)],
['Handler files with vi.mock', String(mockHeavyHandlerFiles.length)],
['Provider scenario files', String(providerScenarioStats.files)],
['Provider scenario LOC', String(providerScenarioStats.lines)],
['Provider scenario tests', String(providerScenarioStats.tests)],
['Provider scenario support files', String(providerScenarioSupportStats.files)],
['Provider scenario support LOC', String(providerScenarioSupportStats.lines)],
['Provider scenario / handler LOC', ratio(providerScenarioStats.lines, handlerStats.lines)],
[
'Public commands covered by provider-backed integration',
`${publicCommandRows.length - missingPublicCommands.length}/${publicCommandRows.length}`,
],
[
'Public commands missing provider-backed integration coverage',
String(missingPublicCommands.length),
],
[
'Device-observable workflow flags covered by provider-backed integration',
`${flagCoverageRows.length - missingFlagRows.length}/${flagCoverageRows.length}`,
],
[
'Device-observable workflow flags missing provider-backed integration coverage',
String(missingFlagRows.length),
],
[
'Public CLI flags intentionally outside provider-backed integration',
String(excludedFlagRows.reduce((sum, group) => sum + group.keys.length, 0)),
],
['Public CLI flags unclassified by progress script', String(unclassifiedFlagKeys.length)],
];
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
if (coverage) {
summaryRows.push(
['Coverage statements', formatPercent(coverage.statements)],
['Coverage branches', formatPercent(coverage.branches)],
['Coverage functions', formatPercent(coverage.functions)],
['Coverage lines', formatPercent(coverage.lines)],
);
} else {
summaryRows.push(['Coverage summary', 'not available; run pnpm test:coverage first']);
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
return {
coverage,
excludedFlagRows,
flagCoverageRows,
lowCoverageFiles,
missingFlagRows,
missingPublicCommands,
mockHeavyHandlerRows,
providerPressureRows,
publicCommandRows,
summaryRows,
unclassifiedFlagKeys,
};
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
export function buildIntegrationProgressFailures(progress) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
const failures = [];
if (progress.missingPublicCommands.length > 0) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
failures.push(
`missing Provider-backed integration command coverage: ${progress.missingPublicCommands.map((row) => row.command).join(', ')}`,
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
);
}
if (progress.missingFlagRows.length > 0) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
failures.push(
`missing Provider-backed integration workflow flag coverage: ${progress.missingFlagRows.map((row) => row.key).join(', ')}`,
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
);
}
if (progress.unclassifiedFlagKeys.length > 0) {
failures.push(`unclassified public CLI flags: ${progress.unclassifiedFlagKeys.join(', ')}`);
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
return failures;
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
function summarizeProviderScenarioFlagCoverage(files) {
const flagTargets = [
['platform', 'selection across platform-specific provider-backed integration flows'],
['target', 'target-class routing such as tv/mobile/desktop'],
['device', 'human-readable device selection'],
['udid', 'Apple device selection'],
['serial', 'Android device selection'],
['iosSimulatorDeviceSet', 'iOS simulator-set scoping reaches inventory resolution'],
['androidDeviceAllowlist', 'Android serial allowlist reaches inventory resolution'],
['session', 'named session routing'],
feat: add doctor command (#883) * feat: add doctor command * fix: reduce doctor command complexity * fix: classify doctor integration flags * fix: simplify doctor setup * refactor: split doctor checks * fix: simplify doctor check set * fix: include stopped android avds in devices * fix: report doctor device inventory * refactor: reuse device inventory selectors * fix: summarize doctor inventory by platform * fix: show metro cwd in doctor * refactor: simplify metro doctor lookup * fix: update doctor imports after apple consolidation * feat: make doctor Metro probe controllable and surface hidden toolchain failures Two gaps found while verifying the doctor command on a real environment: - Metro host/port were uncontrollable from the CLI: --metro-host/--metro-port were rejected by allowedFlags, and readDoctorOptions only read them from req.runtime (populated by remote/connection profiles, never a plain CLI flag). The Metro check's own hint told users to 'pass the correct --metro-host/--metro-port', which did not exist. Declare the flags and read them from req.flags (runtime kept as fallback) so the probe can target any endpoint, e.g. from outside an RN/Expo project directory. - A broken per-platform toolchain was silently hidden: readDoctorDeviceInventory dropped inventory failures whenever any other platform returned devices, so a broken Xcode or Android SDK still reported a green 'pass'. Keep the failures and surface each as a warn (device-<platform>) when other platforms have devices; scoped --platform runs stay quiet. * fix: align doctor CI expectations * feat: extend doctor preflight checks * fix: keep doctor checks within ci gates * fix: simplify doctor metro surface * refactor: trim doctor bundle impact * fix: restore useful doctor diagnostics * refactor: reuse doctor output helpers * refactor: share device inventory grouping * refactor: keep doctor focused on preflight checks * refactor: simplify doctor toolchain probes * fix: keep scoped simulator hint generic * fix: clarify doctor Xcode selection context * fix: recognize provider scope in remote doctor * fix: address doctor review gaps * fix: keep doctor metro checks inferred
2026-07-01 14:24:32 +02:00
['targetApp', 'doctor target app discovery without opening a session'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['surface', 'macOS app/frontmost/desktop/menubar surfaces'],
['activity', 'Android explicit launch activity'],
['launchConsole', 'iOS simulator launch console capture'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['saveScript', 'open/close replay recording output'],
['relaunch', 'open terminates before launch'],
['shutdown', 'close/disconnect shutdown behavior'],
['appsFilter', 'apps --all vs default filtering'],
['header', 'install-from-source URL headers', ['headers']],
['retainPaths', 'retained install-source materialization'],
['retentionMs', 'install-source materialization TTL'],
['count', 'repeated press/click/swipe input'],
['pointerCount', 'one- vs two-pointer pan gesture topology'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['fps', 'recording frame-rate request'],
['quality', 'recording quality scaling'],
['hideTouches', 'recording without touch overlays'],
['recordingScope', 'recording app vs whole-screen scope', ['scope']],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['intervalMs', 'repeated press interval'],
['delayMs', 'typing/fill delay'],
['recordAs', 'parameterized fill publication for recorded scripts'],
['durationMs', 'scroll, gesture, and TV remote duration'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['holdMs', 'press hold duration'],
['jitterPx', 'press jitter'],
['pixels', 'scroll distance'],
['doubleTap', 'double tap gesture'],
['clickButton', 'desktop mouse button selection', ['button']],
['backMode', 'explicit app/system back behavior', ['mode']],
['pauseMs', 'swipe repeat pause'],
['pattern', 'swipe repeat pattern'],
['snapshotInteractiveOnly', 'interactive snapshot/ref refresh', ['interactiveOnly']],
['snapshotDepth', 'scoped snapshot depth', ['depth']],
['snapshotScope', 'scoped snapshot capture', ['scope']],
['snapshotRaw', 'raw snapshot node output', ['raw']],
['out', 'artifact output path plumbing'],
['overlayRefs', 'screenshot ref overlay annotation'],
['screenshotFullscreen', 'screenshot full-screen capture mode'],
feat: add scale-only screenshot sizing (#1617) * feat: add scale-only screenshot sizing * fix: refuse retired --max-size inputs on every released surface Released sizing inputs must fail closed with migration guidance instead of silently producing native-size artifacts: - contracts: RETIRED_SCREENSHOT_MAX_SIZE declaration + SCREENSHOT_SCALE_LIMITS as the single source for the scale bounds and migration messages - .ad parser: released 'screenshot ... --max-size N' and 'record start ... --max-size N' lines now refuse at parse time (frozen replay-compat witnesses) - daemon: screenshot rejects old-client screenshotMaxSize like recording does; the recording guard now shares the same contract data - Node client: screenshot/record daemon writers refuse the removed { maxSize } option before transport - CLI: --max-size unknown-flag error carries the migration guidance - config/env: stale screenshotMaxSize config keys and the retired AGENT_DEVICE_SCREENSHOT_MAX_SIZE env var are refused for sizing commands (other commands keep working) Quality: numberField now reuses the canonical readOptionalNumber contract helper (AppError bounds instead of plain Error); png-resize inlines one-use wrappers and restores the worker-thread rationale; docs typo fixed. * test: drop retired maxSize entries from the MCP undocumented-input allowlist * fix: refuse retired maxSize at the MCP field-projection seam + release-provenance corpus witnesses - readFieldInput silently dropped undeclared keys before the daemon writers could refuse them, so an MCP call carrying { maxSize } reached transport and returned native-size success. New retiredField() combinator declares the removed key in the field map: the projection seam refuses it with the canonical migration message and the JSON schema no longer advertises it. Real-route MCP executor regressions cover screenshot and record. - replay-compat corpus: derived v0.20.5 witnesses for the released screenshot and record --max-size forms (SHA-256 pinned, new retired-capture-size coverage surface) so check:replay-compat proves the shipped syntax refuses with migration guidance instead of degrading silently. --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
2026-08-06 15:11:00 +02:00
['screenshotScale', 'screenshot proportional scaling post-processing'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['screenshotNoStabilize', 'screenshot stabilization opt-out', ['stabilize']],
['restart', 'logs clear --restart workflow'],
['networkInclude', 'network dump include modes', ['include']],
['noRecord', 'action recording suppression'],
fix(replay): default-exclude observation-only reads from repair heals, add --record opt-in (#1271 stage 2) (#1303) * fix(replay): default-exclude observation-only reads from repair heals, add --record opt-in (#1271 stage 2) Amends ADR 0012 decision 6: snapshot/get/is/a read-only find are excluded from a repair-armed heal by default (session.saveScriptBoundary set), never from ordinary open --save-script authoring recording. wait keeps recording (flow timing, not observation). The corrective-read trap (wave-3 E3: the diverged step was itself a get) means blanket read-exclusion is unsafe, so a new --record flag forces one action through when the correction is itself a read. --record/--no-record are mutually exclusive (INVALID_ARGS if both are set) and are plumbed identically across CLI, the Node client, and MCP. The exclusion lives at the single daemon-side choke point (recordActionEntry/isExcludedRepairSegmentObservation), so an excluded read never grows session.actions.length -- the same counter the existing record-and-heal resume watermark (describeUnperformedRecordAndHeal) already checks, so the empty-segment fail-loud guard falls out for free (message updated to mention --record). Also fixes a latent bug found along the way: the get/is/find/snapshot CLI readers never forwarded --no-record/--record into the built request (only `open` did), so stage 1's "use --no-record" guidance was silently inert via the CLI. * test(integration): cover --record with a provider-backed repair-segment scenario (#1271 stage 2) The progress ratchet (test:integration:progress:check) flagged `record` as an unclassified public CLI flag. Classifying alone would only trade that failure for "missing Provider-backed integration workflow flag coverage" -- and the exclusions bucket is for config/output/transport flags, not behavior flags, so using it would dodge the ratchet rather than satisfy it. Adds a focused provider-backed scenario instead, next to the `--no-record` precedent in android-lifecycle.test.ts. It drives the real request router, session store, replay runtime, and script writer (only the ADB provider is faked), and proves the flag's actual purpose end-to-end: inside a repair-armed `replay --save-script` segment that diverged, the SAME `get text <selector>` runs twice differing only in `--record`; exactly one line lands in the committed healed .ad. Also asserts `--record` + `--no-record` is INVALID_ARGS. Verified the scenario reproduces the bug: with the exclusion neutered it fails on "a diagnostic read inside a repair segment must not be recorded". * fix(replay): key the repair-segment exclusion on provenance, scope --record (#1271 review) Addresses the maintainer review on #1303. P1 — the exclusion dropped PLANNED reads from the heal. It discriminated by command class, but the real discriminator is provenance. Replayed plan steps dispatch through the ordinary request path, so an authored get/is/find step hit the same recordIfSession -> exclusion path as an interactive read and never reached session.actions -- and the heal IS session.actions.slice(boundary). A repaired flow therefore replayed its authored `is visible` assertion and then silently dropped it from its own healed script: the heal quietly stops checking what it used to check, which for a 10x-QA-replay suite is the worst failure mode. Fix: an explicit provenance marker, not a heuristic. `internal.replayPlanStep` is stamped by invokeResolvedReplayAction -- the single point every plan step is dispatched, so it covers annotated and unannotated steps alike. `internal` is daemon-only (toDaemonRequest never copies it off the wire), so authored provenance cannot be spoofed; same channel as replayTargetGuard. The rule now lives once in isInteractiveObservation and both recording call sites consume it, so the mock fixture uses the production classifier instead of mirroring it. Planned observations survive automatically -- users never annotate their own .ad steps. --record is no longer a common flag: removed from COMMON_COMMAND_SUPPORTED_FLAG_KEYS, statically scoped via allowedFlags to snapshot/get/is, and validated dynamically for find (read-only allows; a mutating find click|fill|focus|type is INVALID_ARGS before any device work, sharing one isReadOnlyFindAction predicate with the read-only routing so the two cannot disagree). --no-record stays shared -- it applies to every recordable command. Removed from `open`, which is never observation-only. Rebased onto #1304 and dropped the four hand-rolled reader blocks. Split its helper rather than broadening it: noRecordInputFromFlags (all 13 readers) + observationRecordInputFromFlags (snapshot/get/is/find only). Two named helpers over one `allowRecord` policy arg -- the capability is then the helper's NAME, so a mutating reader physically cannot forward --record, whereas a policy arg would let a future mutating reader opt in by flipping a literal with no schema change. ADR-0012 decision 6 now states the provenance rule, not a command-class rule. The scenario gates the P1: its authored step is a distinguishable `is visible`, and it fails without the provenance check ("the authored 'is visible' step must survive the heal"). * test(daemon): pin that wire-supplied `internal` never reaches a daemon request #1271 stage 2 made `DaemonRequest.internal` semantics-affecting: `internal.replayPlanStep` decides whether an observation-only command is an authored plan step (kept in a repair heal) or an out-of-band diagnostic (excluded). That makes "internal means internally-stamped" worth pinning rather than leaving to convention. The invariant already holds, structurally and twice over: the boundary's `commandRpcParamsSchema` is an allowlist projection emitting only its eight named fields, and `toDaemonRequest` then builds the request field by field. Neither can carry `internal` off the wire. This posts a real JSON-RPC request carrying `internal: { replayPlanStep: true }` through a loopback server and asserts the dispatched request has no `internal`. Verified it fails ("a wire-supplied `internal` must never reach the daemon request") when both allowlists are regressed, so it guards the composite contract instead of restating one layer.
2026-07-16 20:31:05 +02:00
['record', 'repair-segment observation-only recording opt-in (ADR 0012)'],
feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement (#1211) * feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement Step 5 (decision 4, resume): replay --from <n> --plan-digest <sha256> resumes at a 1-based plan step, skipping 1..n-1 without executing them. Every divergence report now carries a real resume object (allowed, from, planDigest, reason?) computed by a preflight that rejects INVALID_ARGS before any action when: the plan digest no longer matches the current script (edits/includes/platform-conditioned expansion), --from is out of range, a skipped step can produce outputEnv values, or the skipped range or resume target is runtime control flow (retry/runFlow.when — these are single plan entries, never individually addressable). `test` rejects --from/--plan-digest both at the CLI-schema layer and at the daemon dispatch layer (the original command name is only visible before test rewrites its nested request to `command: 'replay'`). New modules: src/replay/plan-digest.ts (canonical SHA-256 plan digest) and src/daemon/handlers/session-replay-resume.ts (preflight + the report's resume object), kept out of src/replay/ to avoid a replay<->compat import cycle. Step 6 (decision 1, retirement): --update/-u no longer rewrites .ad files. The ADR mandates a no-op, not an error or flag removal: --update now runs identically to a plain replay and returns the same bounded suggestions every divergence already carries. Removed: healReplayAction's retry-and-rewrite arm and its exclusive helpers (collectReplaySelectorCandidates stays — decision 1's suggestions still use it), the write call from the runtime loop, and the env/${VAR}-interpolation/compat-flow refusal guards that existed only to protect that rewrite. writeReplayScript itself keeps its own round-trip tests but is otherwise unused now; deleted after the production-exports gate flagged it as dead. Docs: cli-help.ts workflow topic + --update/--from flag help, AGENTS.md selector pipeline note, maestro-compat-debt-map.md, website replay-e2e.md and commands.md updated for the retired rewrite and the new resume loop. * fix(ci): classify resume flags + provider-scenario resume coverage The Integration Tests job's architecture-progress gate (test:integration:progress:check) requires every public CLI flag to be classified; --from/--plan-digest (replayFrom/replayPlanDigest) were unclassified. Classify them as device-observable workflow flags and add real provider-backed coverage to the Android lifecycle scenario: a full replay diverges on a missing selector, the report's resume object is asserted (allowed/from/planDigest), and resuming at the next index replays only the tail. Also refresh the stale replayUpdate reason ("selector-healing replay update" -> the retired no-op). * fix: bind replay resume digest to execution plan * test: align replay runtime module topology * fix: clear replay CI regressions * docs: clarify replay repair and resume paths * docs: clarify replay resume step semantics * docs(replay): note that ${VAR} values stay out of the plan digest (ADR 0012 + workflow help) Settled decision from the PR #1211 re-review (maintainer-approved): interpolated ${VAR}/--env/AD_VAR_* VALUES are deliberately NOT part of the resume plan digest. Substitution happens after the digest is computed over the still-unsubstituted ${VAR} text, so re-running the same script with different variable values keeps the same digest and stays resumable — supplying the right values on resume is the caller's responsibility. The digest still binds the script/includes, the effective --platform/--target, and per-action runtime hints + target-v1 identity. Documented in ADR 0012 decision 4 and the `help workflow` resume topic. * docs: clarify replay digest interpolation
2026-07-11 15:06:49 +02:00
['replayUpdate', 'retired --update no-op replays without rewriting (ADR 0012)', ['update']],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['replayEnv', 'replay/test variable injection', ['env']],
feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement (#1211) * feat(replay): ADR 0012 migration steps 5+6 — resume + --update retirement Step 5 (decision 4, resume): replay --from <n> --plan-digest <sha256> resumes at a 1-based plan step, skipping 1..n-1 without executing them. Every divergence report now carries a real resume object (allowed, from, planDigest, reason?) computed by a preflight that rejects INVALID_ARGS before any action when: the plan digest no longer matches the current script (edits/includes/platform-conditioned expansion), --from is out of range, a skipped step can produce outputEnv values, or the skipped range or resume target is runtime control flow (retry/runFlow.when — these are single plan entries, never individually addressable). `test` rejects --from/--plan-digest both at the CLI-schema layer and at the daemon dispatch layer (the original command name is only visible before test rewrites its nested request to `command: 'replay'`). New modules: src/replay/plan-digest.ts (canonical SHA-256 plan digest) and src/daemon/handlers/session-replay-resume.ts (preflight + the report's resume object), kept out of src/replay/ to avoid a replay<->compat import cycle. Step 6 (decision 1, retirement): --update/-u no longer rewrites .ad files. The ADR mandates a no-op, not an error or flag removal: --update now runs identically to a plain replay and returns the same bounded suggestions every divergence already carries. Removed: healReplayAction's retry-and-rewrite arm and its exclusive helpers (collectReplaySelectorCandidates stays — decision 1's suggestions still use it), the write call from the runtime loop, and the env/${VAR}-interpolation/compat-flow refusal guards that existed only to protect that rewrite. writeReplayScript itself keeps its own round-trip tests but is otherwise unused now; deleted after the production-exports gate flagged it as dead. Docs: cli-help.ts workflow topic + --update/--from flag help, AGENTS.md selector pipeline note, maestro-compat-debt-map.md, website replay-e2e.md and commands.md updated for the retired rewrite and the new resume loop. * fix(ci): classify resume flags + provider-scenario resume coverage The Integration Tests job's architecture-progress gate (test:integration:progress:check) requires every public CLI flag to be classified; --from/--plan-digest (replayFrom/replayPlanDigest) were unclassified. Classify them as device-observable workflow flags and add real provider-backed coverage to the Android lifecycle scenario: a full replay diverges on a missing selector, the report's resume object is asserted (allowed/from/planDigest), and resuming at the next index replays only the tail. Also refresh the stale replayUpdate reason ("selector-healing replay update" -> the retired no-op). * fix: bind replay resume digest to execution plan * test: align replay runtime module topology * fix: clear replay CI regressions * docs: clarify replay repair and resume paths * docs: clarify replay resume step semantics * docs(replay): note that ${VAR} values stay out of the plan digest (ADR 0012 + workflow help) Settled decision from the PR #1211 re-review (maintainer-approved): interpolated ${VAR}/--env/AD_VAR_* VALUES are deliberately NOT part of the resume plan digest. Substitution happens after the digest is computed over the still-unsubstituted ${VAR} text, so re-running the same script with different variable values keeps the same digest and stays resumable — supplying the right values on resume is the caller's responsibility. The digest still binds the script/includes, the effective --platform/--target, and per-action runtime hints + target-v1 identity. Documented in ADR 0012 decision 4 and the `help workflow` resume topic. * docs: clarify replay digest interpolation
2026-07-11 15:06:49 +02:00
['replayFrom', 'replay resume skips completed steps (ADR 0012)', ['resumeFrom']],
['replayPlanDigest', 'replay resume plan-digest preflight binding', ['resumePlanDigest']],
['replayKeepSession', 'native replay terminal-close suppression', ['keepSession']],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
['failFast', 'test suite stops after first failure'],
['timeoutMs', 'wait/test timeout flags'],
['retries', 'test suite retry budget flows through request path'],
['artifactsDir', 'test artifact root'],
['steps', 'batch inline steps'],
['batchOnError', 'batch stop-on-error policy', ['onError']],
['batchMaxSteps', 'batch max-step guard', ['maxSteps']],
['findFirst', 'find first disambiguation'],
['findLast', 'find last disambiguation'],
feat: --settle returns the settled diff in the interaction response (#1101) (#1106) * feat: --settle returns the settled diff in the interaction response (#1101) press/click/fill/longpress --settle executes the action, waits for the UI to go quiet (wait stable's loop, shared via stable-capture.ts), and returns the settled diff vs the pre-action tree in the same response — one round trip instead of the interact -> observe pair. - payload: changed lines only (bounded), summary counts, added-line refs, refsGeneration; best-effort (settled:false + hint on never-quiet content, never an action failure); --verify shares the settle captures - ref issuance: the settled tree becomes the session snapshot; a diff-carrying settle response clears snapshotRefsStale and the MCP layer merge-only re-pins added-line refs at the settle generation - grammar: --settle + --settle-quiet <ms> + --timeout <ms> (flag-sourced descriptor budget with new envelope:'widen' semantics mirroring wait) - ADR 0011: new settleObservation guarantee classified on every path with contract scenarios per enforced/delegated cell * test: give the two contention-flaky doctor scenarios explicit budgets The doctor provider scenarios sit at ~5s of real daemon-harness work on a loaded host and flake at vitest's 5s default during full-suite runs (the known contention flake AGENTS.md documents). Same in-file precedent as the Metro-probe scenario's 10s budget. * fix: move SettleParams to contracts to satisfy the layering DAG daemon/handlers/interaction-flags.ts imported the type across the daemon -> commands boundary (R2 commands-floor). The tuning params are part of the interaction contract like SettleObservation, so they live in contracts/interaction.ts and both layers import from there. * feat: keep settle diffs content-first — drop Key nodes, added lines win the cap Bluesky dogfood: a fill that summons the iOS keyboard spent 49 of the 80 capped diff lines spelling out QWERTY keys, and a screen transition with 269 removals could starve out the added lines entirely. Key-type nodes are now filtered from both diff sides (the [keyboard] container line still signals presence), and under truncation added lines — the ones carrying fresh refs — win slots over removals. * docs: state the core loop in the top-level help starting point Benchmarked with headless haiku/sonnet agents given only --help: both models skipped the help-workflow pointer and started with plain snapshot (38KB payloads they then had to re-read from files). One core-loop line at the starting point is what teaches snapshot -i and --settle to models that never read a second help page. * fix: preserve settle digest refs for mcp * fix: reduce settle fallow complexity * fix: surface settle output in CLI text * fix: complete settle handling for longpress * refactor: localize daemon timeout envelopes * refactor: deepen post-action observation * refactor: centralize post-action observation planning * refactor: derive settle capability from descriptors * refactor: trim settle descriptor helpers
2026-07-06 20:18:44 +02:00
['verify', 'descriptor post-action evidence capture'],
['settle', 'descriptor post-action settled-diff observation'],
['settleQuietMs', 'settle quiet-window tuning'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
];
const sources = files.map((file) => fs.readFileSync(file, 'utf8')).join('\n');
return flagTargets.map(([key, reason, aliases = []]) => {
const references = [key, ...aliases].reduce(
(count, candidate) => count + countFlagReferences(sources, candidate),
0,
);
return { key, reason, references };
});
}
function countFlagReferences(text, key) {
const escaped = key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
return text.match(new RegExp(`\\b${escaped}\\s*:`, 'g'))?.length ?? 0;
}
function summarizeProviderScenarioFlagExclusions() {
return [
{
name: 'config, output, diagnostics, and transport',
owner: 'args/CLI transport/auth tests',
keys: [
'config',
'remoteConfig',
'stateDir',
'daemonBaseUrl',
'daemonAuthToken',
'daemonTransport',
'daemonServerMode',
feat: add doctor command (#883) * feat: add doctor command * fix: reduce doctor command complexity * fix: classify doctor integration flags * fix: simplify doctor setup * refactor: split doctor checks * fix: simplify doctor check set * fix: include stopped android avds in devices * fix: report doctor device inventory * refactor: reuse device inventory selectors * fix: summarize doctor inventory by platform * fix: show metro cwd in doctor * refactor: simplify metro doctor lookup * fix: update doctor imports after apple consolidation * feat: make doctor Metro probe controllable and surface hidden toolchain failures Two gaps found while verifying the doctor command on a real environment: - Metro host/port were uncontrollable from the CLI: --metro-host/--metro-port were rejected by allowedFlags, and readDoctorOptions only read them from req.runtime (populated by remote/connection profiles, never a plain CLI flag). The Metro check's own hint told users to 'pass the correct --metro-host/--metro-port', which did not exist. Declare the flags and read them from req.flags (runtime kept as fallback) so the probe can target any endpoint, e.g. from outside an RN/Expo project directory. - A broken per-platform toolchain was silently hidden: readDoctorDeviceInventory dropped inventory failures whenever any other platform returned devices, so a broken Xcode or Android SDK still reported a green 'pass'. Keep the failures and surface each as a warn (device-<platform>) when other platforms have devices; scoped --platform runs stay quiet. * fix: align doctor CI expectations * feat: extend doctor preflight checks * fix: keep doctor checks within ci gates * fix: simplify doctor metro surface * refactor: trim doctor bundle impact * fix: restore useful doctor diagnostics * refactor: reuse doctor output helpers * refactor: share device inventory grouping * refactor: keep doctor focused on preflight checks * refactor: simplify doctor toolchain probes * fix: keep scoped simulator hint generic * fix: clarify doctor Xcode selection context * fix: recognize provider scope in remote doctor * fix: address doctor review gaps * fix: keep doctor metro checks inferred
2026-07-01 14:24:32 +02:00
'remote',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
'tenant',
'sessionIsolation',
'runId',
'leaseId',
'leaseBackend',
'json',
'help',
'version',
'verbose',
feat: opt-in agent-cost wallClockMs behind --cost — Phase 4 slice 1 (#922) * feat: opt-in agent-cost wallClockMs behind --cost Add per-command wall-clock latency as a purely additive, opt-in response field (cost.wallClockMs) gated behind a new global --cost flag. The flag plumbs end to end mirroring --debug: cli-flags definition + GLOBAL_FLAG_KEYS, AgentDeviceClientConfig/overrides, buildClientConfig, buildMeta (meta.includeCost), the DaemonRequestMeta contract, and the boundary parse in daemonCommandRequestSchema so it survives the HTTP edge. The graft lives in request-router handleRequest (the seam that owns the outer wall-clock incl. lock + execute + finalize). It mirrors the conditional registerDownloadableArtifacts spread: when --cost is off OR the response is an error, the response is returned untouched. Only on an opted-in successful response is cost appended, so the default serialized DaemonResponse is byte-identical to today (Maestro .ad recompare safe). Proven by the parity test (flag-off identity, flag-on additive-only, error path, boundary survival). Additive / semver-minor. MCP exposure and richer signals (roundTrips, nodeCount) are deferred to follow-up slices. * test: classify --cost flag as outside provider-backed integration The integration progress guard (test:integration:progress:check) treats every public CLI flag as either device-observable (requiring provider-backed coverage) or intentionally excluded. --cost is a diagnostics/output flag (a purely additive response field, not device-observable), so it joins json/help/version/verbose in the 'config, output, diagnostics, and transport' exclusion bucket.
2026-06-29 13:50:12 +02:00
'cost',
feat: leveled response views + --level knob with snapshot digest — Phase 4 (#942) * feat: leveled response views + --level knob, with a snapshot digest — Phase 4 Add the agent-cost leveled-response system: a responseLevel knob (digest | default | full) plumbed end to end behind a global --level flag (mirroring --cost), and a per-command ResponseView registry applied in the router on the success path. - contracts: RESPONSE_LEVELS/ResponseLevel + meta.responseLevel + boundary schema whitelist. Plumbing mirrors --cost: cli-flags FlagDefinition + GLOBAL_FLAG_KEYS, AgentDeviceClientConfig + overrides, buildClientConfig, buildMeta. ResponseLevel exported from the public root. - src/daemon/response-views.ts: the ResponseView registry. Seeds the snapshot digest — the full node tree (the dominant token sink) collapses to { nodeCount, refs: first 12 hittable/non-occluded refs with labels } plus the cheap top-level signals (truncated/visibility/snapshotQuality). full returns today's shape (nothing richer is computed yet). - router graft (applyResponseLevelView + applyAgentCostGrafts): composes with the existing cost block. With responseLevel default (or unset) AND no registered view AND no --cost, the original response is returned UNCHANGED — byte-identical to today (Maestro .ad recompare safe). cost.nodeCount reads the original node tree so it stays accurate even after a digest. Tests: snapshot view unit test (digest filters hittable/occluded, drops the tree, keeps cheap signals; default/full passthrough); router graft test via an injected view (default identity byte-identical, digest applies, full passthrough, digest+cost composition, unregistered-command passthrough, boundary parse). Verified: tsc, oxfmt + oxlint --deny-warnings, fallow audit clean, rslib build, Layering Guard empty, 1106 daemon/contracts/client tests pass (incl. the existing cost/typed-error grafts after the restructure). * fix: repoint MCP output-schemas import to kernel/device (rebase fixup) The kernel move (#940) deleted src/utils/device.ts; #941's command-output-schemas.ts (merged after #940's codemod ran) still imported the old path. Same one-line fix as #943; de-dups once that lands. * fix: re-classify responseLevel flag in integration-progress model The --level/responseLevel flag is a diagnostics/output flag (not device- observable), classified in the exclusion bucket alongside --cost. (Lost in an earlier rebase; re-applying.)
2026-06-30 07:53:28 +02:00
'responseLevel',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
],
},
{
name: 'remote connection and session-lock policy',
owner: 'connection/runtime/request policy tests',
chore: remove verified dead code and migration scaffolding (~700 LOC) (#1367) * chore: remove verified dead code and migration scaffolding Multi-agent audit of accumulated waste, every finding adversarially verified against call sites, git history, and the published surface before removal. Net -710 lines. - delete src/core/platform-descriptor/ (superseded ADR-0009 migration scaffold; parity tests now assert an inline table) - remove test-only seams: registry introspection exports, CommandFacet.extraDaemonWriters, MaestroEngineOptions.timing - remove dead flexibility: backend capability allow-list, screenshot-diff maxRegions, CloudWebDriverSupportLevel 'partial', clearFirst on the TS+Swift runner wire contract - remove dead deprecated surface: --session-locked / --session-lock-conflicts aliases (hard migration error now points at --session-lock), replay export --format single-value enum, unused Lease*Payload contract types, runtime-layer rotate duplicate - collapse pass-throughs/duplication: withRetry adapter, default-cloud-artifact-provider, connect-profile client-id hashing (3x sha256 impls -> one helper, byte-identical output), shared scripts walker, cloneValue -> structuredClone, fill-diagnostics moved into android/ BREAKING CHANGE: --session-locked and --session-lock-conflicts now fail with a migration error pointing at --session-lock; replay export --format is removed (Maestro was the only value); Lease*Payload types are dropped from the ./contracts subpath. * chore: satisfy fallow gates tightened by #1363/#1364 after rebase - drop the consumer-less AndroidFillVerificationNode re-export - reuse requireSnapshotSession in resolveSnapshotForRef instead of inlining the same authorized-frame resolution (fallow clone group); the helper's return type now guarantees the session it already throws for * chore: address review — keep cloud-webdriver partial capability metadata The partial/supported/unsupported levels and their notes are part of the lease-response capability contract for genuinely limited operations (Appium page-source snapshots, upload-then-install), not dead scaffolding. Restore them and the asserting tests unchanged from main. Also add the missing CHANGELOG entry for the Lease*Payload type removal from agent-device/contracts.
2026-07-23 07:36:52 +02:00
keys: ['force', 'noLogin', 'sessionLock'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
},
{
name: 'cloud artifact provider lookup',
owner:
'cloud provider profile, artifact provider, CLI output, and cloud WebDriver provider scenario tests',
keys: [
'provider',
'providerSessionId',
'providerApp',
'providerOsVersion',
'providerProject',
'providerBuild',
'providerSessionName',
feat(cloud-webdriver): BrowserStack device-feature capabilities, and fix cloud orientation (#1544) * feat(cloud-webdriver): support BrowserStack device-feature capabilities Adds the eight BrowserStack "device feature" session capabilities that had no representation in agent-device: deviceOrientation, geoLocation, timezone, language, locale, networkProfile, customNetwork, and resignApp. These are vendor capabilities, so they are emitted inside `bstack:options` rather than at the top level. BrowserStack's YAML config lists them unnested and its SDK relocates them; agent-device talks to the hub directly, so it nests them itself. A single spec table drives both the flag reader and the capability builder, so adding a capability is a table row rather than a branch in each. A structural test asserts every field owns exactly one row, since a field the table forgets would parse off the CLI, ride the profile, and then be silently dropped before the hub ever saw it. Rejects combinations the provider cannot act on unambiguously: an unknown orientation is caught at the flag boundary instead of being forwarded to a hub that accepts and then ignores it, --provider-no-resign-app is refused on Android, and a named network profile cannot be combined with a custom network shape. Also fixes a latent shallow-merge bug in buildBrowserStackCapabilities: a caller supplying its own `bstack:options` replaced the whole object and silently dropped the project, build, and session labels. It is now merged per key. * fix(cloud-webdriver): rotate via WebDriver orientation endpoints `setOrientation` on the cloud WebDriver path sent `mobile: rotate`, which is not a driver command at all. UiAutomator2's own error enumerates its extensions and `rotate` is absent from the list, so `agent-device orientation` was a hard failure on every hosted provider. It also forwarded agent-device's four-way rotation vocabulary verbatim ("landscape-left", "portrait-upside-down"), where the protocol accepts only uppercase PORTRAIT/LANDSCAPE. Every other platform has a translation layer; this path was the only one without one. Now two transports, ordered by backend. `POST /rotation` takes exact four-way degrees and leads on Android, since it is the only endpoint that can express upside-down and left-versus-right. `POST /orientation` is two-way and leads on XCUITest, which rejects `/rotation`. Each falls back to the other, because only BrowserStack's UiAutomator2 is verified and a provider whose driver disagrees should degrade rather than hard-fail. Verified live against BrowserStack App Automate: POST /rotation {"x":0,"y":0,"z":0} -> 200 {"value":"ROTATION_0"} The rotation-to-surface-index mapping moves to contracts/device-rotation.ts and the existing adb path now reads from it, so the local and hosted mappings cannot drift apart. Note this rotates the current display, not persistent device rotation, so an activity that does not pin its own orientation may still need rotating once it is in the foreground. The capability was declared "partial" without the transport existing, and no test covered setOrientation on the cloud path; only adb and the Apple runner were covered. Both gaps are now closed. * fix(cloud-webdriver): narrow orientation fallback and gate provider-owned flags Addresses review on #1544. The orientation fallback caught every error, so a timeout, an auth rejection, a dead session or a provider 5xx on the first transport was swallowed and retried against the second. When that one also failed the caller got "rejected both endpoints" with the real cause discarded. Fallback is now keyed on structured unsupported-endpoint signals only — HTTP 404/405, or a W3C `unknown command` / `unknown method` code — matching the repo rule of keying on typed details rather than message text. Everything else rethrows unchanged. Device-feature capabilities are BrowserStack-owned, but the flags were accepted by any cloud provider, persisted into the generated profile, and then silently dropped at session creation. `connect aws-device-farm` now rejects them with a typed error naming each offending flag, raised before the provider's own required-argument checks so the caller is told what is unsupported rather than what else is missing. Ownership is modelled on the capability spec table, so a new capability inherits the guard without a second list to maintain. Adds provider-backed orientation scenarios driven through public daemon dispatch against the fake WebDriver provider: the four-way endpoint on the happy path, the documented collapse onto the two-way endpoint when the driver does not implement `/rotation`, and a provider 5xx that must surface without consulting the second transport. The fake server's route handling became a table in the process — it had grown to ten branches in one function. * fix(cloud-webdriver): read W3C error codes before status, enforce ownership at the runtime boundary Addresses the second review pass on #1544. The fallback classifier returned on any 404/405 before consulting the W3C error code, so an HTTP 404 carrying `invalid session id` was masked as a missing route and retried against the second transport. The structured code now takes precedence whenever the driver sent one; bare status is consulted only when no code exists. Two cases pin it: a 404 `invalid session id` and a 405 `timeout` must both surface rather than fall through. Provider ownership was enforced only in the CLI profile builder, which the typed client and hand-authored remote-config profiles bypass entirely — both reach session preparation without passing through `connect`, so the capabilities were accepted and then dropped. The check now lives on the capability-ownership module and runs inside AWS Device Farm's `prepareSession`, with the CLI builder calling the same helper instead of its own copy. Covered by a scenario that drives the runtime boundary directly and asserts the rejection happens before any provider session is created.
2026-08-02 11:47:49 +05:30
'providerDeviceOrientation',
'providerGeoLocation',
'providerTimezone',
'providerLanguage',
'providerLocale',
'providerNetworkProfile',
'providerCustomNetwork',
'providerNoResignApp',
'awsProjectArn',
'awsDeviceArn',
'awsAppArn',
'awsRegion',
'awsInteractionMode',
],
},
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
{
name: 'Metro and React Native runtime preparation',
owner: 'Metro companion integration and parser tests',
keys: [
'metroHost',
'metroPort',
'metroProjectRoot',
'metroKind',
'metroPublicBaseUrl',
'metroProxyBaseUrl',
'metroBearerToken',
'metroPreparePort',
'metroListenHost',
'metroStatusHost',
'metroStartupTimeoutMs',
'metroProbeTimeoutMs',
'metroRuntimeFile',
'metroNoReuseExisting',
'metroNoInstallDeps',
'bundleUrl',
'launchUrl',
],
},
{
name: 'Apple launch and perf artifact options',
owner: 'iOS platform, observability command, and parser tests',
keys: [
'deviceHub',
'kind',
'launchArgs',
'perfTemplate',
'iosXctestrunFile',
'iosXctestDerivedDataPath',
'iosXctestEnvDir',
],
},
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
{
name: 'parser/client-only command flags',
owner: 'args, CLI, debug-symbols, screenshot-diff, and batch tests',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
keys: [
'artifact',
'dsym',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
'githubActionsArtifact',
'snapshotDiff',
'snapshotForceFull',
'baseline',
'threshold',
'reporter',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
'reportJunit',
'replayMaestro',
'recordVideo',
'shardAll',
'shardSplit',
'searchPath',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
'stepsFile',
'proxyHost',
'proxyPort',
'stale',
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
],
},
{
name: 'daemon lifecycle control',
owner: 'daemon CLI lifecycle tests',
keys: ['clean'],
},
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
{
name: 'platform boot fallback without provider seam',
owner: 'handler and Android platform unit tests',
feat: add Android test IME helper for deterministic text entry (#1198) (#1201) * feat: add Android test IME helper for deterministic text entry (#1198) Ships a headless InputMethodService (android-ime-helper) as a third Android helper APK, replacing the visible system keyboard during automated sessions. Renders zero accessibility nodes and accepts Unicode/CJK/emoji text over a base64-encoded broadcast channel, fixing both the settle-diff IME-chrome flood and the ASCII-only adb-shell text entry limit in one structural fix. - android-ime-helper/: InputMethodService + build/package scripts on the existing helper-APK toolchain (javac+d8+aapt2+zipalign+apksigner). - src/platforms/android/ime-helper.ts, ime-lifecycle.ts: install/version lifecycle (shared with the other two helpers via the new helper-package-install.ts), activation on session open, and on-device restore-hygiene (previous IME persisted to a device settings key so any daemon/state-dir can recover it; restored on close, daemon teardown, and daemon startup for orphans left by a crashed run). - input-actions.ts: fill/type route through the helper's broadcast channel when active, unicode-safe; unchanged ASCII-shell fallback otherwise. - doctor: new android-test-ime check flags a stuck helper IME with a copy-pasteable `adb shell ime set` remediation command. - Gating: default-on for emulators, opt-in via `open --test-ime` on real devices. - Dead-weight: rewrote the manual ADBKeyBoard workaround doc, dropped the now-provably-live skillgym non-ASCII eval case, updated the ASCII fallback's error message to point at the helper instead of dead-ending. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): permission-gate the IME receiver, fix CI, add opt-out Addresses the independent review's blockers and should-fixes. SECURITY (blocker 1): the text-injection receiver was RECEIVER_EXPORTED with no gate — any co-installed app could inject text into the focused field while the test IME was active. Fixed by requiring the WRITE_SECURE_SETTINGS sender permission on the (in-process, dynamically-registered) receiver: adb shell holds it, third-party apps cannot. The reviewer's suggested exported=false + explicit-component approach was tried first but empirically breaks delivery on API 36 (adb shell cannot reach a non-exported receiver there) — documented in the helper README. Live-verified: a purpose-built rogue APK's broadcasts (implicit and package-scoped, no permission) are silently dropped, field unchanged; adb shell's bare broadcast still injects. Added ime-helper-security.test.ts asserting the permission gate and that no permissionless exported registration returns. CI (blocker 2): (a) added `testIme` to integration-progress-model flag buckets (Integration Tests was red on the unclassified flag). (b) mocked resolveAndroidImeHelperArtifact in session-doctor-android / ime-lifecycle / input-actions-test-ime tests so they no longer depend on android-ime-helper/dist existing on disk (Coverage was red on a fresh checkout); verified by running them with dist removed. Should-fixes: added `--no-test-ime` to opt out on emulators (tri-state gating, parser-tested); PR body's "byte-identical" claim corrected to size/CRC-match. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * docs(#1201): pin the API-36 exported-receiver constraint in a comment The RECEIVER_EXPORTED flag cannot express why it must stay exported. Add a one-line note so a future hardening pass doesn't switch to RECEIVER_NOT_EXPORTED and silently break the CLI (adb shell can't deliver explicit broadcasts to non-exported components on API 36+; WRITE_SECURE_SETTINGS is the actual gate). 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): harden IME restore lifecycle (blockers 1 & 2) Blocker 1 — a failed restore no longer deletes the recovery value. restore now reads back default_input_method after `ime set` and only clears the persisted previous-IME record on a confirmed-successful restore; a failed set keeps the value so a later retry / startup recovery / doctor remediation can still un-strand the user off the helper IME. Blocker 2 — startup orphan-recovery no longer overwrites/races user state. It only restores when the device's CURRENT default IME is still our helper (so a user who legitimately switched away is left alone), and skips any device a live session in this process owns (the fire-and-forget startup vs. concurrent `open` race — activate now marks the device active BEFORE the `ime set`, so any recovery pass that could observe the helper active also observes the flag and skips). Never persists the helper itself as the previous IME. activate also verifies its own switch via read-back. Exported ANDROID_IME_HELPER_SERVICE_COMPONENT so restore compares the active IME without reading the packaged artifact from disk. Tests: failed-restore keeps the value (+ later recovery succeeds), startup no-op when current != helper, startup skips a live-owned device. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * chore(#1201): delete unused ACTION_ENTER path, baseline test-only export seams Rebased onto main (#1202 production-unused-exports gate). Two follow-ups: - Deleted the unused ACTION_ENTER broadcast end-to-end (TS sendAndroidImeHelperEnter + its test, Java handler, README): nothing routes through it — `keyboard enter` uses the keyevent ENTER path — so the new production-exports gate flagged it as dead production code. Removed rather than grandfathered. - Added the three legitimate test-only seams (resetAndroidImeHelperInstallCache, resetAndroidTestImeActivationCacheForTests, setAndroidTestImeActiveForTests) to fallow-baselines/production-unused-exports.json, matching how the sibling helper reset functions (resetAndroidMultiTouchHelperInstallCache, ...) are already grandfathered there. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201): stop daemon-startup adb spawn on non-Android hosts (macOS Smoke) Root cause of the red macOS Smoke shard (proven, not hand-waved): the fire-and-forget restoreOrphanedAndroidTestImeOnDaemonStartup ran `adb devices` at EVERY daemon startup, on every platform. GitHub macOS runners ship the Android SDK, so this cold-started the adb server mid-replay and destabilized the macOS System Settings replay timing — the failed job's cleanup shows "Terminate orphan process: pid (N) (adb)"; main's green runs spawn no adb. Fix: gate the startup orphan scan behind a host-side marker written in the daemon state dir when a session activates the test IME (mirrors the managed-web-browser orphan-cleanup `installed` gate). A host that never uses the Android test IME — the macOS CI runner included — never writes the marker and so never spawns adb at startup. The marker is cleared once nothing is left stuck. Adds SessionStore.resolveStateDir(); tests: startup recovery does not scan adb when no marker exists (+ marker cleared after a clean scan). 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * chore(#1201): suppress fallow class-member false-positive on state-dir accessor CI's Fallow audit flags SessionStore.resolveDaemonStateDir as an unused class member, but it is called via sessionStore.resolveDaemonStateDir() in session-open.ts — fallow's class-member tracer just doesn't resolve a method call sited inside a call argument. Renamed for clarity (avoids the collision with config.ts's free resolveStateDir) and added the localized fallow-ignore-next-line unused-class-member suppression. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> * fix(#1201 review): durable persist before switch + device-scoped recovery markers Addresses devin-ai-integration's two P1 restore-safety blockers on 19cbce79d. P1.1 — durably persist the restore target BEFORE the global IME switch. writePersistedPreviousIme now checks the `settings put` exit code AND reads the value back, returning a boolean. activate persists first and, if it cannot be persisted, fails open to the existing input path WITHOUT switching — a rejected `settings put` can no longer strand the user on the helper with no restore target. Regression test added. P1.2 — close the marker crash/offline blind spot. Recovery intent is now recorded per device, BEFORE the switch (ordering: durable record -> marker -> ime set), eliminating the post-switch/pre-marker crash window. Markers are device-scoped and each is retained until that device is actually observed clean: an offline/disconnected-but-stuck device keeps its marker and is recovered on reconnect instead of being cleared because the current `adb devices` scan saw no set-failed. Close-time restore clears only that device's marker (stateDir plumbed through teardown/close). Tests cover the persist-failure, post-switch/pre-marker crash, offline-then-reconnect, live-session-owned, and user-switched-away cases. 🤖 Generated with Claude Code Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
2026-07-10 19:38:37 +02:00
keys: ['headless', 'testIme'],
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
},
{
name: 'open foreground auto-resolution (RFC prototype)',
owner: 'daemon session-open-foreground handler unit tests',
keys: ['foreground'],
},
{
name: 'Apple simulator screenshot rendering options',
owner: 'iOS platform and screenshot-diff runtime tests',
keys: ['screenshotNormalizeStatusBar', 'screenshotPixelDensity'],
},
feat(ios): expose AX custom actions on merged accessibility elements (#1665) * feat(ios): expose AX custom actions on merged accessibility elements Apps that merge a card into one accessibility element for VoiceOver (React Native's `accessible` prop) publish the card's real affordances as UIAccessibilityCustomActions rather than as child elements. Our snapshot showed only the merged node, so an agent looking for a feed card's options control had nothing to aim at and fell back to coordinate guessing. `snapshot --actions` now names them: @e8 [link] "feedItem-by-whiskers.test" actions: ["Reply", "Repost", "Open post options menu"] Opt-in, because the AX server cannot serve custom actions in a bulk tree request: adding the attribute makes testmanagerd's reply decoder reject the nested arrays a custom action serializes into, drop the reply, and time the request out (~65s vs ~110ms). Only per-element reads answer, at ~100ms each, so the runner reads at most 12 labelled childless nodes and stops at the capture-plan deadline. The request pins the private-AX backend, since no other backend can read the attribute, and reports that as its own `requested-backend` verdict so a deliberate pin never renders as a degradation warning. Invocation is not shipped: the actions are readable but not invocable from the runner. RunnerAXSnapshotBridge.h records the five APIs that were tried. * fix(ios): disclose a capped custom-action pass, and read on-screen elements first Two gaps in the first cut. An element the bounded pass never reached rendered identically to one with no custom actions, so a capped capture silently taught the reader that later feed cards have no affordances — the exact mis-inference this feature exists to prevent. The runner already counted reads against candidates; it now carries both to the response, and the verdict renders one response-level line when the pass was incomplete. A complete pass stays silent, and "never asked" stays distinguishable from "read none" (the key is absent, not (0, 0)). The obvious remedy for a capped pass would be a scoped re-run, but scope is applied when the Swift walk builds nodes, long after the read pass, so it does not redirect the budget at all — measured: reads=12 candidates=18 with and without --scope. Rather than print a remedy that does nothing, the read pass now orders candidates on-screen first. That makes the budget land on elements an agent can act on, and makes the disclosed remedy true: scrolling changes the on-screen set, so a re-run reads elements the previous pass could not. Also states plainly, in the flag help and the tool/SDK field description, that the names are for planning: nothing invokes them, so the affordance is reached through the element's detail screen, the same control exposed elsewhere, or coordinates. * test(snapshot): pin the custom-action coverage pair in the verdict shape assertion * chore(scripts): classify the --actions flag in the integration progress model The completeness gate flagged snapshotCustomActions as unclassified, which is what it is for. It gets its own bucket rather than joining the provider-scenario table: the values come from the private AX client inside the runner process, and the fake runner derives its behavior from fixture tables that cannot fabricate custom actions, so there is no provider-backed scenario to claim. The owning coverage is named instead — runner XCTest unit, snapshot-lines, snapshot-quality. * fix(ios): fail closed on unserviceable --actions, bound each read, cap output, and count actions in identity Four review findings. 1. `--actions` with `--raw`, or on any target that is not an iOS simulator, used to succeed and return nodes with no actions — a requested capability silently no-opped, indistinguishable from "this screen has none". Both now fail closed. The raw pairing is rejected at the shared request seam (INVALID_ARGS) so CLI, Node client and MCP answer alike before any device work; the platform case is rejected once the session device is resolved (UNSUPPORTED_OPERATION), naming the resolved target. `diff --actions` was already rejected as an unsupported flag. 2. The per-element AX read had no timeout, so one wedged element could consume the whole capture budget. Each read now runs off-thread behind a 1s wait. A timed-out element counts as unread, never as "read, and it has no actions", so the existing partial-pass disclosure already covers it. 3. The element budget bounded element count only; one element could still return an unbounded list of unbounded names. Capped at 8 names of 80 characters, and clipped elements are counted into the coverage so a truncated list is disclosed rather than silently presented as complete. 4. Action names were rendered unescaped, and no comparison key read them. Names now get the same escaping as text previews plus control-character folding, so an app-authored name cannot split or corrupt a line. `actions` joins the diff comparable key, the unchanged-comparison projection, and — the sharper bug — the snapshot presentation key, without which `snapshot` followed by `snapshot --actions` on a still screen answered "unchanged" and never delivered the actions that were explicitly requested. * fix(ios): contain a hung custom-action read instead of accumulating orphans The 1s read deadline frees the caller, but the underlying AX call is a synchronous XPC round trip that cannot be cancelled — it keeps running. On a global concurrent queue that meant repeated `snapshot --actions` against a wedged element piled up orphaned reads, all using the shared XCAXClient concurrently. The deadline was containment for the capture, not for the runner. Since the call cannot be cancelled, contain it instead: - every read runs on one dedicated serial queue, so a wedged call can never be joined by a second concurrent user of the shared client; - a single-flight guard refuses to dispatch at all while an abandoned read is still outstanding, so a repeated capture adds no work — the dispatch counter stands still; - the read pass stops at that point rather than paying a deadline per element on reads that would all be refused, and reports `blocked` so the capture stays honest. That gets its own line, because the partial-pass remedy (scroll and re-run) cannot clear a hang and would send the reader in circles. Recovery needs no reset: when the hung call finally returns, in-flight drops to zero and reads resume. The regression drives a fake AX client that never returns, and asserts the three things the fix exists for — exactly one in-flight read with no further dispatches across repeated captures, immediate returns with the skip disclosed instead of the scroll remedy, and reads working again once the wedge clears. * ci(ios): execute the custom-action runner regressions instead of only compiling them The iOS workflow runs a targeted -only-testing list, so a runner test that is not named there is compiled by the build step and then never executed. All seven custom-action tests were in that gap — including the containment regression, which is the only executable proof that a hung AX read cannot accumulate orphaned in-flight reads. Red/green against the containment regression, with the fix reverted to its pre-fix concurrency behavior (global concurrent queue, no single-flight guard, no blocked exit): RED in-flight 6 (want 1), dispatches 6 (want 1), each repeat paid the full 1.004s deadline (want <0.2s), blocked=false (want true), and the in-flight drain never completed — "Exceeded timeout of 5 seconds". GREEN 7/7 pass, containment regression in 1.02s.
2026-08-07 17:35:48 +02:00
{
// Reading accessibility custom actions has no provider-scenario surface:
// the values come from the private AX client inside the runner process,
// and the fake runner derives its behavior from fixture tables that
// cannot fabricate them. Covered instead by the runner's XCTest unit
// bundle (option→backend-pin projection, node carry, coverage counting
// and disclosure) plus TS presentation and quality-verdict tests.
name: 'Apple simulator private-AX capture options',
owner: 'runner XCTest unit, snapshot-lines, and snapshot-quality tests',
keys: ['snapshotCustomActions'],
},
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
];
}
function readPublicCliFlagKeys() {
return new Set(
getFlagDefinitions()
.filter((definition) => definition.names.some((name) => name.startsWith('-')))
.map((definition) => definition.key),
);
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
function isCommandContractSource(file) {
return (
file.endsWith('.ts') &&
!file.endsWith('.test.ts') &&
!file.includes(`${path.sep}__tests__${path.sep}`)
);
}
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
function summarizeFiles(files) {
let lines = 0;
let tests = 0;
for (const file of files) {
const text = fs.readFileSync(file, 'utf8');
lines += text.split('\n').length;
tests += countTestDeclarations(text);
}
return { files: files.length, lines, tests };
}
function summarizeMockHeavyHandlerFiles(root, files) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
return files
.map((file) => {
const text = fs.readFileSync(file, 'utf8');
return {
file: path.relative(root, file),
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
lines: text.split('\n').length,
tests: countTestDeclarations(text),
};
})
.sort((a, b) => b.lines - a.lines)
.slice(0, 12);
}
function summarizeProviderPressure(files) {
const surfaces = [
{
name: 'Android ADB provider',
pattern:
/\bAndroidAdbProvider\b|\bandroidAdbProvider\b|\badbProvider\b|\badb\.(?:exec|installer|puller|portReverse)\b/g,
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
},
{
name: 'Apple runner provider',
pattern: /\bAppleRunnerProvider\b|\bappleRunnerProvider\b|\b(?:ios|macos|tvos)\.runner\b/g,
},
{
name: 'Apple simctl/devicectl provider',
pattern: /\bsimctl\b|\bdevicectl\b|\brunXcrun\b|\bsimctl\s*:|\bdevicectl\s*:/g,
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
},
{
name: 'Apple macOS helper provider',
pattern: /\bmacos-helper\b|\bagent-device-macos-helper\b|\bmacosHelper\s*:/g,
},
{
name: 'Apple macOS host provider',
pattern:
/\bmacos-host\b|\bmacosHost\s*:|\bAppleMacOsHostProvider\b|\bopenBundle\b|\bopenTarget\b|\breadClipboard\b|\bwriteClipboard\b|\breadDarkMode\b|\bsetDarkMode\b|\blistApps\b/g,
},
{
name: 'Apple generic host-tool provider',
pattern: buildAppleGenericHostToolPattern(),
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
},
{
name: 'Linux semantic desktop provider',
pattern: /\bdesktop\b|\bopenTarget\b|\bcloseApp\b/g,
},
{
name: 'Linux semantic accessibility/clipboard/screenshot provider',
pattern:
/\baccessibility\b|\bcaptureTree\b|\bclipboard\b|\breadText\b|\bwriteText\b|\bscreenshot\b|\bcapture\s*:/g,
},
{
name: 'Linux semantic input provider',
pattern: /\bLinuxInputProvider\b|\bprovider\.input\b|\binput\s*:|\['input'/g,
},
{
name: 'Linux generic tool provider',
pattern:
/\bLinuxToolProvider\b|\blinuxToolProvider\b|\brunCommand\b|\bwhichCommand\b|\bxdotool\b|\bydotool\b|\bxclip\b|\bscrot\b|\bgrim\b|\bwmctrl\b|\bpkill\b/g,
},
{
name: 'Web semantic provider',
pattern:
/\bWebProvider\b|\bwebProvider\b|\bwithWebProvider\b|\bresolveWebProvider\b|\['web'/g,
},
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
{
name: 'Recording provider',
pattern: /\bRecordingProvider\b|\brecordingProvider\b|\bstartRecording\b/g,
},
];
return surfaces
.map((surface) => ({ name: surface.name, ...countSurfaceReferences(files, surface.pattern) }))
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
.filter((surface) => surface.references > 0);
}
function buildAppleGenericHostToolPattern(): RegExp {
const hostTools = [
'xcrun',
'open',
'pbcopy',
'pbpaste',
'plutil',
'osascript',
'swift',
'codesign',
'mdfind',
'ps',
'pkill',
].join('|');
return new RegExp(
[
String.raw`\brunAppleToolCommand\b`,
String.raw`\brunCommand\s*\(\s*['"](?:${hostTools})['"]`,
String.raw`\bassertFlatToolCall\([^,\n]+,\s*\[\s*['"](?:${hostTools})['"]`,
String.raw`\bcalls\.push\(\[\s*['"](?:${hostTools})['"]`,
].join('|'),
'g',
);
}
function countSurfaceReferences(files, pattern) {
let references = 0;
let filesWithReferences = 0;
for (const file of files) {
const matches = countPatternReferences(fs.readFileSync(file, 'utf8'), pattern);
references += matches;
filesWithReferences += matches > 0 ? 1 : 0;
}
return { references, files: filesWithReferences };
}
function countPatternReferences(text, pattern) {
return text.match(pattern)?.length ?? 0;
}
function summarizePublicCommandCoverage(files, clientCommandMethods) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
const publicCommands = readPublicCommands();
const commandRefsByFile = files.map((file) => ({
file,
commands: extractProviderScenarioCommandReferences(
fs.readFileSync(file, 'utf8'),
clientCommandMethods,
),
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}));
return publicCommands.map((command) => {
let references = 0;
let filesWithReferences = 0;
for (const file of commandRefsByFile) {
const count = file.commands.filter((candidate) => candidate === command).length;
references += count;
if (count > 0) filesWithReferences += 1;
}
return { command, references, files: filesWithReferences };
});
}
function readPublicCommands() {
const metadataNames = new Set(listCommandMetadata().map((metadata) => metadata.name));
return Object.values(PUBLIC_COMMANDS)
.map((name) => {
if (!metadataNames.has(name)) {
throw new Error(`Missing command metadata for public command: ${name}`);
}
return name;
})
.sort();
}
function readClientCommandMethods(commandContractFiles) {
const commands = new Map();
for (const file of commandContractFiles) {
const text = fs.readFileSync(file, 'utf8');
for (const block of readCommandContractBlocks(text)) {
for (const method of block.source.matchAll(
/\bclient\.([A-Za-z0-9_]+)\.([A-Za-z0-9_]+)\s*\(/g,
)) {
commands.set(`${method[1]}.${method[2]}`, block.name);
}
}
}
return commands;
}
function readCommandContractBlocks(text) {
const constants = new Map();
for (const match of text.matchAll(/\bconst\s+([A-Z0-9_]+)\s*=\s*['"]([^'"]+)['"]/g)) {
constants.set(match[1], match[2]);
}
const metadataNames = new Map();
for (const match of text.matchAll(
/\bconst\s+([A-Za-z0-9_]+CommandMetadata)\s*=\s*defineFieldCommandMetadata\(\s*([^,\s)]+)/g,
)) {
metadataNames.set(match[1], readMetadataName(match[2], constants));
}
const starts = [
...text.matchAll(/defineExecutableCommand\(\s*metadata\(\s*['"]([^'"]+)['"]\s*\)/g),
...[...text.matchAll(/defineExecutableCommand\(\s*([A-Za-z0-9_]+CommandMetadata)\b/g)].flatMap(
(match) => {
const name = metadataNames.get(match[1]);
return name ? [{ ...match, 1: name }] : [];
},
),
...text.matchAll(/defineFieldCommand\(\s*['"]([^'"]+)['"]/g),
...text.matchAll(/defineCommand\(\s*\{[\s\S]*?\bname:\s*['"]([^'"]+)['"]/g),
]
.map((match) => ({
index: match.index ?? 0,
name: match[1],
}))
.sort((a, b) => a.index - b.index);
return starts.map((start, index) => {
const end = starts[index + 1]?.index ?? text.length;
return {
name: start.name,
source: text.slice(start.index, end),
};
});
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
}
function readMetadataName(token, constants) {
const literal = token.match(/^['"]([^'"]+)['"]$/);
if (literal) return literal[1];
return constants.get(token);
}
function extractProviderScenarioCommandReferences(text, clientCommandMethods) {
return [
...extractLiteralCommandReferences(text),
...extractClientCommandReferences(text, clientCommandMethods),
];
}
function extractLiteralCommandReferences(text) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
const commands = [];
for (const match of text.matchAll(
/\bcommand:\s*['"]([^'"]+)['"]|\.callCommand\(\s*['"]([^'"]+)['"]/g,
)) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
commands.push(match[1] ?? match[2]);
}
return commands;
}
function extractClientCommandReferences(text, clientCommandMethods) {
const commands = [];
for (const [method, command] of clientCommandMethods) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
const escapedMethod = method.replace('.', '\\.');
const matches = countPatternReferences(text, new RegExp(`\\.${escapedMethod}\\s*\\(`, 'g'));
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
for (let index = 0; index < matches; index += 1) commands.push(command);
}
return commands;
}
function countTestDeclarations(text) {
return [...text.matchAll(/(?:^|[^\w.])test\(/g)].length;
}
function readCoverageSummary(coverageSummary) {
const total = readCoverageSummaryJson(coverageSummary)?.total;
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
if (!total) return null;
return {
statements: readCoveragePercent(total, 'statements'),
branches: readCoveragePercent(total, 'branches'),
functions: readCoveragePercent(total, 'functions'),
lines: readCoveragePercent(total, 'lines'),
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
};
}
function readLowCoverageFiles(root, coverageSummary) {
const summary = readCoverageSummaryJson(coverageSummary);
if (!summary) return [];
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
return Object.entries(summary)
.filter(([file]) => file !== 'total')
.map(([file, value]) => readLowCoverageFile(root, file, value))
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
.filter((file) => file.statementTotal >= 10 && file.statementPercent < 60)
.sort((a, b) => b.missingStatements - a.missingStatements)
.slice(0, 10);
}
function readCoverageSummaryJson(coverageSummary) {
if (!fs.existsSync(coverageSummary)) return null;
return JSON.parse(fs.readFileSync(coverageSummary, 'utf8'));
}
function readCoveragePercent(total, key) {
return Number((total[key] ?? EMPTY_COVERAGE_METRIC).pct);
}
function readLowCoverageFile(root, file, value) {
const statements = value.statements ?? EMPTY_STATEMENT_COVERAGE;
const statementTotal = Number(statements.total);
const statementCovered = Number(statements.covered);
return {
file: path.relative(root, file),
statementPercent: Number(statements.pct),
statementTotal,
missingStatements: statementTotal - statementCovered,
};
}
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
function ratio(numerator, denominator) {
if (denominator === 0) return 'n/a';
return `${((numerator / denominator) * 100).toFixed(1)}%`;
}
export function formatPercent(value) {
refactor: add provider-first device lab tests (#542) * refactor: add provider-first device lab tests * refactor: tighten device lab provider seams * test: cover provider lab contracts * docs: record device lab harness direction * ci: run device lab integration tests * test: move device lab under integration * test: extract device lab helpers * refactor: centralize apps filter defaults * test: drop lab-covered unit tests * test: fold platform happy paths into device lab * test: reuse device lab helpers * test: move device lab to in-process harness * test: replace session handler cases with device lab * test: harden device lab scenario contracts * docs: define unit test retention policy * test: expand provider device lab coverage * test: harden provider device lab coverage * test: cover manifest install and runner session contracts * chore: remove unused provider cleanup code * test: split android find device lab scenario * test: track provider lab architecture progress * test: clarify provider lab roadmap progress * test: advance provider lab session coverage * test: move menubar click routing to device lab * test: move menubar snapshots to device lab * refactor: centralize screenshot flag plumbing * refactor: colocate screenshot flag metadata * test: cover all public commands in device lab * test: move macos wait success to device lab * test: drop redundant perf and diff units * test: move push payload paths to device lab * test: move network parsing to device lab * test: move log cleanup to device lab * test: move log restart and boot to device lab * test: move ios physical boot to device lab * test: cover perf startup in device lab * test: extract android and ios device lab worlds * test: trim device lab world surface * test: split snapshot capture unit coverage * test: deepen device lab coverage and trim handler units * test: clean up device lab migration scaffolding * test: report device lab public command coverage * refactor: make Apple provider seams semantic * refactor: tighten device inventory and Linux provider seams * refactor: tighten request provider scoping * refactor: add semantic macos host provider * test: broaden device lab find coverage * test: cover workflow flags in device lab * refactor: promote linux input provider seam * test: clarify device lab flag coverage * test: classify snapshot force-full progress * test: enforce device lab progress in ci * test: stabilize device lab ci * test: move packaged metro smoke to integration * test: drop stale provider seam coverage * test: harden provider scope regression coverage * refactor: remove stale platform barrels * refactor: keep linux clipboard and screenshots semantic * refactor: move macos host tools behind provider * fix: honor remote artifact output paths * test: deepen runtime coverage for daemon and runner paths * test: share loopback test helpers * refactor: make daemon runtime importable * fix: honor replay target metadata * chore: tighten final device lab quality gates * test: share device lab setup helpers * test: remove generic apple lab fallback * test: deduplicate device lab helpers * chore: tighten fallow duplication signal * refactor: share apple diagnostic helpers * fix: detect active android ime during fill verification * test: consolidate provider-backed integration suite * ci: fix fallow and iOS smoke setup * chore: consolidate cleanup after ci fixes * test: split vitest unit and integration projects * docs: mention MCP discovery metadata * docs: add agent skills context pointers * fix: close provider recording coverage gaps * fix: restore mcp compatibility smoke * test: cover provider edge regressions * test: consolidate loopback helpers * docs: remove stale provider routing reference * fix: harden final provider review issues * chore: defer mcp cleanup from provider refactor
2026-05-18 14:50:52 +02:00
return `${value.toFixed(2)}%`;
}