fix(android): retire recording evidence stranded by a re-adopted device id (#2564)

* fix(android): retire recording evidence stranded by a re-adopted device id

Android `record start` refused forever with "native recovery evidence already
exists" once an emulator was re-adopted under a new serial: the device-side
marker names the device identity that wrote it, reconciliation retired evidence
only when that identity matched, and the leftover classified as neither
recoverable nor retireable — an `UNKNOWN` internal error whose hint asked for a
bug report, while `record stop` owned nothing to clear.

Start reconciliation now retires evidence whose recording is terminal or whose
device identity the transport can no longer address, and only after every
artifact it names is proven released: a committed recorder through process
inspect, an uncommitted pending artifact against the recorders running on the
device. A recorder still writing is never deleted; the refusal names the writer
in `details.writer`. Remaining refusals — unreadable evidence, the other
transport mode, and an open recording this identity owns — are typed errors
carrying the marker path and the command that clears it.

Closes #2550

* fix(android): keep an unreadable recorder off the delete path

The writer probe filtered candidate processes down to the ones it could prove
were recorders, so a caller that read an empty list as "nothing writes this
path" removed an artifact from under a recorder whose /proc entries could not
be read, along with the marker that named it. A process table it could not read
at all threw a bare Error, which reached callers as an unclassified failure.

The transport answers a writer search with `clear`, `found`, or `uncertain`.
Only `clear` proves an artifact is free: start retirement refuses with
`native_recording_recorder_unproven` and owned cleanup keeps the evidence
pending, so both wait for a conclusive answer instead of deleting on doubt.

* fix(android): keep a mixed writer scan inconclusive

The writer search answered `found` as soon as it identified one recorder, which
hid the candidates it could not read. Start reconciliation already refuses a
found writer, but owned cleanup stops only the recorders it was handed and then
removes the artifact and marker — deleting under an unreadable recorder still
writing that same path.

A search now reports the writers it identified together with whether every
candidate was read. Cleanup requires both halves: an identified recorder does
not prove the others are gone, so an inconclusive scan retains the evidence
before anything is signalled or removed.

* test(android): record retirement side effects through one evidence rig

21 reconciliation scenarios each rebuilt the same marker reader and the same
recording transport stubs. One rig holds the marker and the ordered side-effect
log, so a test names only the probe outcome it is about.

* test(android): keep legacy reconciliation scenarios as they were

The scenarios that already covered retirement were rewritten into a shared rig,
which spent most of this PR's churn budget on moving lines around. They read
from the device marker again as before; only the recorder-state table names the
outcome each state now produces, and the stale row that expected evidence naming
another device identity to be kept is gone, since retiring it is this fix.
New scenarios use the rig.

* test(android): table-drive the inconclusive writer scans

Each inconclusive writer-scan scenario rebuilt the same scoped adb fake and the
same cleanup transport. The transport cases now differ only in which candidate
reads fail, and the owned-cleanup cases differ only in the scan they return, so
both run from one table against the same assertions.
This commit is contained in:
Michał Pierzchała
2026-09-14 11:52:56 +02:00
committed by GitHub
parent 508b750fbd
commit 04052fdcd2
18 changed files with 739 additions and 172 deletions
+19
View File
@@ -55,6 +55,25 @@
process is never signalled. A reused pid that runs a replacement `screenrecord` on the same
remote path proves the old recorder gone but not that the path is free, so that marker and
artifact are retained until the replacement ends, and neither is signalled (#2476).
- Fixed: Android `record start` no longer refuses forever on an emulator re-adopted under a new
serial. The device-side marker records the device identity that wrote it and a later start retired it
only when that identity matched, so a marker left by an earlier session on the same AVD blocked every
recording on the re-adopted device with `Android screenrecord native recovery evidence already exists`
— reported as an internal error whose hint asked for a bug report — while `record stop` answered that
no recording was active and no session could reach the marker, because recovery always binds the
device identity its own record names. A marker now retires when its recording is terminal or when it
names a device identity this transport can no longer address, and only once every recorder it names
has provably released its artifact; an interrupted launch commits no recorder identity, so its
artifact is checked against the recorders running on the device first. A recorder that is still
writing is left alone: `record start` refuses with `DEVICE_IN_USE` and `details.writer` naming whether
the marker's own recorder or another one holds the path, and only the unmanaged one ends on its own at
Android's 180 second limit. A device that cannot answer that question — an unreadable process table,
or a candidate process whose identity cannot be read — is refused rather than assumed free, even when
other recorders writing that path were identified, so an artifact is never removed under a recorder
the probe failed to see. What still refuses — unreadable or undecodable evidence, a marker the other
transport mode wrote, and an open recording this device identity still owns — is now a typed error
naming the marker path and the command that clears it, `record stop --session <name>` or removing the
marker once no session owns it, instead of `UNKNOWN` (#2550).
- Fixed: a polling `wait` no longer surrenders its whole budget the first time the iOS runner
answers `RUNNER_BUSY`. That code means an earlier command exceeded the runner's execution
watchdog and its abandoned main-thread work is still draining, which clears on its own, so a
@@ -94,6 +94,16 @@ export type AndroidScreenRecordingProcessIdentity = Readonly<{
startTime: string;
}>;
/**
* Which recorders write a path that no committed identity names, and whether the scan can prove that
* list is the whole story. `conclusive` means every candidate process was read; an unreadable one
* clears neither, so only a conclusive scan with no writers proves the path is free.
*/
export type AndroidScreenRecordingWriterSearch = Readonly<{
writers: readonly AndroidScreenRecordingProcessIdentity[];
conclusive: boolean;
}>;
/**
* `ownership-lost`: the pid is present, yet the identity readable there names something else — a
* reassigned pid, or an exited task whose command line is already gone. `foreign-writer`: the pid
@@ -168,10 +178,10 @@ export type AndroidScreenRecordingTransport = Readonly<{
): Promise<AndroidScreenRecordingStopOutcome>;
exists(remotePath: string, signal?: AbortSignal): Promise<boolean | 'uncertain'>;
size(remotePath: string, signal?: AbortSignal): Promise<number | undefined | 'uncertain'>;
findRunning(
probeRunningWriters(
remotePath: string,
signal?: AbortSignal,
): Promise<readonly AndroidScreenRecordingProcessIdentity[]>;
): Promise<AndroidScreenRecordingWriterSearch>;
pullPlayable(
input: Readonly<{ remotePath: string; outputPath: string }>,
signal?: AbortSignal,
@@ -2,8 +2,8 @@ import type { CleanupOutcome } from '@agent-device/contracts/durable-resource';
import type { PlatformRuntimeHost } from '@agent-device/contracts/platform-runtime-operations';
import { cleanupChunks, stopOwnedChunks } from './chunks.ts';
import { pending } from './completion.ts';
import type { NativeManifest } from './manifest.ts';
import { removeNativeManifest } from './launch.ts';
import type { NativeChunk, NativeManifest } from './manifest.ts';
import { removeNativeManifest } from './manifest-store.ts';
type Transport = Awaited<ReturnType<PlatformRuntimeHost['screenRecording']['android']['resolve']>>;
@@ -15,16 +15,10 @@ export async function cleanupVerifiedAndroidEvidence(
): Promise<CleanupOutcome> {
try {
const pendingPath = evidence.pendingRemotePath;
const pendingChunks =
pendingPath === undefined
? []
: (await transport.findRunning(pendingPath)).map((processIdentity) => ({
index: evidence.chunks.length + 1,
remotePath: pendingPath,
remotePid: processIdentity.pid,
remoteStartTime: processIdentity.startTime,
}));
await stopOwnedChunks(transport, [...evidence.chunks, ...pendingChunks]);
await stopOwnedChunks(transport, [
...evidence.chunks,
...(await pendingWriterChunks(transport, evidence)),
]);
await cleanupChunks(transport, evidence.chunks);
if (pendingPath !== undefined && !(await transport.remove(pendingPath)))
throw new Error(`failed to remove Android recording artifact: ${pendingPath}`);
@@ -34,3 +28,25 @@ export async function cleanupVerifiedAndroidEvidence(
return pending(error);
}
}
/**
* Recorders writing an artifact the evidence never committed. An inconclusive scan is retained like
* any other uncertainty: one identified recorder does not prove the others are gone, and stopping
* only some of them before deleting would delete under the rest.
*/
async function pendingWriterChunks(
transport: Transport,
evidence: NativeManifest,
): Promise<readonly NativeChunk[]> {
const pendingPath = evidence.pendingRemotePath;
if (pendingPath === undefined) return [];
const writers = await transport.probeRunningWriters(pendingPath);
if (!writers.conclusive)
throw new Error(`cannot list every recorder writing Android artifact: ${pendingPath}`);
return writers.writers.map((writer, offset) => ({
index: evidence.chunks.length + 1 + offset,
remotePath: pendingPath,
remotePid: writer.pid,
remoteStartTime: writer.startTime,
}));
}
@@ -3,7 +3,7 @@ import type { ScreenRecordingLiveSnapshot } from '@agent-device/contracts/screen
import { cleanupChunks, pullChunks, stopOwnedChunks, waitForStableArtifacts } from './chunks.ts';
import { completed } from './completion.ts';
import { createCompletedNativeManifest, type NativeManifest } from './manifest.ts';
import { persistNativeManifest } from './launch.ts';
import { persistNativeManifest } from './manifest-store.ts';
type Transport = Awaited<ReturnType<PlatformRuntimeHost['screenRecording']['android']['resolve']>>;
@@ -75,11 +75,16 @@ export function recordingHost(overrides: Record<string, unknown>): PlatformRunti
},
removeManifest: async (manifestPath: string) =>
legacy.removeManifest ? await legacy.removeManifest(manifestPath) : true,
findRunning: async (remotePath: string) => {
probeRunningWriters: async (remotePath: string) => {
const found = await (legacy.findRunning?.(remotePath) ?? ['42', '43', '66']);
return found.map((entry: string | { pid: string; remotePath: string; startTime: string }) =>
typeof entry === 'string' ? { pid: entry, remotePath, startTime: '1' } : entry,
);
if (!Array.isArray(found)) return found;
return {
writers: found.map(
(entry: string | { pid: string; remotePath: string; startTime: string }) =>
typeof entry === 'string' ? { pid: entry, remotePath, startTime: '1' } : entry,
),
conclusive: true,
};
},
};
return {
@@ -98,7 +98,8 @@ test('cancellation during active manifest publication retains evidence after unc
readManifest: async () => ({ status: 'read' as const, contents: manifest }),
start: async () => recordingProcess('43'),
});
await expect(replacement.screenRecordingStart(recordingInput())).rejects.toThrow(
'native recovery evidence already exists',
);
await expect(replacement.screenRecordingStart(recordingInput())).rejects.toMatchObject({
code: 'DEVICE_IN_USE',
details: { reason: 'native_recovery_evidence_open' },
});
});
@@ -65,9 +65,10 @@ test('retained native evidence blocks replacement before output preparation or l
owner: localRuntimeOwner('android'),
signal: new AbortController().signal,
});
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toThrow(
'native recovery evidence already exists',
);
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toMatchObject({
code: 'COMMAND_FAILED',
details: { reason: 'native_recovery_evidence_unreadable' },
});
expect({ starts, writes, prepared }).toEqual({ starts: 0, writes: 0, prepared: 0 });
});
@@ -98,9 +99,10 @@ test('unavailable native evidence blocks replacement before output preparation o
owner: localRuntimeOwner('android'),
signal: new AbortController().signal,
});
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toThrow(
'native recovery evidence is unavailable',
);
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toMatchObject({
code: 'COMMAND_FAILED',
details: { reason: 'native_recovery_evidence_unavailable' },
});
expect({ starts, writes, prepared }).toEqual({ starts: 0, writes: 0, prepared: 0 });
});
@@ -1,27 +1,17 @@
import type { DeviceInfo } from '@agent-device/kernel/device';
import type { PlatformRuntimeHost } from '@agent-device/contracts/platform-runtime-operations';
import type { ScreenRecordingStartInput } from '@agent-device/contracts/screen-recording-runtime';
import { provesAndroidScreenRecordPathUnclaimed } from '@agent-device/contracts/screen-recording-runtime-host';
import {
cleanupChunks,
AndroidScreenRecordingStartRollbackUnconfirmed,
candidateRemotePaths,
rollbackChunks,
startChunkAt,
} from './chunks.ts';
import {
createNativeManifest,
decodeNativeManifest,
type NativeChunk,
type NativeManifest,
} from './manifest.ts';
import { createNativeManifest, type NativeChunk } from './manifest.ts';
import { persistNativeManifest, removeNativeManifest } from './manifest-store.ts';
import { reconcileStartEvidence } from './start-reconciliation.ts';
type Transport = Awaited<ReturnType<PlatformRuntimeHost['screenRecording']['android']['resolve']>>;
type ManifestCandidate = Readonly<{
manifestPath: string;
read: Awaited<ReturnType<Transport['readManifest']>>;
}>;
type CompletedCandidate = Readonly<{ manifestPath: string; evidence: NativeManifest }>;
export async function startInitialTransaction(params: {
transport: Transport;
@@ -32,7 +22,7 @@ export async function startInitialTransaction(params: {
prepareOutput: () => Promise<void>;
}): Promise<Readonly<{ chunk: NativeChunk; manifestPath: string }>> {
const { transport, device, input, startedAt, signal, prepareOutput } = params;
await reconcileCompletedStartEvidence(transport, device);
await reconcileStartEvidence(transport, device);
await prepareOutput();
let last: unknown;
for (const remotePath of candidateRemotePaths(undefined)) {
@@ -97,91 +87,6 @@ async function rollbackPublishedChunk(
await removeNativeManifest(transport, manifestPath).catch(() => {});
}
/**
* A terminal marker outlives native cleanup until a later, fenced start reconciles it. Never
* retire open or uncertain evidence: that would erase the only recovery authority after a crash.
*/
async function reconcileCompletedStartEvidence(
transport: Transport,
device: DeviceInfo,
): Promise<void> {
const candidates = await readManifestCandidates(transport);
const completed = candidates.flatMap((candidate) =>
completedCandidate(candidate, device, transport.mode),
);
for (const candidate of completed) {
await retireCompletedEvidence(transport, candidate.evidence, candidate.manifestPath);
}
}
async function readManifestCandidates(transport: Transport): Promise<readonly ManifestCandidate[]> {
return await Promise.all(
candidateRemotePaths(undefined).map(async (remotePath) => {
const manifestPath = transport.manifestPathFor(remotePath);
return { manifestPath, read: await transport.readManifest(manifestPath) };
}),
);
}
function completedCandidate(
candidate: ManifestCandidate,
device: DeviceInfo,
transportMode: NativeManifest['transportMode'],
): readonly CompletedCandidate[] {
if (candidate.read.status === 'missing') return [];
if (candidate.read.status !== 'read') throw unavailableEvidence();
const evidence = decodeNativeManifest(candidate.read.contents);
if (!isRetireableCompletedEvidence(evidence, device, transportMode)) throw existingEvidence();
return [{ manifestPath: candidate.manifestPath, evidence }];
}
function isRetireableCompletedEvidence(
evidence: NativeManifest | undefined,
device: DeviceInfo,
transportMode: NativeManifest['transportMode'],
): evidence is NativeManifest {
return (
evidence !== undefined &&
evidence.completion !== undefined &&
evidence.pendingRemotePath === undefined &&
evidence.deviceId === device.id &&
evidence.transportMode === transportMode
);
}
function unavailableEvidence(): Error {
return new Error('Android screenrecord native recovery evidence is unavailable');
}
function existingEvidence(): Error {
return new Error('Android screenrecord native recovery evidence already exists');
}
async function retireCompletedEvidence(
transport: Transport,
evidence: NativeManifest,
manifestPath: string,
): Promise<void> {
for (const chunk of evidence.chunks) {
const state = await transport.inspect({
pid: chunk.remotePid,
remotePath: chunk.remotePath,
startTime: chunk.remoteStartTime,
});
if (state === 'foreign-writer')
throw new Error(
'Android screenrecord completed evidence names an artifact another recorder is writing; it is retained until that recorder ends',
);
if (!provesAndroidScreenRecordPathUnclaimed(state))
throw new Error('Android screenrecord completed evidence cannot be safely retired');
}
await cleanupChunks(transport, evidence.chunks);
await removeNativeManifest(transport, manifestPath);
const confirmed = await transport.readManifest(manifestPath);
if (confirmed.status !== 'missing')
throw new Error('Android screenrecord completed evidence removal could not be confirmed');
}
export async function startPendingChunk(params: {
transport: Transport;
device: DeviceInfo;
@@ -230,21 +135,3 @@ async function removeFailedCandidateManifest(
throw new AndroidScreenRecordingStartRollbackUnconfirmed(launchError);
}
}
export async function persistNativeManifest(
transport: Transport,
manifestPath: string,
evidence: NativeManifest,
signal?: AbortSignal,
): Promise<void> {
await transport.writeManifest({ manifestPath, contents: JSON.stringify(evidence) }, signal);
}
export async function removeNativeManifest(
transport: Transport,
manifestPath: string,
): Promise<void> {
if (!(await transport.removeManifest(manifestPath))) {
throw new Error(`failed to remove Android recording manifest: ${manifestPath}`);
}
}
@@ -0,0 +1,23 @@
import type { PlatformRuntimeHost } from '@agent-device/contracts/platform-runtime-operations';
import type { NativeManifest } from './manifest.ts';
type Transport = Awaited<ReturnType<PlatformRuntimeHost['screenRecording']['android']['resolve']>>;
/** Publish evidence atomically, so a reader never observes a half-written marker. */
export async function persistNativeManifest(
transport: Transport,
manifestPath: string,
evidence: NativeManifest,
signal?: AbortSignal,
): Promise<void> {
await transport.writeManifest({ manifestPath, contents: JSON.stringify(evidence) }, signal);
}
export async function removeNativeManifest(
transport: Transport,
manifestPath: string,
): Promise<void> {
if (!(await transport.removeManifest(manifestPath))) {
throw new Error(`failed to remove Android recording manifest: ${manifestPath}`);
}
}
@@ -131,6 +131,56 @@ test('cleans verified dead evidence so a later start is admitted', async () => {
expect(starts).toBe(2);
});
const inconclusiveWriterScans = [
['no recorder at all', { writers: [], conclusive: false }],
[
'one recorder and an unreadable candidate',
{
writers: [{ pid: '88', remotePath: '/sdcard/agent-device-recording-9.mp4', startTime: '4' }],
conclusive: false,
},
],
];
test.each(inconclusiveWriterScans)(
'retains evidence when the pending artifact writer scan reports %s',
async (_name, scan) => {
const pendingPath = '/sdcard/agent-device-recording-9.mp4';
let manifest = '';
const removed: string[] = [];
const signalled: string[] = [];
const runtime = await start({
writeManifest: async ({ contents }: { contents: string }) => {
manifest = contents;
},
readManifest: async () =>
manifest ? { status: 'read' as const, contents: manifest } : { status: 'missing' as const },
removeManifest: async () => {
manifest = '';
return true;
},
remove: async (path: string) => {
removed.push(path);
return true;
},
signal: async ({ pid }: { pid: string }) => {
signalled.push(pid);
return true;
},
findRunning: async () => scan,
isRunning: async () => false,
});
const started = await runtime.screenRecordingStart(recordingInput());
manifest = JSON.stringify({ ...JSON.parse(manifest), pendingRemotePath: pendingPath });
await expect(
runtime.screenRecordingCleanup({ envelope: started.envelope }),
).resolves.toMatchObject({ status: 'cleanup-pending' });
expect(signalled).toEqual([]);
expect(removed).toEqual([]);
expect(manifest).not.toBe('');
},
);
test('retains evidence when the recorder presence probe is uncertain', async () => {
vi.useFakeTimers();
try {
@@ -153,9 +203,10 @@ test('retains evidence when the recorder presence probe is uncertain', async ()
reason: 'transport-failed',
});
expect(manifest).not.toBe('');
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toThrow(
'native recovery evidence already exists',
);
await expect(runtime.screenRecordingStart(recordingInput())).rejects.toMatchObject({
code: 'DEVICE_IN_USE',
details: { reason: 'native_recovery_evidence_open' },
});
} finally {
vi.useRealTimers();
}
@@ -19,7 +19,8 @@ import {
type NativeChunk,
} from './manifest.ts';
import { rollbackChunks, stopChunk } from './chunks.ts';
import { persistNativeManifest, startInitialTransaction, startPendingChunk } from './launch.ts';
import { startInitialTransaction, startPendingChunk } from './launch.ts';
import { persistNativeManifest } from './manifest-store.ts';
import { snapshot } from './completion.ts';
import { finalizeAndroidRecording } from './finalize.ts';
import { cleanupVerifiedAndroidEvidence } from './cleanup.ts';
@@ -99,9 +99,9 @@ test('retires completed evidence whose recorder pid was reassigned, signaling no
});
test.each([
['an unconfirmed recorder identity', 'uncertain'],
['a live recorder', 'owned-alive'],
])('refuses completed evidence named by %s', async (_name, ownership) => {
['an unconfirmed recorder identity', 'uncertain', 'cannot be safely retired'],
['a live recorder', 'owned-alive', 'its recorder is writing'],
])('refuses completed evidence named by %s', async (_name, ownership, message) => {
const marker = JSON.stringify(completedEvidence());
const calls: string[] = [];
const runtime = await start({
@@ -132,9 +132,7 @@ test.each([
return recordingProcess('77');
},
});
await expect(runtime.screenRecordingStart(newInput())).rejects.toThrow(
'cannot be safely retired',
);
await expect(runtime.screenRecordingStart(newInput())).rejects.toThrow(message);
expect(calls).toEqual([]);
});
@@ -264,7 +262,6 @@ test.each([
),
],
['corrupt', '{broken'],
['wrong device', JSON.stringify({ ...completedEvidence(), deviceId: 'other-device' })],
[
'changed completion outPath',
JSON.stringify(tamperCompletion({ outPath: '/tmp/unrelated.mp4' })),
@@ -306,6 +303,197 @@ test.each([
expect(calls).toEqual([]);
});
/**
* One marker the runtime can read, plus transport stubs that record every retirement side effect in
* the order it happens. New scenarios name only the probe outcome they are about; the scenarios
* above build their transport inline.
*/
function evidenceRig(marker: string, directory = '/sdcard') {
let contents = marker;
const calls: string[] = [];
const transport: Record<string, unknown> = {
readManifest: async (path: string) =>
path.startsWith(directory) && contents
? { status: 'read' as const, contents }
: { status: 'missing' as const },
remove: async (remotePath: string) => {
calls.push(`artifact:${remotePath}`);
return true;
},
removeManifest: async () => {
calls.push('manifest');
contents = '';
return true;
},
outputs: {
prepare: async () => {
calls.push('prepare');
},
},
start: async () => {
calls.push('launch');
return recordingProcess('77');
},
};
return {
calls,
remains: () => contents !== '',
clear: () => {
contents = '';
},
bind: async (overrides: Record<string, unknown> = {}) =>
await bindAndroidScreenRecordingRuntime({
host: recordingHost({ ...transport, ...overrides }),
device: androidRecordingDevice,
owner: localRuntimeOwner('android'),
signal: new AbortController().signal,
}),
};
}
const retirementOf = (remotePath: string) => [
`artifact:${remotePath}`,
'manifest',
'prepare',
'launch',
];
test('retires evidence a re-adopted device identity stranded, before output preparation or launch', async () => {
const rig = evidenceRig(strandedEvidence(completedEvidence()));
const runtime = await rig.bind({ inspect: async () => 'missing' });
const started = await runtime.screenRecordingStart(newInput());
expect(rig.calls).toEqual(retirementOf('/sdcard/agent-device-recording-1.mp4'));
await started.pendingHandle.transfer().forceCleanup();
});
test('retains open evidence from this device identity even after its recorder is gone', async () => {
const rig = evidenceRig(JSON.stringify(openEvidence()));
const runtime = await rig.bind({ inspect: async () => 'missing' });
await expect(runtime.screenRecordingStart(newInput())).rejects.toMatchObject({
code: 'DEVICE_IN_USE',
details: { reason: 'native_recovery_evidence_open', sessionId: 'one', retriable: false },
});
expect(rig.calls).toEqual([]);
});
test('refuses stranded evidence whose recorder is still writing its artifact', async () => {
const rig = evidenceRig(strandedEvidence(completedEvidence()));
const runtime = await rig.bind({ inspect: async () => 'owned-alive' });
await expect(runtime.screenRecordingStart(newInput())).rejects.toMatchObject({
code: 'DEVICE_IN_USE',
details: {
reason: 'native_recording_artifact_claimed',
writer: 'named-recorder',
retriable: false,
},
});
expect(rig.calls).toEqual([]);
});
test('retires a stranded interrupted launch and drops the artifact it never committed', async () => {
const rig = evidenceRig(strandedEvidence(interruptedEvidence()));
const runtime = await rig.bind({ findRunning: async () => [] });
const started = await runtime.screenRecordingStart(newInput());
expect(rig.calls).toEqual(retirementOf('/sdcard/agent-device-recording-9.mp4'));
await started.pendingHandle.transfer().forceCleanup();
});
test('refuses a stranded interrupted launch whose artifact a recorder is still writing', async () => {
const rig = evidenceRig(strandedEvidence(interruptedEvidence()));
const runtime = await rig.bind({
findRunning: async () => ({
writers: [{ pid: '88', remotePath: '/sdcard/x', startTime: '4' }],
conclusive: true,
}),
});
await expect(runtime.screenRecordingStart(newInput())).rejects.toMatchObject({
code: 'DEVICE_IN_USE',
details: { reason: 'native_recording_artifact_claimed', writer: 'other-recorder' },
});
expect(rig.calls).toEqual([]);
});
test('refuses a stranded interrupted launch whose writers cannot be read', async () => {
const rig = evidenceRig(strandedEvidence(interruptedEvidence()));
const runtime = await rig.bind({
findRunning: async () => ({ writers: [], conclusive: false }),
});
await expect(runtime.screenRecordingStart(newInput())).rejects.toMatchObject({
code: 'COMMAND_FAILED',
details: {
reason: 'native_recording_recorder_unproven',
remotePath: '/sdcard/agent-device-recording-9.mp4',
},
});
expect(rig.calls).toEqual([]);
});
test('retires stranded evidence parked in the fallback directory', async () => {
const rig = evidenceRig(
strandedEvidence({
...openEvidence(),
chunks: [
{
index: 1,
remotePath: '/data/local/tmp/agent-device-recording-1.mp4',
remotePid: '41',
remoteStartTime: '7',
},
],
}),
'/data/local/tmp',
);
const runtime = await rig.bind({ inspect: async () => 'missing' });
const started = await runtime.screenRecordingStart(newInput());
expect(rig.calls).toEqual(retirementOf('/data/local/tmp/agent-device-recording-1.mp4'));
await started.pendingHandle.transfer().forceCleanup();
});
test('refuses evidence a different transport mode wrote', async () => {
const rig = evidenceRig(
JSON.stringify({ ...openEvidence(), transportMode: 'transport-composed' }),
);
const runtime = await rig.bind();
await expect(runtime.screenRecordingStart(newInput())).rejects.toMatchObject({
code: 'COMMAND_FAILED',
details: { reason: 'native_recovery_evidence_transport_mismatch' },
});
expect(rig.calls).toEqual([]);
});
function openEvidence() {
return createNativeManifest(
androidRecordingDevice,
recordingInput(),
1,
[
{
index: 1,
remotePath: '/sdcard/agent-device-recording-1.mp4',
remotePid: '41',
remoteStartTime: '7',
},
],
undefined,
'local',
);
}
function interruptedEvidence() {
return createNativeManifest(
androidRecordingDevice,
recordingInput(),
1,
[],
'/sdcard/agent-device-recording-9.mp4',
'local',
);
}
function strandedEvidence(evidence: object) {
return JSON.stringify({ ...evidence, deviceId: 'emulator-5556' });
}
function completedEvidence() {
const input = recordingInput();
return createCompletedNativeManifest(
@@ -0,0 +1,246 @@
import type { DeviceInfo } from '@agent-device/kernel/device';
import { AppError } from '@agent-device/kernel/errors';
import type { PlatformRuntimeHost } from '@agent-device/contracts/platform-runtime-operations';
import { provesAndroidScreenRecordPathUnclaimed } from '@agent-device/contracts/screen-recording-runtime-host';
import { candidateRemotePaths, cleanupChunks } from './chunks.ts';
import { decodeNativeManifest, type NativeManifest } from './manifest.ts';
import { removeNativeManifest } from './manifest-store.ts';
type Transport = Awaited<ReturnType<PlatformRuntimeHost['screenRecording']['android']['resolve']>>;
type ManifestCandidate = Readonly<{
manifestPath: string;
read: Awaited<ReturnType<Transport['readManifest']>>;
}>;
type RetireableCandidate = Readonly<{ manifestPath: string; evidence: NativeManifest }>;
/**
* A marker outlives the native cleanup that would have removed it, and a later start is the only
* operation that always looks at it. Retire what nothing can recover any more and refuse what
* somebody still can: open evidence on this device identity still belongs to a live session or to
* its daemon's recovery, and unreadable evidence cannot be verified at all.
*/
export async function reconcileStartEvidence(
transport: Transport,
device: DeviceInfo,
): Promise<void> {
const candidates = await readManifestCandidates(transport);
const retireable = candidates.flatMap((candidate) =>
retireableCandidate(candidate, device, transport.mode),
);
for (const candidate of retireable) {
await retireEvidence(transport, candidate.evidence, candidate.manifestPath);
}
}
async function readManifestCandidates(transport: Transport): Promise<readonly ManifestCandidate[]> {
return await Promise.all(
candidateRemotePaths(undefined).map(async (remotePath) => {
const manifestPath = transport.manifestPathFor(remotePath);
return { manifestPath, read: await transport.readManifest(manifestPath) };
}),
);
}
function retireableCandidate(
candidate: ManifestCandidate,
device: DeviceInfo,
transportMode: NativeManifest['transportMode'],
): readonly RetireableCandidate[] {
if (candidate.read.status === 'missing') return [];
if (candidate.read.status !== 'read')
throw unavailableEvidence(candidate.manifestPath, candidate.read.message);
const evidence = decodeNativeManifest(candidate.read.contents);
if (evidence === undefined) throw corruptEvidence(candidate.manifestPath);
if (evidence.transportMode !== transportMode)
throw foreignTransportEvidence(candidate.manifestPath, evidence, transportMode);
if (!isRetireableEvidence(evidence, device))
throw openEvidenceOnDevice(candidate.manifestPath, evidence, device);
return [{ manifestPath: candidate.manifestPath, evidence }];
}
/**
* Terminal evidence is spent: its recorder ended and its terminal result is already in the marker.
* Evidence written under another device identity has nobody else to retire it either a durable
* envelope always names the device identity it was created for, so no session this transport serves
* can reach it. Only the old identity, which this device no longer has, could still come back for
* those artifacts, and `proveArtifactsReleased` is what refuses while a recorder here is writing.
*/
function isRetireableEvidence(evidence: NativeManifest, device: DeviceInfo): boolean {
return evidence.completion !== undefined || evidence.deviceId !== device.id;
}
async function retireEvidence(
transport: Transport,
evidence: NativeManifest,
manifestPath: string,
): Promise<void> {
await proveArtifactsReleased(transport, evidence);
await cleanupChunks(transport, evidence.chunks);
await removePendingArtifact(transport, evidence);
await removeNativeManifest(transport, manifestPath);
const confirmed = await transport.readManifest(manifestPath);
if (confirmed.status !== 'missing') throw retirementUnconfirmed(manifestPath);
}
/**
* Refuse while any artifact the marker names still has a recorder writing it, or while the device
* cannot prove that none is. A committed chunk answers with its own identity; an interrupted launch
* committed none, so the device is asked which recorders write that path.
*/
async function proveArtifactsReleased(
transport: Transport,
evidence: NativeManifest,
): Promise<void> {
for (const chunk of evidence.chunks) {
const state = await transport.inspect({
pid: chunk.remotePid,
remotePath: chunk.remotePath,
startTime: chunk.remoteStartTime,
});
if (provesAndroidScreenRecordPathUnclaimed(state)) continue;
if (state === 'uncertain') throw unprovenRecorder(chunk.remotePath);
throw artifactClaimed(
chunk.remotePath,
state === 'owned-alive' ? 'named-recorder' : 'other-recorder',
);
}
const pendingPath = evidence.pendingRemotePath;
if (pendingPath === undefined) return;
const scan = await transport.probeRunningWriters(pendingPath);
if (scan.writers.length > 0) throw artifactClaimed(pendingPath, 'other-recorder');
if (!scan.conclusive) throw unprovenRecorder(pendingPath);
}
async function removePendingArtifact(
transport: Transport,
evidence: NativeManifest,
): Promise<void> {
const pendingPath = evidence.pendingRemotePath;
if (pendingPath === undefined) return;
if (!(await transport.remove(pendingPath))) throw artifactRemovalFailed(pendingPath);
}
function unavailableEvidence(manifestPath: string, probeMessage: string): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord native recovery evidence is unavailable: ${manifestPath}`,
{
reason: 'native_recovery_evidence_unavailable',
manifestPath,
hint: 'Confirm the device is online and its storage readable, then run record start again.',
},
new Error(probeMessage),
);
}
function corruptEvidence(manifestPath: string): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord native recovery evidence is unreadable: ${manifestPath}`,
{
reason: 'native_recovery_evidence_unreadable',
manifestPath,
hint: markerRemovalHint(manifestPath),
},
);
}
function foreignTransportEvidence(
manifestPath: string,
evidence: NativeManifest,
transportMode: NativeManifest['transportMode'],
): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord native recovery evidence was written by the ${evidence.transportMode} transport, not this ${transportMode} one: ${manifestPath}`,
{
reason: 'native_recovery_evidence_transport_mismatch',
manifestPath,
evidenceTransportMode: evidence.transportMode,
hint: markerRemovalHint(manifestPath),
},
);
}
function openEvidenceOnDevice(
manifestPath: string,
evidence: NativeManifest,
device: DeviceInfo,
): AppError {
return new AppError(
'DEVICE_IN_USE',
`Android screenrecord native recovery evidence already exists: ${manifestPath} (open recording of session "${evidence.sessionId}" on ${device.id})`,
{
reason: 'native_recovery_evidence_open',
manifestPath,
sessionId: evidence.sessionId,
// Retrying this start cannot free the marker; only the owning session's stop or close can.
retriable: false,
hint: `Run record stop --session ${evidence.sessionId} to retire it, or close that session. If it is already closed, ${markerRemovalHint(manifestPath)}`,
},
);
}
function artifactClaimed(
remotePath: string,
writer: 'named-recorder' | 'other-recorder',
): AppError {
const named = writer === 'named-recorder';
return new AppError(
'DEVICE_IN_USE',
`Android screenrecord recovery evidence is retained: ${
named ? 'its recorder' : 'another recorder'
} is writing ${remotePath}`,
{
reason: 'native_recording_artifact_claimed',
remotePath,
writer,
// A recorder this marker names is a live recording: only its owner's stop frees it. An
// unmanaged writer ends by itself at Android's 180 second limit, so waiting recovers.
...(named ? { retriable: false } : {}),
hint: named
? 'Run record stop for the session that owns that recording before starting another.'
: 'Android ends any recorder after 180 seconds. Run record start again once that recorder has stopped.',
},
);
}
function artifactRemovalFailed(remotePath: string): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord recovery evidence could not remove ${remotePath}`,
{
reason: 'native_recording_artifact_removal_failed',
remotePath,
hint: 'Confirm the device is online and run record start again.',
},
);
}
function retirementUnconfirmed(manifestPath: string): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord recovery evidence removal could not be confirmed: ${manifestPath}`,
{
reason: 'native_recovery_evidence_retirement_unconfirmed',
manifestPath,
hint: 'Run record start again; the marker is retired from the beginning.',
},
);
}
function unprovenRecorder(remotePath: string): AppError {
return new AppError(
'COMMAND_FAILED',
`Android screenrecord recovery evidence cannot be safely retired: ${remotePath}`,
{
reason: 'native_recording_recorder_unproven',
remotePath,
hint: 'Confirm the device is online and run record start again.',
},
);
}
function markerRemovalHint(manifestPath: string): string {
return `no recording session can retire this marker; remove it on the device with \`adb shell rm -f ${manifestPath}\`.`;
}
+1 -1
View File
@@ -104,7 +104,7 @@ export const recordCommandFacet = defineCommandFacet({
text: {
summary: 'Start or stop screen recording',
cliDetail:
'The default --scope app requires an active app session from open <app>; use --scope device/system to explicitly request whole-screen recording where the selected backend supports it. Android record start publishes a durable device manifest, recordings longer than the 180s adb screenrecord limit are returned as multiple MP4 chunks while the daemon stays alive, and daemon-restart recovery uses only manifest-owned chunks. HarmonyOS supports whole-screen recording on physical devices only: use --scope device/system; --fps, --quality, and --hide-touches are unsupported. Use --quality to choose medium or high export quality on supported backends. An iOS simulator host recording lock returns non-retriable DEVICE_IN_USE with reason apple_simulator_recording_busy. Stop the recording in its owning session; if a dead recorder left the host locked, ask the host operator to restart the CoreSimulator stream service.',
'The default --scope app requires an active app session from open <app>; use --scope device/system to explicitly request whole-screen recording where the selected backend supports it. Android record start publishes a durable device manifest, recordings longer than the 180s adb screenrecord limit are returned as multiple MP4 chunks while the daemon stays alive, and daemon-restart recovery uses only manifest-owned chunks. An Android manifest left by an unreachable recording is retired on the next start once its recorders are proven gone; one still owned refuses with non-retriable DEVICE_IN_USE and reason native_recovery_evidence_open naming the session to run record stop for, and a recorder that is still writing an artifact refuses with DEVICE_IN_USE and reason native_recording_artifact_claimed, which clears itself once that recorder ends at the 180s limit. HarmonyOS supports whole-screen recording on physical devices only: use --scope device/system; --fps, --quality, and --hide-touches are unsupported. Use --quality to choose medium or high export quality on supported backends. An iOS simulator host recording lock returns non-retriable DEVICE_IN_USE with reason apple_simulator_recording_busy. Stop the recording in its owning session; if a dead recorder left the host locked, ask the host operator to restart the CoreSimulator stream service.',
},
metadata: recordCommandMetadata,
run: (client, input) => client.recording.record(input as RecordOptions),
@@ -97,13 +97,83 @@ test('finds only exact screenrecord processes for the canonical remote path', as
{ serial: android.id },
async () => {
const transport = await createAndroidScreenRecordingTransport(android);
await expect(transport.findRunning(remotePath)).resolves.toEqual([
{ pid: '41', remotePath, startTime: '41' },
{ pid: '44', remotePath, startTime: '44' },
]);
await expect(transport.probeRunningWriters(remotePath)).resolves.toEqual({
writers: [
{ pid: '41', remotePath, startTime: '41' },
{ pid: '44', remotePath, startTime: '44' },
],
conclusive: true,
});
},
);
});
const writerScanRemotePath = '/sdcard/agent-device-recording-123.mp4';
type WriterScanDevice = Readonly<{ pids?: string; unreadablePid?: string; writerPid?: string }>;
const inconclusiveWriterScans: readonly (readonly [
string,
WriterScanDevice,
{
writers: readonly { pid: string; remotePath: string; startTime: string }[];
conclusive: boolean;
},
])[] = [
['no process table at all', {}, { writers: [], conclusive: false }],
[
'an unreadable candidate',
{ pids: '41\n42\n', unreadablePid: '42' },
{ writers: [], conclusive: false },
],
[
'one writer and an unreadable candidate',
{ pids: '41\n42\n', unreadablePid: '42', writerPid: '41' },
{
writers: [{ pid: '41', remotePath: writerScanRemotePath, startTime: '41' }],
conclusive: false,
},
],
];
function unreadableProcStat(device: WriterScanDevice, pid: string) {
return pid === device.unreadablePid
? result('', `cat: /proc/${pid}/stat: Permission denied`, 1)
: result(procStat(Number(pid), pid));
}
function screenrecordArgv(device: WriterScanDevice, pid: string): string {
return device.writerPid === pid
? ['/system/bin/screenrecord', '--bit-rate', '8000000', writerScanRemotePath, ''].join('\0')
: ['/system/bin/sh', '-c', 'screenrecord', writerScanRemotePath, ''].join('\0');
}
test.each(inconclusiveWriterScans)(
'reports the writer scan inconclusive on %s',
async (_name, device: WriterScanDevice, expected) => {
await withAndroidAdbProvider(
{
exec: async (args) => {
const command = args[1] ?? '';
if (command === 'ps -A -o pid=') {
return device.pids ? result(device.pids) : result('', 'adb: device offline', 1);
}
if (/^test -d \/proc\/\d+$/.test(command)) return result();
const pid = /\/proc\/(\d+)\//.exec(command)?.[1] ?? '';
return command.endsWith('/stat')
? unreadableProcStat(device, pid)
: result(screenrecordArgv(device, pid));
},
},
{ serial: android.id },
async () => {
const transport = await createAndroidScreenRecordingTransport(android);
await expect(transport.probeRunningWriters(writerScanRemotePath)).resolves.toEqual(
expected,
);
},
);
},
);
test('revalidates start-time and exact argv before SIGINT', async () => {
const commands: string[] = [];
@@ -67,11 +67,9 @@ export async function createAndroidScreenRecordingTransport(
const size = Number(result.stdout.trim());
return Number.isSafeInteger(size) && size >= 0 ? size : 'uncertain';
},
findRunning: async (remotePath, signal) => {
probeRunningWriters: async (remotePath, signal) => {
const result = await shell('ps -A -o pid=', signal);
if (result.exitCode !== 0) {
throw new Error('failed to enumerate Android screenrecord processes');
}
if (result.exitCode !== 0) return { writers: [], conclusive: false };
const pids = result.stdout.split(/\s+/).filter((pid) => /^\d+$/.test(pid));
const inspected = await Promise.all(
pids.map(
@@ -83,9 +81,12 @@ export async function createAndroidScreenRecordingTransport(
),
),
);
return inspected.flatMap((outcome) =>
outcome.status === 'owned-alive' && outcome.process ? [outcome.process] : [],
);
return {
writers: inspected.flatMap((outcome) =>
outcome.status === 'owned-alive' && outcome.process ? [outcome.process] : [],
),
conclusive: !inspected.some((outcome) => outcome.status === 'uncertain'),
};
},
pullPlayable: async ({ remotePath, outputPath }, signal) => {
const result = await adb(['pull', remotePath, outputPath], {
@@ -33,6 +33,7 @@ export function buildAndroidRecordingManifest(options: {
remotePath: string;
sessionName: string;
startedAt?: number;
deviceId?: string;
chunks?: Array<{
index: number;
remotePath: string;
@@ -49,7 +50,7 @@ export function buildAndroidRecordingManifest(options: {
fenceToken: 'provider-fixture-fence',
fenceGeneration: 1,
sessionId: options.sessionName,
deviceId: PROVIDER_SCENARIO_ANDROID.id,
deviceId: options.deviceId ?? PROVIDER_SCENARIO_ANDROID.id,
startedAt: options.startedAt ?? 123456789,
outputPath: options.outPath,
scope: 'device',
@@ -198,6 +198,52 @@ test('Provider-backed integration Android record start retires completed evidenc
);
});
test('Provider-backed integration Android record start retires evidence stranded by a re-adopted emulator serial', async () => {
await withAndroidRecordingScenario(
'agent-device-provider-scenario-android-re-adopted-',
async (tmpDir) => {
const calls: string[][] = [];
const outputPath = path.join(tmpDir, 're-adopted.mp4');
const remotePath = '/sdcard/agent-device-recording-823456789.mp4';
const manifest = buildAndroidRecordingManifest({
outPath: path.join(tmpDir, 'abandoned.mp4'),
remotePath,
sessionName: 'parked',
deviceId: 'emulator-5588',
chunks: [{ index: 1, remotePath, remotePid: '4004', remoteStartTime: '3766' }],
});
const daemon = await createAndroidRecordingScenarioHarness({
androidAdbProvider: () =>
createAndroidRecordingProvider({ calls, manifests: [manifest], deadPids: ['4004'] }),
deviceInventoryProvider: async () => [PROVIDER_SCENARIO_ANDROID],
});
try {
const started = await daemon.callCommand('record', ['start', outputPath], {
platform: 'android',
serial: PROVIDER_SCENARIO_ANDROID.id,
recordingScope: 'device',
});
assert.equal(assertRpcOk<{ recording?: unknown }>(started).recording, 'started');
assert.ok(
calls.some((args) => args.join(' ') === `shell rm -f '${remotePath}'`),
'retired the stranded chunk artifact',
);
assert.ok(
calls.some((args) => args.join(' ') === ANDROID_MARKER_REMOVAL),
'retired the stranded marker',
);
assert.equal(
calls.some((args) => args[1]?.startsWith('kill ')),
false,
'a dead recorder is never signalled',
);
} finally {
await daemon.close();
}
},
);
});
test('Provider-backed integration Android record start retains completed evidence while a replacement recorder writes its path', async () => {
await withAndroidRecordingScenario(
'agent-device-provider-scenario-android-foreign-writer-',
@@ -236,7 +282,7 @@ test('Provider-backed integration Android record start retains completed evidenc
serial: PROVIDER_SCENARIO_ANDROID.id,
recordingScope: 'device',
}),
'UNKNOWN',
'DEVICE_IN_USE',
/another recorder is writing/,
);
assert.equal(
@@ -297,8 +343,8 @@ test('Provider-backed integration Android record start refuses completed evidenc
serial: PROVIDER_SCENARIO_ANDROID.id,
recordingScope: 'device',
}),
'UNKNOWN',
/cannot be safely retired/,
'DEVICE_IN_USE',
/its recorder is writing/,
);
assert.equal(
calls.some((args) => args[1]?.startsWith('kill ')),