mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
* fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) #1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe` and `gesture rotate`'s `velocity`, but shipped without the migration guide, the repository sweep, or the parse-time error that issue #1216's own checklist gates a removal on. The sweep finds what that left behind: both `06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe and would fail at replay, two tests still asserted the removed shapes, and two branches still read the retired positional. Argument arity for every public gesture syntax now lives in one table keyed off the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed from the CLI is removed from `.ad` in the same edit. Both callers read it: the CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is parsed — before the replay executes any device action — naming the line and computing its rewrite, instead of running up to that step and failing as a repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve after planning, and interpolation never splits a token, so the count is decidable while the values are not. Deleting the dead duration read in `readSwipeGeometry` would have left `replay export` emitting no duration, handing Maestro's 400ms default to a gesture the script runs at 100ms, so the export now states `duration: 100`. `.ad` positional gesture parsing is NOT removed. Its only remaining callers are the CLI argv parse and the `.ad` line parse, both the current public syntax rather than a bridge to an older one, so there is nothing to migrate off. ADR 0013 records that and drops the "compatibility" framing that made it read as debt. Both migrated fixtures verified on real devices with the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s. * fix(gestures): reject removed swipe input at the Node/MCP boundary Review findings on d88c6ed8. P1: `interactionDaemonWriters.swipe` hand-projects five fields, so a JavaScript caller's `durationMs` was dropped before the daemon's `readSwipeInput` could reject it and a default-duration fling ran instead — the exact silent reinterpretation the guide promises does not happen. `gesture` was already safe because its writer runs `readGestureInput` -> `readGesturePayload`, which rejects the removed keys; `swipe` was the one surface with no reader of its own. The rejection now lives in contracts and is shared by the client writer and the daemon handler, so there is one rule and one message. The SDK regression covers all four removed keys and asserts the transport is never reached; reverting the writer call fails it on `swipe durationMs`. P2: the preflight's retired-slot test required a numeric token, so `swipe 197 650 197 300 ${DURATION}` fell back to bare usage text. An unresolved `${VAR}` now counts as the retired slot and is carried into the pan rewrite, while a stray flag or word stays a plain usage error. P2: the removal shipped in 0.20.0, not 0.21 — removal commit6d99914f4is contained in tag v0.20.0. The guide said 0.21 because the CHANGELOG still files it under `Unreleased`; the tag is the truth (headings lag several releases repo-wide, so that is pre-existing and left alone). The `.ad` grep recipe now matches variable-backed durations too. * docs(gestures): make the migration sweep and MCP claim accurate Re-review findings on 328bad8e (both migration-guide accuracy). The saved-script sweep matched only the five-argument `swipe` form; it missed timed `gesture fling`, timed `gesture swipe`, and `gesture rotate ... velocity`, so the repository-cleanliness step was incomplete. Provide one grep per retired form (number-or-`${VAR}` token), each verified to flag the removed shape and skip the valid one. The MCP section claimed the structured rejection carries the CLI's concrete replacement command. It does not: `readGesturePayload` and `assertNoRemovedSwipeInput` return a message that names the removed key and the replacement command (e.g. `gesture fling does not accept durationMs; use gesture pan for timed movement`) but not the fully-substituted coordinate rewrite, because the structured request carries no positional string to rewrite. Describe what the structured path actually returns. * docs(gestures): make the .ad sweep parser-aligned Re-review finding on4fd5d39: the documented sweep required literal single spaces and unquoted numeric tokens, but the `.ad` tokenizer separates on any whitespace (`/\s/`, so tabs too) and accepts double-quoted tokens, so tab-separated or quoted-duration stale lines the parser rejects were missed. Rewrite the patterns to follow the tokenizer: `[[:space:]]+` between tokens and a numeric slot that accepts a bare or double-quoted number (optionally signed) or `${VAR}`. Requiring a digit in the numeric slot keeps a trailing flag like `--count` from being read as the retired positional. Verified against a fixture of tab/space/quoted/quoted-var/negative encodings that the four greps flag exactly the lines `parseReplayScriptDetailed` rejects and none it accepts, and that the live repo sweeps clean. Also state plainly what a regex cannot promise: the parser is the authoritative gate — every retired form is rejected at parse time before any device action, so running the suite finds every stale line by construction and a missed grep can never reach execution. The grep stays a bulk pre-flight, and the "every affected line" claim is scoped to that.
This commit is contained in:
Michał Pierzchała
committed by
GitHub
GitHub
parent
256887f194
commit
287cc18c29
@@ -13,6 +13,9 @@
|
||||
- `gesture swipe preset` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
|
||||
- `gesture rotate degrees [x] [y]` no longer accepts a trailing `velocity` positional; rotation pacing is derived from `degrees`.
|
||||
- MCP/Node schemas no longer advertise `velocity` or `durationMs` on `swipe`/`fling`/`gesture swipe`; `durationMs` remains on `gesture pan` and `gesture transform`.
|
||||
- A `.ad` script that still carries a removed positional now fails when the script is parsed, before the replay executes any device action, naming the line and its rewrite (for example `swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" ...`). Previously the script ran up to that step and then failed as a replay divergence.
|
||||
- Published the [gesture migration guide](https://agent-device.dev/docs/migrating-gestures) covering CLI, Node.js, MCP, and saved `.ad` recordings, plus the deprecation policy the next such removal follows (#1216).
|
||||
- `replay export` now writes an explicit `duration: 100` for `swipe` — the canonical fling duration — instead of omitting it and letting Maestro apply its own 400ms default. Maestro flows replayed by `agent-device` are unaffected; a timed Maestro `swipe` still normalizes to `gesture pan` with the `endpoint-hold` profile.
|
||||
- Breaking: the deprecated `rotate` CLI command alias has been removed. Use `orientation` instead; invoking `rotate` now fails with `rotate was renamed to orientation; for the two-finger gesture use: gesture rotate`.
|
||||
- Breaking (ADR 0014, session ref-frame lifetime): a mutation through an `@ref` now expires the session's ref frame, so a later ref mutation without a fresh observation fails closed with a typed `details.reason` (`ref_frame_expired`, `ref_generation_mismatch`, `plain_ref_requires_complete_frame`, or `ref_not_issued`) instead of acting on a possibly-navigated screen. A ref-oriented sequence that performs several mutations must re-`snapshot` between them, consume an honestly issued `--settle` ref in pinned `@eN~s<gen>` form, or use selectors. Enforcement applies on every platform, not just iOS. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must capture between mutations or use selectors.
|
||||
- Ref reads resolve against the authorized ref frame's source tree, so an internal read-only capture (including Android freshness) can no longer retarget an admitted `@ref` by positional coincidence. Read-only ref consumers keep the structured staleness warning while the frame retains the ref's evidence.
|
||||
|
||||
@@ -96,9 +96,25 @@ Platform adapters consume the canonical plan:
|
||||
The `Interactor` and backend expose one compositional `performGesture(plan)` primitive instead of a
|
||||
method per semantic alias. The old scalar Apple and Android multi-touch executors and the
|
||||
public-command alias-to-positionals-to-reparse route are deleted. `.ad` keeps its established
|
||||
positional syntax through one named replay compatibility codec; CLI, Node.js, and MCP send
|
||||
structured input. Providers should compose transport/device bindings with the shared platform
|
||||
adapter rather than reimplement the interaction runtime.
|
||||
positional syntax through one named codec; CLI, Node.js, and MCP send structured input. Providers
|
||||
should compose transport/device bindings with the shared platform adapter rather than reimplement
|
||||
the interaction runtime.
|
||||
|
||||
That `.ad` codec is the script format's own syntax, not a compatibility shim, and is not scheduled
|
||||
for removal (amended 2026-07, issue #1216). Its only remaining callers are the CLI argv parse and
|
||||
the `.ad` line parse — both the current public syntax — so there is nothing left to migrate off.
|
||||
A structured `.ad` gesture payload was rejected: it would make recordings unreadable and ungreppable
|
||||
for no behavioral gain.
|
||||
|
||||
Argument arity for both callers comes from one table (`PUBLIC_GESTURE_SYNTAX`), so a form removed
|
||||
from the CLI is removed from `.ad` in the same edit. A `.ad` script that still carries a removed
|
||||
positional fails when the script is parsed, before the replay executes any device action, with the
|
||||
offending line and its rewrite. Arity is the only thing checked at parse time: `${VAR}` tokens
|
||||
resolve after planning, and interpolation never splits a token, so the count is decidable while the
|
||||
values are not. The removal process these inputs follow — announce, warn for one minor release,
|
||||
publish the migration guide, prove the repository is clean, then remove the branch and its tests —
|
||||
is documented in the public
|
||||
[gesture migration guide](https://agent-device.dev/docs/migrating-gestures).
|
||||
|
||||
Repeated coordinate swipes are bounded at the public command contract and daemon trust boundary.
|
||||
Individual count and pause limits prevent pathological fields, while the combined planned gesture
|
||||
|
||||
@@ -673,6 +673,55 @@ test('interactions.rotateGesture rejects partial centers on the client side', as
|
||||
assert.equal(setup.calls.length, 0);
|
||||
});
|
||||
|
||||
test('removed gesture inputs are rejected client-side, never dropped from the projection', async () => {
|
||||
const setup = createTransport(async () => {
|
||||
throw new Error('transport should not run for invalid input');
|
||||
});
|
||||
const client = createAgentDeviceClient(setup.config, { transport: setup.transport });
|
||||
// A JavaScript caller (or a stale compiled build) can still reach these keys
|
||||
// past the removed compile-time fields. `swipe` has no structured reader of
|
||||
// its own, so before #1216 its writer silently dropped `durationMs` and ran a
|
||||
// default-duration fling instead of failing.
|
||||
const removed: Array<[string, () => Promise<unknown>, string]> = [
|
||||
[
|
||||
'swipe durationMs',
|
||||
() =>
|
||||
client.interactions.swipe({
|
||||
from: { x: 197, y: 650 },
|
||||
to: { x: 197, y: 300 },
|
||||
durationMs: 300,
|
||||
} as Parameters<typeof client.interactions.swipe>[0]),
|
||||
'swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
],
|
||||
[
|
||||
'fling durationMs',
|
||||
() =>
|
||||
client.interactions.fling({ direction: 'down', x: 100, y: 200, durationMs: 300 } as never),
|
||||
'gesture fling does not accept durationMs; use gesture pan for timed movement',
|
||||
],
|
||||
[
|
||||
'gesture swipe durationMs',
|
||||
() => client.interactions.swipeGesture({ preset: 'left', durationMs: 300 } as never),
|
||||
'gesture swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
],
|
||||
[
|
||||
'rotate velocity',
|
||||
() => client.interactions.rotateGesture({ degrees: 35, velocity: 800 } as never),
|
||||
'gesture rotate does not accept velocity; rotation pacing derives from degrees',
|
||||
],
|
||||
];
|
||||
|
||||
for (const [label, call, message] of removed) {
|
||||
await assert.rejects(
|
||||
call,
|
||||
(error: unknown) =>
|
||||
error instanceof AppError && error.code === 'INVALID_ARGS' && error.message === message,
|
||||
label,
|
||||
);
|
||||
}
|
||||
assert.equal(setup.calls.length, 0);
|
||||
});
|
||||
|
||||
test('interactions.pan projects one- and two-finger requests through typed gesture input', async () => {
|
||||
const setup = createTransport(async () => ({ ok: true, data: { message: 'Panned' } }));
|
||||
const client = createAgentDeviceClient(setup.config, { transport: setup.transport });
|
||||
|
||||
@@ -11,7 +11,10 @@ import {
|
||||
readInteractionTargetFromPositionals,
|
||||
} from '../../core/interaction-positionals.ts';
|
||||
import { AppError } from '../../kernel/errors.ts';
|
||||
import { swipePayloadFromPositionals } from '../../contracts/gesture-normalization.ts';
|
||||
import {
|
||||
assertNoRemovedSwipeInput,
|
||||
swipePayloadFromPositionals,
|
||||
} from '../../contracts/gesture-normalization.ts';
|
||||
import type { ScrollInputDirection } from './runtime/gestures.ts';
|
||||
import {
|
||||
commonInputFromFlags,
|
||||
@@ -124,14 +127,16 @@ export const interactionDaemonWriters = {
|
||||
longpress: direct(PUBLIC_COMMANDS.longPress, (input) =>
|
||||
longPressPositionals(input as LongPressOptions),
|
||||
),
|
||||
swipe: (input) =>
|
||||
request(PUBLIC_COMMANDS.swipe, [], input, {
|
||||
swipe: (input) => {
|
||||
assertNoRemovedSwipeInput(input);
|
||||
return request(PUBLIC_COMMANDS.swipe, [], input, {
|
||||
from: input.from,
|
||||
to: input.to,
|
||||
count: input.count,
|
||||
pauseMs: input.pauseMs,
|
||||
pattern: input.pattern,
|
||||
}),
|
||||
});
|
||||
},
|
||||
focus: direct(PUBLIC_COMMANDS.focus, (input) => [String(input.x), String(input.y)]),
|
||||
type: direct(PUBLIC_COMMANDS.type, (input) => typePositionals(input as TypeTextOptions)),
|
||||
fill: direct(PUBLIC_COMMANDS.fill, (input) => fillPositionals(input as FillOptions)),
|
||||
|
||||
@@ -43,7 +43,7 @@ screenshot "./artifacts/checkout"
|
||||
test('exports coordinate gestures and sleep waits with warnings', () => {
|
||||
const result = exportReplayScriptToMaestro(`open com.example.app
|
||||
click 120 240
|
||||
swipe 200 700 200 200 300 --count 2
|
||||
swipe 200 700 200 200 --count 2
|
||||
wait 500
|
||||
`);
|
||||
|
||||
@@ -52,8 +52,8 @@ wait 500
|
||||
[
|
||||
'launchApp',
|
||||
{ tapOn: { point: '120,240' } },
|
||||
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
|
||||
{ swipe: { start: '200,700', end: '200,200', duration: 300 } },
|
||||
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
|
||||
{ swipe: { start: '200,700', end: '200,200', duration: 100 } },
|
||||
{ waitForAnimationToEnd: { timeout: 500 } },
|
||||
],
|
||||
]);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { SessionAction } from '../../daemon/types.ts';
|
||||
import { GESTURE_FLING_DURATION_MS } from '../../contracts/gesture-plan.ts';
|
||||
import { parseSelectorChain, type Selector } from '../../selectors/index.ts';
|
||||
import type { SelectorTerm } from '../../selectors/parse.ts';
|
||||
import { AppError } from '../../kernel/errors.ts';
|
||||
@@ -39,7 +40,7 @@ type ActionConverter = (action: SessionAction) => ConvertedAction;
|
||||
type SwipeGeometry = {
|
||||
start: string;
|
||||
end: string;
|
||||
duration?: number;
|
||||
duration: number;
|
||||
};
|
||||
|
||||
const TEXT_SELECTOR_KEYS = new Set<string>(MAESTRO_TEXT_SELECTOR_KEYS);
|
||||
@@ -362,13 +363,18 @@ function convertSwipeAction(action: SessionAction): ConvertedAction {
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* `swipe` carries no duration positional, so the export states the canonical
|
||||
* fling duration rather than letting Maestro apply its own 400 ms default to a
|
||||
* gesture the script runs at 100 ms.
|
||||
*/
|
||||
function readSwipeGeometry(action: SessionAction): SwipeGeometry | undefined {
|
||||
const [x1, y1, x2, y2, duration] = action.positionals;
|
||||
const [x1, y1, x2, y2] = action.positionals;
|
||||
if (!isNumber(x1) || !isNumber(y1) || !isNumber(x2) || !isNumber(y2)) return undefined;
|
||||
return {
|
||||
start: formatMaestroPoint(x1, y1),
|
||||
end: formatMaestroPoint(x2, y2),
|
||||
...(duration && isNumber(duration) ? { duration: Number(duration) } : {}),
|
||||
duration: GESTURE_FLING_DURATION_MS,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { test } from 'vitest';
|
||||
import {
|
||||
describeReplayGestureArityError,
|
||||
gesturePayloadFromPositionals,
|
||||
gesturePayloadToPositionals,
|
||||
normalizePublicGesture,
|
||||
@@ -57,6 +58,86 @@ test('gesture recording codec round-trips fling with distance', () => {
|
||||
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
|
||||
});
|
||||
|
||||
test('a retired trailing positional reports its migration, not a bare usage line', () => {
|
||||
assert.throws(() => swipePayloadFromPositionals(['197', '650', '197', '300', '300']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message:
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2. The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
|
||||
});
|
||||
assert.throws(() => gesturePayloadFromPositionals(['fling', 'down', '100', '200', '80', '500']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: /trailing durationMs positional was removed: use "gesture fling down 100 200 80"/,
|
||||
});
|
||||
assert.throws(() => gesturePayloadFromPositionals(['swipe', 'left', '500']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: /trailing durationMs positional was removed: use "gesture swipe left"/,
|
||||
});
|
||||
assert.throws(() => gesturePayloadFromPositionals(['rotate', '35', '100', '200', '800']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: /trailing velocity positional was removed: use "gesture rotate 35 100 200"/,
|
||||
});
|
||||
});
|
||||
|
||||
test('a malformed line is a usage error rather than a migration claim', () => {
|
||||
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '5', '6']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
|
||||
});
|
||||
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', '--unknown']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
|
||||
});
|
||||
assert.throws(() => swipePayloadFromPositionals(['1', '2', '3', '4', 'hello']), {
|
||||
code: 'INVALID_ARGS',
|
||||
message: 'swipe accepts 4 arguments: x1 y1 x2 y2.',
|
||||
});
|
||||
});
|
||||
|
||||
test('the .ad preflight names the offending line and leaves live syntax alone', () => {
|
||||
assert.equal(
|
||||
describeReplayGestureArityError('swipe', ['197', '650', '197', '300', '300'], 'line 6'),
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
|
||||
);
|
||||
assert.equal(
|
||||
describeReplayGestureArityError('gesture', ['pan', '110', '443', '48', '0', '500'], 'line 2'),
|
||||
undefined,
|
||||
);
|
||||
assert.equal(describeReplayGestureArityError('click', ['120', '240'], 'line 2'), undefined);
|
||||
});
|
||||
|
||||
test('a variable-backed retired positional still gets the migration', () => {
|
||||
assert.equal(
|
||||
describeReplayGestureArityError('swipe', ['197', '650', '197', '300', '${DURATION}'], 'line 6'),
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 ${DURATION}" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
|
||||
);
|
||||
assert.equal(
|
||||
describeReplayGestureArityError(
|
||||
'swipe',
|
||||
['${X1}', '${Y1}', '${X2}', '${Y2}', '${DURATION}'],
|
||||
'line 6',
|
||||
),
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs positional was removed: use "gesture pan x1 y1 dx dy durationMs" for the same timed drag, or "swipe ${X1} ${Y1} ${X2} ${Y2}" for a default-duration swipe.',
|
||||
);
|
||||
assert.match(
|
||||
describeReplayGestureArityError('gesture', ['rotate', '35', '195', '443', '${V}'], 'line 4') ??
|
||||
'',
|
||||
/trailing velocity positional was removed: use "gesture rotate 35 195 443"/,
|
||||
);
|
||||
});
|
||||
|
||||
test('the .ad preflight defers to dispatch for values it cannot know yet', () => {
|
||||
// `${VAR}` tokens resolve after planning, and interpolation never splits a
|
||||
// token, so only the argument count is decidable at parse time.
|
||||
assert.equal(
|
||||
describeReplayGestureArityError('swipe', ['${X1}', '${Y1}', '${X2}', '${Y2}'], 'line 3'),
|
||||
undefined,
|
||||
);
|
||||
assert.equal(
|
||||
describeReplayGestureArityError('gesture', ['${KIND}', '1', '2', '3'], 'line 4'),
|
||||
undefined,
|
||||
);
|
||||
});
|
||||
|
||||
test('pinch and rotate syntax rejects a partial origin', () => {
|
||||
assert.throws(() => gesturePayloadFromPositionals(['pinch', '1.5', '100']), {
|
||||
code: 'INVALID_ARGS',
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import type { Point } from '../kernel/snapshot.ts';
|
||||
import { AppError } from '../kernel/errors.ts';
|
||||
import { readGesturePayload, type GesturePayload } from './gesture-input.ts';
|
||||
import { readGesturePayload, type GESTURE_KINDS, type GesturePayload } from './gesture-input.ts';
|
||||
import type { GestureSemanticInput } from './gesture-plan-types.ts';
|
||||
|
||||
export type NormalizedPublicGesture = {
|
||||
@@ -15,14 +15,174 @@ export type SwipePayload = {
|
||||
pattern?: 'one-way' | 'ping-pong';
|
||||
};
|
||||
|
||||
function assertPositionalCount(
|
||||
args: readonly string[],
|
||||
max: number,
|
||||
usageMessage: string,
|
||||
): asserts args is { length: number } & typeof args {
|
||||
if (args.length > max) {
|
||||
throw new AppError('INVALID_ARGS', usageMessage);
|
||||
/** Derived from the canonical kinds, so a new gesture kind cannot skip the arity table. */
|
||||
type GestureSyntaxKey = 'swipe' | `gesture ${(typeof GESTURE_KINDS)[number]}`;
|
||||
|
||||
type PublicGestureSyntax = {
|
||||
/** Highest accepted positional count, flags excluded. */
|
||||
max: number;
|
||||
usage: string;
|
||||
/**
|
||||
* The trailing positional this syntax used to accept and no longer does. One
|
||||
* extra numeric argument reports its migration instead of a bare usage line,
|
||||
* which is what a saved recording or hand-written `.ad` from before the
|
||||
* removal actually carries.
|
||||
*/
|
||||
retired?: {
|
||||
positional: string;
|
||||
migrate: (args: readonly string[], canonical: string) => string;
|
||||
};
|
||||
};
|
||||
|
||||
/**
|
||||
* The positional arity of every public gesture syntax, in one table so the CLI
|
||||
* argv parse and the `.ad` script preflight reject the same shapes and report
|
||||
* the same migration.
|
||||
*/
|
||||
const PUBLIC_GESTURE_SYNTAX: Record<GestureSyntaxKey, PublicGestureSyntax> = {
|
||||
swipe: {
|
||||
max: 4,
|
||||
usage: 'swipe accepts 4 arguments: x1 y1 x2 y2',
|
||||
retired: { positional: 'durationMs', migrate: migrateRetiredSwipeDuration },
|
||||
},
|
||||
'gesture pan': {
|
||||
max: 5,
|
||||
usage: 'gesture pan accepts at most 5 arguments: x y dx dy [durationMs]',
|
||||
},
|
||||
'gesture fling': {
|
||||
max: 4,
|
||||
usage: 'gesture fling accepts at most 4 arguments: direction x y [distance]',
|
||||
retired: {
|
||||
positional: 'durationMs',
|
||||
migrate: (_args, canonical) => `use "${canonical}", or gesture pan for timed movement.`,
|
||||
},
|
||||
},
|
||||
'gesture swipe': {
|
||||
max: 1,
|
||||
usage: 'gesture swipe accepts 1 argument: preset',
|
||||
retired: {
|
||||
positional: 'durationMs',
|
||||
migrate: (_args, canonical) => `use "${canonical}", or gesture pan for timed movement.`,
|
||||
},
|
||||
},
|
||||
'gesture pinch': {
|
||||
max: 3,
|
||||
usage: 'gesture pinch accepts at most 3 arguments: scale [x] [y]',
|
||||
},
|
||||
'gesture rotate': {
|
||||
max: 3,
|
||||
usage: 'gesture rotate accepts at most 3 arguments: degrees [x] [y]',
|
||||
retired: {
|
||||
positional: 'velocity',
|
||||
migrate: (_args, canonical) => `use "${canonical}"; rotation pacing derives from degrees.`,
|
||||
},
|
||||
},
|
||||
'gesture transform': {
|
||||
max: 7,
|
||||
usage: 'gesture transform accepts at most 7 arguments: x y dx dy scale degrees [durationMs]',
|
||||
},
|
||||
};
|
||||
|
||||
/** `swipe x1 y1 x2 y2 durationMs` translates to the equivalent timed pan. */
|
||||
function migrateRetiredSwipeDuration(args: readonly string[], canonical: string): string {
|
||||
const [x1, y1, x2, y2, durationMs] = args;
|
||||
const dx = coordinateDelta(x1, x2);
|
||||
const dy = coordinateDelta(y1, y2);
|
||||
if (dx === undefined || dy === undefined) {
|
||||
return `use "gesture pan x1 y1 dx dy durationMs" for the same timed drag, or "${canonical}" for a default-duration swipe.`;
|
||||
}
|
||||
return `use "gesture pan ${x1} ${y1} ${dx} ${dy} ${durationMs}" for the same timed drag, or "${canonical}" for a default-duration swipe.`;
|
||||
}
|
||||
|
||||
function coordinateDelta(from: string | undefined, to: string | undefined): number | undefined {
|
||||
const delta = Number(to) - Number(from);
|
||||
return Number.isFinite(delta) ? delta : undefined;
|
||||
}
|
||||
|
||||
/** A rejected argument count, split so a caller can name where the line came from. */
|
||||
type GestureArityError = { usage: string; migration?: string };
|
||||
|
||||
/**
|
||||
* The arity error for one public gesture syntax, or `undefined` when the
|
||||
* argument count is accepted. Only the count is inspected: a `.ad` line can
|
||||
* still hold unresolved `${VAR}` tokens, and interpolation never splits a token.
|
||||
*/
|
||||
function describeGestureArityError(
|
||||
key: GestureSyntaxKey,
|
||||
args: readonly string[],
|
||||
): GestureArityError | undefined {
|
||||
const syntax = PUBLIC_GESTURE_SYNTAX[key];
|
||||
if (args.length <= syntax.max) return undefined;
|
||||
const retired = syntax.retired;
|
||||
if (!retired || args.length !== syntax.max + 1 || !isRetiredSlotArgument(args[syntax.max])) {
|
||||
return { usage: syntax.usage };
|
||||
}
|
||||
const canonical = `${key} ${args.slice(0, syntax.max).join(' ')}`;
|
||||
return {
|
||||
usage: syntax.usage,
|
||||
migration: `The trailing ${retired.positional} positional was removed: ${retired.migrate(args, canonical)}`,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether one extra argument occupies the retired slot. A retired positional was
|
||||
* always a number, and a `.ad` script may hold it as an unresolved `${VAR}`, so
|
||||
* both report the migration while a stray flag or word stays a usage error.
|
||||
*/
|
||||
function isRetiredSlotArgument(value: string | undefined): boolean {
|
||||
if (value === undefined || value.trim().length === 0) return false;
|
||||
return value.startsWith('${') || Number.isFinite(Number(value));
|
||||
}
|
||||
|
||||
/**
|
||||
* `swipe` is the one public gesture surface whose structured input has no reader
|
||||
* of its own — the daemon writer projects known fields — so a removed key would
|
||||
* be dropped before the daemon could reject it and a default-duration fling
|
||||
* would run instead. Rejecting here covers the Node and MCP boundaries at the
|
||||
* same point `readGesturePayload` rejects the gesture kinds' removed keys.
|
||||
*/
|
||||
export function assertNoRemovedSwipeInput(input: unknown): void {
|
||||
if (!input || typeof input !== 'object' || Array.isArray(input)) return;
|
||||
if ((input as Record<string, unknown>).durationMs === undefined) return;
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
);
|
||||
}
|
||||
|
||||
function formatGestureArityError(error: GestureArityError, source?: string): string {
|
||||
const usage = source ? `${error.usage} (${source})` : error.usage;
|
||||
return error.migration ? `${usage}. ${error.migration}` : `${usage}.`;
|
||||
}
|
||||
|
||||
function assertGestureArity(key: GestureSyntaxKey, args: readonly string[]): void {
|
||||
const error = describeGestureArityError(key, args);
|
||||
if (error) throw new AppError('INVALID_ARGS', formatGestureArityError(error));
|
||||
}
|
||||
|
||||
/**
|
||||
* The `.ad` preflight for one authored or recorded gesture line, so a script
|
||||
* written before the timed-gesture removal reports its migration before the
|
||||
* replay executes any device action. `source` names the offending line.
|
||||
*/
|
||||
export function describeReplayGestureArityError(
|
||||
command: string,
|
||||
positionals: readonly string[],
|
||||
source: string,
|
||||
): string | undefined {
|
||||
const error = readReplayGestureArityError(command, positionals);
|
||||
return error ? formatGestureArityError(error, source) : undefined;
|
||||
}
|
||||
|
||||
function readReplayGestureArityError(
|
||||
command: string,
|
||||
positionals: readonly string[],
|
||||
): GestureArityError | undefined {
|
||||
if (command === 'swipe') return describeGestureArityError('swipe', positionals);
|
||||
if (command !== 'gesture') return undefined;
|
||||
const key = `gesture ${positionals[0]}`;
|
||||
if (!Object.hasOwn(PUBLIC_GESTURE_SYNTAX, key)) return undefined;
|
||||
return describeGestureArityError(key as GestureSyntaxKey, positionals.slice(1));
|
||||
}
|
||||
|
||||
function readOriginDelta(args: readonly string[]): { origin: Point; delta: Point } {
|
||||
@@ -42,11 +202,7 @@ export function gesturePayloadFromPositionals(
|
||||
const args = positionals.slice(1);
|
||||
switch (kind) {
|
||||
case 'pan': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
5,
|
||||
'gesture pan accepts at most 5 arguments: x y dx dy [durationMs]',
|
||||
);
|
||||
assertGestureArity('gesture pan', args);
|
||||
const { origin, delta } = readOriginDelta(args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
@@ -57,11 +213,7 @@ export function gesturePayloadFromPositionals(
|
||||
});
|
||||
}
|
||||
case 'fling': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
4,
|
||||
'gesture fling accepts at most 4 arguments: direction x y [distance]; for timed movement use gesture pan',
|
||||
);
|
||||
assertGestureArity('gesture fling', args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
direction: args[0],
|
||||
@@ -70,18 +222,14 @@ export function gesturePayloadFromPositionals(
|
||||
});
|
||||
}
|
||||
case 'swipe': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
1,
|
||||
'gesture swipe accepts 1 argument: preset; for timed movement use gesture pan',
|
||||
);
|
||||
assertGestureArity('gesture swipe', args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
preset: args[0],
|
||||
});
|
||||
}
|
||||
case 'pinch': {
|
||||
assertPositionalCount(args, 3, 'gesture pinch accepts at most 3 arguments: scale [x] [y]');
|
||||
assertGestureArity('gesture pinch', args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
scale: Number(args[0]),
|
||||
@@ -89,7 +237,7 @@ export function gesturePayloadFromPositionals(
|
||||
});
|
||||
}
|
||||
case 'rotate': {
|
||||
assertPositionalCount(args, 3, 'gesture rotate accepts at most 3 arguments: degrees [x] [y]');
|
||||
assertGestureArity('gesture rotate', args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
degrees: Number(args[0]),
|
||||
@@ -97,11 +245,7 @@ export function gesturePayloadFromPositionals(
|
||||
});
|
||||
}
|
||||
case 'transform': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
7,
|
||||
'gesture transform accepts at most 7 arguments: x y dx dy scale degrees [durationMs]',
|
||||
);
|
||||
assertGestureArity('gesture transform', args);
|
||||
const { origin, delta } = readOriginDelta(args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
@@ -158,12 +302,7 @@ export function swipePayloadFromPositionals(
|
||||
positionals: string[],
|
||||
options: Omit<SwipePayload, 'from' | 'to'> = {},
|
||||
): SwipePayload {
|
||||
if (positionals.length > 4) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2; for timed movement use gesture pan',
|
||||
);
|
||||
}
|
||||
assertGestureArity('swipe', positionals);
|
||||
return {
|
||||
from: { x: Number(positionals[0]), y: Number(positionals[1]) },
|
||||
to: { x: Number(positionals[2]), y: Number(positionals[3]) },
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { readGesturePayload, type GesturePayload } from '../../contracts/gesture-input.ts';
|
||||
import {
|
||||
assertNoRemovedSwipeInput,
|
||||
gesturePayloadToPositionals,
|
||||
normalizePublicGesture,
|
||||
normalizePublicSwipeMotion,
|
||||
@@ -189,13 +190,8 @@ function readSwipeInput(input: unknown): SwipePayload {
|
||||
if (!input || typeof input !== 'object' || Array.isArray(input)) {
|
||||
throw new AppError('INVALID_ARGS', 'swipe requires structured object input');
|
||||
}
|
||||
assertNoRemovedSwipeInput(input);
|
||||
const record = input as Record<string, unknown>;
|
||||
if (record.durationMs !== undefined) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
);
|
||||
}
|
||||
const pattern = record.pattern;
|
||||
if (pattern !== undefined && pattern !== 'one-way' && pattern !== 'ping-pong') {
|
||||
throw new AppError('INVALID_ARGS', 'swipe pattern must be one-way or ping-pong');
|
||||
|
||||
@@ -298,11 +298,7 @@ function buildSwipeEvents(
|
||||
|
||||
const durationMs = resolveRecordingDurationMs(
|
||||
gestureDurationMs,
|
||||
[
|
||||
readRecordingNumber(result.effectiveDurationMs),
|
||||
readRecordingNumber(result.durationMs),
|
||||
readRecordingNumber(positionals[4]),
|
||||
],
|
||||
[readRecordingNumber(result.effectiveDurationMs), readRecordingNumber(result.durationMs)],
|
||||
DEFAULT_SWIPE_DURATION_MS,
|
||||
);
|
||||
const count = clampInt(readRecordingNumber(result.count), 1) ?? 1;
|
||||
|
||||
@@ -133,13 +133,43 @@ test('snapshot replay script writes interactive refresh flags', () => {
|
||||
assert.match(script, /snapshot -i -d 2 -s @e1/);
|
||||
});
|
||||
|
||||
test('a pre-removal gesture line fails the whole script instead of step N', () => {
|
||||
assert.throws(
|
||||
() =>
|
||||
parseReplayScriptDetailed(
|
||||
[
|
||||
'open com.apple.Preferences --relaunch',
|
||||
'',
|
||||
'swipe 197 650 197 300 300',
|
||||
'wait 500',
|
||||
'',
|
||||
].join('\n'),
|
||||
),
|
||||
{
|
||||
code: 'INVALID_ARGS',
|
||||
message:
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2 (line 3). The trailing durationMs positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300" for a default-duration swipe.',
|
||||
},
|
||||
);
|
||||
assert.throws(
|
||||
() =>
|
||||
parseReplayScriptDetailed(
|
||||
['open com.example.app', 'gesture rotate 35 195 443 800'].join('\n'),
|
||||
),
|
||||
{
|
||||
code: 'INVALID_ARGS',
|
||||
message: /gesture rotate accepts at most 3 arguments: degrees \[x\] \[y\] \(line 2\)\./,
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('gesture replay script parses pan, fling, swipe, pinch, and rotate gesture commands', () => {
|
||||
const parsed = parseReplayScriptDetailed(
|
||||
[
|
||||
'gesture pan 195 443 80 0 --pointer-count 2',
|
||||
'wait "pan changed yes" 5000',
|
||||
'gesture fling right 195 443 180',
|
||||
'gesture swipe right-edge 300',
|
||||
'gesture swipe right-edge',
|
||||
'gesture pinch 1.25 195 443',
|
||||
'gesture rotate 35 195 443',
|
||||
'',
|
||||
@@ -153,7 +183,7 @@ test('gesture replay script parses pan, fling, swipe, pinch, and rotate gesture
|
||||
assert.deepEqual(parsed[0]?.positionals, ['pan', '195', '443', '80', '0']);
|
||||
assert.equal(parsed[0]?.flags.pointerCount, 2);
|
||||
assert.deepEqual(parsed[2]?.positionals, ['fling', 'right', '195', '443', '180']);
|
||||
assert.deepEqual(parsed[3]?.positionals, ['swipe', 'right-edge', '300']);
|
||||
assert.deepEqual(parsed[3]?.positionals, ['swipe', 'right-edge']);
|
||||
assert.deepEqual(parsed[4]?.positionals, ['pinch', '1.25', '195', '443']);
|
||||
assert.deepEqual(parsed[5]?.positionals, ['rotate', '35', '195', '443']);
|
||||
});
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { AppError } from '../kernel/errors.ts';
|
||||
import { recordingQualityInputToExportQuality } from '../core/recording-export-quality.ts';
|
||||
import { describeReplayGestureArityError } from '../contracts/gesture-normalization.ts';
|
||||
import { readScreenshotScriptFlag } from '../contracts/screenshot.ts';
|
||||
import type { DeviceTarget, PlatformSelector } from '../kernel/device.ts';
|
||||
import { PLATFORM_SELECTORS } from '../kernel/device.ts';
|
||||
@@ -99,6 +100,12 @@ export function parseReplayScriptDetailed(script: string): ParsedReplayScript {
|
||||
if (pending) rejectUnbound(pending, index, 'did not parse as an action');
|
||||
continue;
|
||||
}
|
||||
const gestureArityError = describeReplayGestureArityError(
|
||||
parsed.command,
|
||||
parsed.positionals ?? [],
|
||||
`line ${index + 1}`,
|
||||
);
|
||||
if (gestureArityError) throw new AppError('INVALID_ARGS', gestureArityError);
|
||||
if (pending) {
|
||||
parsed.targetEvidence = pending.evidence;
|
||||
pending = undefined;
|
||||
|
||||
@@ -5,11 +5,11 @@ open settings --relaunch
|
||||
snapshot -i
|
||||
find text "Search" exists
|
||||
|
||||
swipe 206 700 206 250 300
|
||||
swipe 206 700 206 250
|
||||
wait 500
|
||||
snapshot -i
|
||||
|
||||
swipe 206 650 206 300 300 --count 2 --pause-ms 200 --pattern one-way
|
||||
swipe 206 650 206 300 --count 2 --pause-ms 200 --pattern one-way
|
||||
wait 500
|
||||
snapshot -i
|
||||
|
||||
|
||||
@@ -3,14 +3,14 @@ context platform=ios target=mobile
|
||||
|
||||
open com.apple.Preferences --relaunch
|
||||
|
||||
swipe 197 650 197 300 300
|
||||
swipe 197 650 197 300
|
||||
wait 500
|
||||
|
||||
swipe 197 600 197 350 300
|
||||
swipe 197 600 197 350
|
||||
wait 500
|
||||
|
||||
snapshot -i
|
||||
|
||||
swipe 197 300 197 650 300
|
||||
swipe 197 300 197 650
|
||||
wait 500
|
||||
find text "General" exists
|
||||
|
||||
@@ -88,5 +88,10 @@
|
||||
"name": "known-limitations",
|
||||
"type": "file",
|
||||
"label": "Known Limitations"
|
||||
},
|
||||
{
|
||||
"name": "migrating-gestures",
|
||||
"type": "file",
|
||||
"label": "Migrating Gestures"
|
||||
}
|
||||
]
|
||||
|
||||
@@ -373,6 +373,8 @@ Android text entry is owned by `agent-device`: provider-native injection when av
|
||||
`click --button secondary` is the desktop context-menu flow on macOS.
|
||||
`click --button middle` is reserved for future runner support and currently returns an explicit unsupported-operation error on macOS.
|
||||
`swipe` is a quick, fixed-duration directional throw. Use `gesture pan` for deliberate timed movement.
|
||||
Neither `swipe` nor `gesture fling` takes a duration, and `gesture rotate` takes no velocity — see
|
||||
[Migrating Gestures](/docs/migrating-gestures) if you have scripts or recordings that still pass one.
|
||||
Repeated coordinate swipes accept at most 200 repetitions and 10000ms pauses, and their combined
|
||||
gesture/pause schedule must fit within 60000ms.
|
||||
`gesture pan` accepts `x y dx dy [durationMs]` for deliberate drags. It uses one pointer by default. Add `--pointer-count 2` for a parallel two-finger pan with constant contact span and angle; this shares the bounded two-contact synthesizer used by transform while retaining pan intent. Android preserves the requested travel duration; iOS uses XCTest drag primitives for one-pointer pan and private XCTest synthesis for two-pointer pan.
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
---
|
||||
title: Migrating Gestures
|
||||
---
|
||||
|
||||
# Migrating Gestures
|
||||
|
||||
`agent-device` 0.20.0 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe`, and
|
||||
the `velocity` argument of `gesture rotate`. Nothing is silently reinterpreted: every removed form
|
||||
now fails with an `INVALID_ARGS` error that names its replacement.
|
||||
|
||||
This page is the migration for all four public surfaces — CLI, Node.js, MCP, and saved `.ad`
|
||||
recordings — plus the policy that governs the next such removal.
|
||||
|
||||
## What changed
|
||||
|
||||
Gesture vocabulary now separates a *throw* from a *deliberate drag*:
|
||||
|
||||
- `swipe` and `gesture fling` are quick, fixed-duration directional throws. They do not take a
|
||||
duration, because a caller-supplied duration made them something else.
|
||||
- `gesture pan` is the deliberate timed translation. It keeps `durationMs`.
|
||||
- `gesture rotate` derives its pacing from the requested `degrees`.
|
||||
|
||||
`durationMs` remains on `gesture pan` and `gesture transform`.
|
||||
|
||||
## CLI
|
||||
|
||||
| Removed | Use instead |
|
||||
| -------------------------------------------- | ----------------------------------------------- |
|
||||
| `swipe x1 y1 x2 y2 durationMs` | `gesture pan x1 y1 dx dy durationMs` |
|
||||
| `gesture fling <direction> x y distance durationMs` | `gesture fling <direction> x y distance` |
|
||||
| `gesture swipe <preset> durationMs` | `gesture swipe <preset>` |
|
||||
| `gesture rotate degrees x y velocity` | `gesture rotate degrees x y` |
|
||||
|
||||
`gesture pan` takes an origin plus a **delta**, where `swipe` took two absolute points, so
|
||||
`dx = x2 - x1` and `dy = y2 - y1`:
|
||||
|
||||
```bash
|
||||
# before
|
||||
agent-device swipe 197 650 197 300 300
|
||||
# after — same motion, same 300ms
|
||||
agent-device gesture pan 197 650 0 -350 300
|
||||
```
|
||||
|
||||
Drop the duration instead when the timing was incidental and a throw is what you wanted. The
|
||||
resulting gesture is a 100ms fling, which travels further on a scrollable list than a 300ms drag
|
||||
over the same distance:
|
||||
|
||||
```bash
|
||||
agent-device swipe 197 650 197 300
|
||||
```
|
||||
|
||||
The error message computes this for you:
|
||||
|
||||
```
|
||||
swipe accepts 4 arguments: x1 y1 x2 y2. The trailing durationMs positional was removed:
|
||||
use "gesture pan 197 650 0 -350 300" for the same timed drag, or "swipe 197 650 197 300"
|
||||
for a default-duration swipe.
|
||||
```
|
||||
|
||||
## Node.js
|
||||
|
||||
`interactions.swipe`, `interactions.fling`, and `interactions.swipeGesture` no longer accept
|
||||
`durationMs`; `interactions.rotateGesture` no longer accepts `velocity`. Passing them is a type
|
||||
error at compile time and an `INVALID_ARGS` rejection at runtime — the rejection happens
|
||||
client-side, before the request reaches the daemon, so a plain JavaScript caller or a stale
|
||||
compiled build gets the error rather than a silently retimed gesture.
|
||||
|
||||
```ts
|
||||
// before
|
||||
await device.interactions.swipe({
|
||||
from: { x: 197, y: 650 },
|
||||
to: { x: 197, y: 300 },
|
||||
durationMs: 300,
|
||||
});
|
||||
|
||||
// after — same motion, same 300ms
|
||||
await device.interactions.pan({ x: 197, y: 650, dx: 0, dy: -350, durationMs: 300 });
|
||||
|
||||
// after — a default-duration throw
|
||||
await device.interactions.swipe({ from: { x: 197, y: 650 }, to: { x: 197, y: 300 } });
|
||||
```
|
||||
|
||||
`interactions.pan` and `interactions.transformGesture` keep `durationMs`.
|
||||
|
||||
## MCP
|
||||
|
||||
The `swipe`, `gesture` (`fling`, `swipe` kinds), and `gesture rotate` tool schemas no longer
|
||||
advertise `durationMs` or `velocity`, so an agent reading the schema will not produce the removed
|
||||
form. An agent that sends one anyway — from a cached schema or a memorized example — gets an
|
||||
`INVALID_ARGS` rejection that names the removed key and the command to use instead, for example
|
||||
`gesture fling does not accept durationMs; use gesture pan for timed movement`. This is the
|
||||
structured-input equivalent of the CLI error; it names the replacement command but, unlike the CLI
|
||||
and `.ad` messages, does not echo a fully-substituted rewrite, because the structured request
|
||||
carries no positional string to rewrite.
|
||||
|
||||
No MCP server configuration changes.
|
||||
|
||||
## Saved `.ad` recordings
|
||||
|
||||
`.ad` scripts keep their positional syntax: it is the same vocabulary as the CLI, and it is not
|
||||
scheduled for removal (see [Positional `.ad` syntax](#positional-ad-syntax) below). Only the removed
|
||||
arguments have to go.
|
||||
|
||||
A script that still carries one fails **when it is parsed**, before the replay executes any device
|
||||
action, and the error names the line:
|
||||
|
||||
```
|
||||
Error (INVALID_ARGS): swipe accepts 4 arguments: x1 y1 x2 y2 (line 6). The trailing durationMs
|
||||
positional was removed: use "gesture pan 197 650 0 -350 300" for the same timed drag, or
|
||||
"swipe 197 650 197 300" for a default-duration swipe.
|
||||
```
|
||||
|
||||
Apply the same rewrites as the CLI table above. Flags are unaffected: `--count`, `--pause-ms`, and
|
||||
`--pattern` stay on `swipe`, and `--pointer-count` stays on `gesture pan`.
|
||||
|
||||
```
|
||||
# before
|
||||
swipe 206 650 206 300 300 --count 2 --pause-ms 200 --pattern one-way
|
||||
# after
|
||||
swipe 206 650 206 300 --count 2 --pause-ms 200 --pattern one-way
|
||||
```
|
||||
|
||||
A duration held in a variable (`swipe 197 650 197 300 ${DURATION}`) is reported the same way — the
|
||||
preflight counts arguments, so it does not need the value.
|
||||
|
||||
**The authoritative check is the parser itself.** Every retired form is rejected when the script is
|
||||
parsed, before the replay runs any device action, so running the suite (`agent-device test <glob>`,
|
||||
or `agent-device replay <file>.ad`) finds every stale line by construction and names it — a missed
|
||||
grep can never let one reach execution. The patterns below are a bulk pre-flight to locate them
|
||||
without a device. They follow the `.ad` tokenizer: tokens are separated by any whitespace (space or
|
||||
tab), and the numeric slots accept a bare or double-quoted number or `${VAR}` — so
|
||||
`swipe\t…\t"300"` is flagged like `swipe … 300`. They still stop short of the tokenizer's rarer
|
||||
encodings (a quoted command word, backslash escapes inside a quoted token), which is why the parser,
|
||||
not the grep, is the gate.
|
||||
|
||||
```bash
|
||||
# a numeric slot: bare/quoted number (optionally signed) or ${VAR}. Requiring a
|
||||
# digit keeps a following flag like --count from being read as the retired slot.
|
||||
num='("-?[0-9][0-9.]*"|"\$\{[^}]*\}"|-?[0-9][0-9.]*|\$\{[^}]*\})'
|
||||
# swipe x1 y1 x2 y2 durationMs
|
||||
grep -rnE "\\bswipe([[:space:]]+$num){5}" --include='*.ad' .
|
||||
# gesture fling <direction> x y distance durationMs
|
||||
grep -rnE "\\bgesture[[:space:]]+fling[[:space:]]+\"?[a-z]+\"?([[:space:]]+$num){4}" --include='*.ad' .
|
||||
# gesture swipe <preset> durationMs
|
||||
grep -rnE "\\bgesture[[:space:]]+swipe[[:space:]]+\"?[a-z-]+\"?[[:space:]]+$num" --include='*.ad' .
|
||||
# gesture rotate degrees x y velocity
|
||||
grep -rnE "\\bgesture[[:space:]]+rotate([[:space:]]+$num){4}" --include='*.ad' .
|
||||
```
|
||||
|
||||
Re-recording also produces a migrated script: the recorder writes the canonical form, so a fresh
|
||||
`open` → interact → `close --save-script` run is a valid alternative to editing by hand.
|
||||
|
||||
### Maestro flows
|
||||
|
||||
Maestro `swipe` with a `duration` is **not** affected. `agent-device` runs Maestro YAML through its
|
||||
own compatibility engine, which normalizes a timed Maestro swipe to the canonical `gesture pan`
|
||||
input and preserves Maestro's fast-swipe-then-hold execution profile. Maestro flows need no
|
||||
migration.
|
||||
|
||||
`replay export` writes an explicit `duration: 100` — the canonical fling duration — so an exported
|
||||
flow runs at the speed the `.ad` script ran, rather than picking up Maestro's own 400ms default.
|
||||
|
||||
## Deprecation policy
|
||||
|
||||
This is the process a public gesture input follows on its way out, and the bar the next removal has
|
||||
to clear:
|
||||
|
||||
1. **Announce.** The input is documented as deprecated and recorded in `CHANGELOG.md` under
|
||||
`Unreleased`, together with the replacement.
|
||||
2. **Warn for one minor release.** The input keeps working and normalizes to the replacement, with a
|
||||
`deprecations` entry in the response so an agent sees the migration while the call still
|
||||
succeeds.
|
||||
3. **Publish the migration.** A section on this page covers CLI, Node.js, MCP, and `.ad`, with a
|
||||
concrete before/after per surface.
|
||||
4. **Prove the repository is clean.** A repository-wide search for the removed shape returns no hits
|
||||
in fixtures, examples, skills, docs, or tests. `agent-device` has no usage telemetry, so this
|
||||
search — plus the migration guide's publication — is the evidence, and it is the reason the
|
||||
removal cannot be inferred from "nobody complained".
|
||||
5. **Remove.** The input is rejected with an `INVALID_ARGS` error that states the replacement, and
|
||||
the normalization branch and its compatibility tests are deleted in the same change. For an input
|
||||
that can appear in a saved recording, the rejection must fire at parse time and name the line, so
|
||||
a stale script never half-executes.
|
||||
|
||||
The 0.20.0 removal completed all five steps.
|
||||
|
||||
## Positional `.ad` syntax
|
||||
|
||||
Positional gesture parsing in `.ad` is **not** a compatibility shim and is not scheduled for
|
||||
removal.
|
||||
|
||||
`.ad` is a line-based script format whose syntax is the CLI's syntax. Its gesture codec —
|
||||
`gesturePayloadFromPositionals` / `gesturePayloadToPositionals` — is what defines the file format,
|
||||
not a bridge to an older one. CLI, Node.js, and MCP already send structured input directly; the
|
||||
codec's only remaining jobs are parsing CLI argv and reading and writing `.ad` lines, and both are
|
||||
the current public syntax. Replacing it with a structured payload would make recordings
|
||||
unreadable and ungreppable for no behavioral gain.
|
||||
|
||||
See [ADR 0013](https://github.com/callstack/agent-device/blob/main/docs/adr/0013-unified-gesture-plans.md)
|
||||
for the gesture normalization and planning model this rests on.
|
||||
Reference in New Issue
Block a user