feat!: remove deprecated gesture duration and rotate velocity inputs (#1218, #1216) (#1315)

* feat!: remove deprecated gesture duration and rotate velocity inputs (#1218, #1216)

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fix: address CI failures - remove dead export, dedupe positional validation, migrate linux-desktop swipe test to pan

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* fixup! preserve Maestro swipe endpoint-hold execution profile via internal seam

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* docs(adr): describe Maestro endpoint-hold internal seam in ADR 0013/0015

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

* feat: surface Maestro swipe executionProfile in replay trace and assert endpoint-hold in differential

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>

---------

Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
devin-ai-integration[bot]
2026-07-17 12:29:41 +02:00
committed by GitHub
parent 3fed8acda5
commit 6d99914f49
41 changed files with 494 additions and 370 deletions
+7
View File
@@ -2,6 +2,13 @@
## Unreleased
- Breaking: removed deprecated gesture duration and rotate velocity inputs (#1218).
- `swipe x1 y1 x2 y2` no longer accepts a trailing `durationMs` positional; use `gesture pan x1 y1 (x2-x1) (y2-y1) durationMs` for deliberate timed drags.
- Maestro `swipe` operations with a duration continue to normalize to `gesture pan` with the `endpoint-hold` execution profile, preserving the Maestro-compatible fast-swipe-then-hold behavior on iOS.
- `gesture fling direction x y` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
- `gesture swipe preset` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
- `gesture rotate degrees [x] [y]` no longer accepts a trailing `velocity` positional; rotation pacing is derived from `degrees`.
- MCP/Node schemas no longer advertise `velocity` or `durationMs` on `swipe`/`fling`/`gesture swipe`; `durationMs` remains on `gesture pan` and `gesture transform`.
- Breaking: the deprecated `rotate` CLI command alias has been removed. Use `orientation` instead; invoking `rotate` now fails with `rotate was renamed to orientation; for the two-finger gesture use: gesture rotate`.
- Breaking (ADR 0014, session ref-frame lifetime): a mutation through an `@ref` now expires the session's ref frame, so a later ref mutation without a fresh observation fails closed with a typed `details.reason` (`ref_frame_expired`, `ref_generation_mismatch`, `plain_ref_requires_complete_frame`, or `ref_not_issued`) instead of acting on a possibly-navigated screen. A ref-oriented sequence that performs several mutations must re-`snapshot` between them, consume an honestly issued `--settle` ref in pinned `@eN~s<gen>` form, or use selectors. Enforcement applies on every platform, not just iOS. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must capture between mutations or use selectors.
- Ref reads resolve against the authorized ref frame's source tree, so an internal read-only capture (including Android freshness) can no longer retarget an admitted `@ref` by positional coincidence. Read-only ref consumers keep the structured staleness warning while the frame retains the ref's evidence.
+18 -9
View File
@@ -22,9 +22,13 @@ the Android planned-touch executor.
## Decision
Public gesture inputs normalize once in `src/contracts/gesture-normalization.ts`. This is the
explicit compatibility boundary: convenience APIs and deprecated arguments become canonical
semantic intent before entering the runtime. The private daemon wire is free to evolve with that
model; compatibility is owed at CLI, Node.js, and MCP.
explicit public compatibility boundary: canonical semantic intent is produced before entering the
runtime. Deprecated arguments that have been removed (timed `swipe`, timed `gesture fling`,
`gesture rotate` `velocity`) are rejected with actionable `INVALID_ARGS` messages rather than
silently reinterpreted. The private daemon wire remains free to carry compatibility-only hints
that do not appear on public surfaces; Maestro timed swipes use
`internal.gestureExecutionProfile: 'endpoint-hold'` to preserve iOS fast-swipe-then-hold behavior
while still routing through the canonical `pan` input.
The runtime plans canonical intent in `src/contracts/gesture-plan.ts`. Contact topology is separate
from motion:
@@ -32,12 +36,17 @@ from motion:
- one contact: pan or fling with a complete pointer trajectory and an explicit execution profile;
- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories.
`swipe` without a duration is public sugar for a fixed-duration fling. Its historical optional
duration normalizes to pan intent with an endpoint-hold execution profile and reports a deprecation
toward explicit pan. Maestro-authored swipes follow the same normalization and materialize
Maestro's 400 ms default when duration is omitted. A genuine pan uses the timed-pan profile, so
compatibility aliases retain their release behavior without becoming a new semantic intent. The
same deprecation-to-pan rule applies to the historical fling duration. Pinch fixes translation and rotation at zero; rotate fixes
`swipe` without a duration remains public sugar for a fixed-duration fling. Timed public forms
(`swipe x1 y1 x2 y2 durationMs`, `gesture fling direction x y distance durationMs`,
`gesture swipe preset durationMs`, and `gesture rotate degrees x y velocity`) are rejected; callers
must use `gesture pan` for deliberate timed translation and `gesture rotate` without `velocity`.
Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs) and carry
the `endpoint-hold` execution profile through a daemon-internal compatibility seam
(`internal.gestureExecutionProfile`). This preserves the iOS fast-swipe-then-hold behavior that
matches Maestro's XCTest driver, without exposing the profile on the public command surface. Maestro
materializes its 400 ms default when duration is omitted. A genuine public `pan` uses the `timed-pan`
profile, so compatibility-only paths retain their release behavior without becoming a new semantic
intent. Pinch fixes translation and rotation at zero; rotate fixes
translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero;
transform can apply all three
components atomically. Intent remains on the plan even when aliases share an executor.
+12 -4
View File
@@ -21,7 +21,11 @@ Android compatibility is materially faster than native Maestro in the pager and
that advantage is a product constraint, not expendable migration headroom.
ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and
native injection. Maestro's supported gesture surface is single-pointer swipe only.
native injection. Maestro's supported gesture surface is single-pointer swipe only. Maestro swipes
normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs) and carry the
`endpoint-hold` execution profile through a daemon-internal compatibility seam
(`internal.gestureExecutionProfile`), preserving iOS fast-swipe-then-hold behavior without exposing
the profile on the public command surface.
## Decision
@@ -55,13 +59,17 @@ The engine does not implement platform input. Absolute and percentage swipes pre
endpoints without a hierarchy capture. Directional horizontal swipes reuse ADR 0013's shared in-page
preset geometry so an iOS right swipe does not become an interactive-back gesture; vertical presets
retain Maestro's platform geometry. All viewport-relative swipes resolve the cheapest fresh interaction
viewport available so ADR 0013 can validate every planned sample.
viewport available so ADR 0013 can validate every planned sample. Maestro swipes pair that resolved
viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request
so ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field.
When normalization already resolves a viewport, the adapter pairs it with the nested public gesture
request as daemon-internal metadata. ADR 0013 planning consumes that exact frame instead of probing
the platform a second time.
Target-relative swipes reuse the target-resolution observation. The resulting typed single-pointer
motion enters ADR 0013 after public compatibility normalization. Maestro code cannot construct or
execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories.
motion enters ADR 0013 as the canonical `pan` input, with the `endpoint-hold` execution profile carried
as daemon-internal metadata. Maestro code cannot construct or execute two-pointer pan, pinch, rotate,
transform, or physical pointer trajectories.
Simple successful target queries return their match, visibility decision, candidate count, and
observation generation in one response. The daemon may retain the provider snapshot behind that
@@ -152,3 +152,48 @@ test('no device scenario claims to prove percent truncation (bug class 1)', () =
'a 1px truncation delta is not app-observable; keep bug class 1 in runtime-port-geometry.test.ts',
);
});
// --- gestureExecutionProfile: proves the iOS endpoint-hold delivery path ---
const PROFILE_INVARIANT: Invariant = {
kind: 'gestureExecutionProfile',
command: 'swipe',
profile: 'endpoint-hold',
because: 'test',
};
const stopWithProfile = (command: string, executionProfile: string) => ({
type: 'replay_action_stop',
step: 1,
command,
ok: true,
durationMs: 300,
resultTiming: { executionProfile },
});
test('a swipe with endpoint-hold in resultTiming holds the gesture execution profile invariant', () => {
const result = evaluateInvariant([stopWithProfile('swipe', 'endpoint-hold')], PROFILE_INVARIANT);
assert.equal(result.status, 'held');
});
test('a swipe with timed-pan violates the endpoint-hold invariant', () => {
const result = evaluateInvariant([stopWithProfile('swipe', 'timed-pan')], PROFILE_INVARIANT);
assert.equal(result.status, 'violated');
assert.match(result.detail, /timed-pan/);
});
test('a trace with no executionProfile reports no-data', () => {
const result = evaluateInvariant([stop('swipe', 300)], PROFILE_INVARIANT);
assert.equal(result.status, 'no-data');
});
test('percent-swipe asserts the endpoint-hold execution profile in its engine-side invariant', () => {
const scenario = DIFFERENTIAL_SCENARIOS.find((s) => s.id === 'percent-swipe');
const invariant = scenario?.engineInvariants?.[0];
assert.ok(invariant, 'percent-swipe must carry an engine-side invariant');
assert.equal(invariant?.kind, 'gestureExecutionProfile');
assert.equal(
invariant?.kind === 'gestureExecutionProfile' ? invariant.profile : undefined,
'endpoint-hold',
);
});
@@ -41,6 +41,12 @@ export type Invariant =
metric: 'tapRetries' | 'hierarchyCaptures' | 'screenshotCaptures';
min: number;
because: string;
}
| {
kind: 'gestureExecutionProfile';
command: string;
profile: 'endpoint-hold' | 'timed-pan';
because: string;
};
export type InvariantResult = {
@@ -107,6 +113,32 @@ export function evaluateInvariant(events: TraceEvent[], invariant: Invariant): I
};
}
if (invariant.kind === 'gestureExecutionProfile') {
const profiles = steps
.map((step) => step.resultTiming?.executionProfile)
.filter((value): value is string => typeof value === 'string');
if (profiles.length === 0) {
return {
invariant,
status: 'no-data',
detail: `no ${invariant.command} step recorded an executionProfile`,
};
}
const firstMismatch = profiles.find((profile) => profile !== invariant.profile);
if (firstMismatch !== undefined) {
return {
invariant,
status: 'violated',
detail: `${invariant.command} executionProfile was ${firstMismatch} (expected ${invariant.profile}): ${invariant.because}`,
};
}
return {
invariant,
status: 'held',
detail: `${invariant.command} executionProfile is ${invariant.profile} on ${profiles.length} step(s)`,
};
}
const timed = steps.filter((step) => typeof step.durationMs === 'number');
if (timed.length === 0) {
return {
@@ -137,6 +137,15 @@ export const DIFFERENTIAL_SCENARIOS: DifferentialScenario[] = [
comparesAcrossEngines:
'A percentage-endpoint swipe completes on both engines. It does NOT compare resolved pixel endpoints — no metric records them and a 1px delta is not app-observable.',
expect: 'pass',
engineInvariants: [
{
kind: 'gestureExecutionProfile',
command: 'swipe',
profile: 'endpoint-hold',
because:
'Maestro timed swipes are normalized to pan with the endpoint-hold execution profile, which the iOS runner delivers as a fast 100ms move followed by a hold.',
},
],
divergenceMeans: 'The swipe behaves differently enough on one engine to fail the flow.',
},
// PARKED: tap-retry-if-no-change (flow kept at differential/flows/, invariant
@@ -263,7 +263,7 @@ test('command usage shows command flags without global flags', async () => {
const help = await usageForCommand('swipe');
if (help === null) throw new Error('Expected command help text');
assert.match(help, /Quick coordinate fling with optional repeat pattern/);
assert.match(help, /duration positional is accepted as a deprecated alias to pan/);
assert.doesNotMatch(help, /duration positional is accepted as a deprecated alias to pan/);
assert.match(help, /Command flags:/);
assert.match(help, /--pattern one-way\|ping-pong/);
assert.doesNotMatch(help, /Global flags:/);
+1 -1
View File
@@ -282,7 +282,7 @@ Read-only and waits:
Ambiguous find: add --first or --last. If info is not visible/exposed, report that gap instead of typing/searching/navigating to reveal it.
Navigation and gestures:
Use scroll for lists; swipe for quick coordinate gestures/carousels; gesture pan for deliberate timed drags; gesture fling for fast directional throws. Historical swipe/fling duration arguments remain compatibility aliases to pan and are deprecated.
Use scroll for lists; swipe for quick coordinate gestures/carousels; gesture pan for deliberate timed drags; gesture fling for fast directional throws.
For fast macOS desktop list traversal, prefer fixed pixel wheel steps and batch them when no snapshot is needed between passes:
agent-device scroll down --pixels 200 --duration-ms 50 --platform macos
agent-device batch --steps '[{"command":"scroll","input":{"direction":"down","pixels":200,"durationMs":50}},{"command":"scroll","input":{"direction":"down","pixels":200,"durationMs":50}}]' --platform macos
-8
View File
@@ -752,8 +752,6 @@ export type LongPressOptions = DeviceCommandBaseOptions &
export type SwipeOptions = DeviceCommandBaseOptions & {
from: { x: number; y: number };
to: { x: number; y: number };
/** @deprecated Timed movement is a pan. Omit this field for swipe/fling semantics. */
durationMs?: number;
count?: number;
pauseMs?: number;
pattern?: SwipePattern;
@@ -773,14 +771,10 @@ export type FlingOptions = DeviceCommandBaseOptions & {
x: number;
y: number;
distance?: number;
/** @deprecated Timed movement is a pan. Omit this field for fling semantics. */
durationMs?: number;
};
export type SwipeGestureOptions = DeviceCommandBaseOptions & {
preset: SwipePreset;
/** @deprecated Timed movement is a pan. Omit this field for swipe/fling semantics. */
durationMs?: number;
};
export type FocusOptions = DeviceCommandBaseOptions & {
@@ -819,8 +813,6 @@ export type RotateGestureOptions = DeviceCommandBaseOptions & {
degrees: number;
x?: number;
y?: number;
/** @deprecated Rotation pacing is derived from degrees. */
velocity?: number;
};
export type TransformGestureOptions = DeviceCommandBaseOptions & TransformGestureParams;
-1
View File
@@ -69,7 +69,6 @@ export type CommandInput = Omit<InternalRequestOptions, 'batchSteps' | 'target'>
update?: boolean;
url?: string;
value?: string;
velocity?: number;
x?: number;
y?: number;
} & Record<string, unknown>;
+3 -7
View File
@@ -107,16 +107,15 @@ const interactionCliSchemas = {
allowedFlags: [...postActionObservationCliFlags('longpress'), ...SELECTOR_SNAPSHOT_FLAGS],
},
swipe: {
helpDescription:
'Quick coordinate fling with optional repeat pattern. The historical duration positional is accepted as a deprecated alias to pan.',
positionalArgs: ['x1', 'y1', 'x2', 'y2', 'durationMs?'],
helpDescription: 'Quick coordinate fling with optional repeat pattern.',
positionalArgs: ['x1', 'y1', 'x2', 'y2'],
allowedFlags: ['count', 'pauseMs', 'pattern'],
},
gesture: {
usageOverride: 'gesture <pan|fling|swipe|pinch|rotate|transform> ...',
listUsageOverride: 'gesture <pan|fling|swipe|pinch|rotate|transform> ...',
helpDescription:
'Run touch gestures: pan <x> <y> <dx> <dy> [durationMs], fling <up|down|left|right> <x> <y> [distance], swipe <left|right|left-edge|right-edge>, pinch <scale> [x] [y], rotate <degrees> [x] [y], or transform <x> <y> <dx> <dy> <scale> <degrees> [durationMs]. Historical swipe/fling duration and rotate velocity arguments remain deprecated compatibility aliases. For command plans, output only command lines. Android transform verification should use all app-observable effects, for example wait text "pan changed yes", wait text "pinch changed yes", and wait text "rotate changed yes", not exact transform values.',
'Run touch gestures: pan <x> <y> <dx> <dy> [durationMs], fling <up|down|left|right> <x> <y> [distance], swipe <left|right|left-edge|right-edge>, pinch <scale> [x] [y], rotate <degrees> [x] [y], or transform <x> <y> <dx> <dy> <scale> <degrees> [durationMs]. For command plans, output only command lines. Android transform verification should use all app-observable effects, for example wait text "pan changed yes", wait text "pinch changed yes", and wait text "rotate changed yes", not exact transform values.',
summary: 'Run pan, fling, swipe, pinch, rotate, or transform gestures',
positionalArgs: ['pan|fling|swipe|pinch|rotate|transform', 'args?'],
allowsExtraPositionals: true,
@@ -453,7 +452,6 @@ function toFlingOptions(input: FlingInput): FlingOptions {
x: input.origin.x,
y: input.origin.y,
distance: input.distance,
durationMs: input.durationMs,
};
}
@@ -461,7 +459,6 @@ function toSwipeGestureOptions(input: SwipeGestureInput): SwipeGestureOptions {
return {
...commonToClientOptions(input),
preset: input.preset,
durationMs: input.durationMs,
};
}
@@ -480,7 +477,6 @@ function toRotateOptions(input: RotateInput): RotateGestureOptions {
degrees: input.degrees,
x: input.origin?.x,
y: input.origin?.y,
velocity: input.velocity,
};
}
@@ -5,7 +5,6 @@ test('swipe writes only typed daemon input', () => {
const request = interactionDaemonWriters.swipe({
from: { x: 10, y: 20 },
to: { x: 30, y: 40 },
durationMs: 300,
count: 2,
pauseMs: 10,
pattern: 'ping-pong',
@@ -15,7 +14,6 @@ test('swipe writes only typed daemon input', () => {
expect(request.input).toEqual({
from: { x: 10, y: 20 },
to: { x: 30, y: 40 },
durationMs: 300,
count: 2,
pauseMs: 10,
pattern: 'ping-pong',
-1
View File
@@ -127,7 +127,6 @@ export const interactionDaemonWriters = {
request(PUBLIC_COMMANDS.swipe, [], input, {
from: input.from,
to: input.to,
durationMs: input.durationMs,
count: input.count,
pauseMs: input.pauseMs,
pattern: input.pattern,
+1 -9
View File
@@ -139,10 +139,6 @@ const longPressFields = {
const swipeFields = {
from: requiredField(pointField('Swipe start point.')),
to: requiredField(pointField('Swipe end point.')),
durationMs: integerField('Deprecated: timed movement is a pan; omit for swipe.', {
min: 16,
max: 10_000,
}),
count: integerField('Number of swipe repetitions.', { min: 1, max: SWIPE_REPETITION_MAX }),
pauseMs: integerField('Pause between repeated swipes.', { min: 0, max: SWIPE_PAUSE_MAX_MS }),
pattern: enumField(SWIPE_PATTERNS),
@@ -221,11 +217,7 @@ const gestureFields = {
distance: integerField('Fling distance.', { min: 0 }),
scale: numberField('Pinch or transform scale.'),
degrees: numberField('Rotation in degrees.'),
velocity: numberField('Deprecated: rotation pacing is derived from degrees; must be non-zero.'),
durationMs: integerField(
'Pan/transform duration. Deprecated on swipe/fling; timed movement is a pan.',
{ min: 16, max: 10_000 },
),
durationMs: integerField('Pan/transform duration.', { min: 16, max: 10_000 }),
pointerCount: integerField('Pan touch pointer count (1 or 2).', { min: 1, max: 2 }),
};
@@ -76,7 +76,7 @@ test.each([
'click',
'snapshot',
'snapshot',
'swipe',
'gesture',
]);
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
},
@@ -90,7 +90,7 @@ test('preserves mutation ordering after a failed dispatch is continued', async (
baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }),
invoke: async (request) => {
requests.push(request);
if (request.command === 'swipe' && swipeAttempts++ === 0) {
if (request.command === 'gesture' && swipeAttempts++ === 0) {
throw new Error('dispatch failed after delivery became possible');
}
return request.command === 'snapshot'
@@ -132,10 +132,10 @@ test('preserves mutation ordering after a failed dispatch is continued', async (
await swipe(1);
expect(requests.map(({ command }) => command)).toEqual([
'swipe',
'gesture',
'snapshot',
'snapshot',
'swipe',
'gesture',
]);
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
});
@@ -189,15 +189,15 @@ test('defers a pending mutation boundary across non-mutating runtime commands',
env: {},
invalidateObservation() {},
});
expect(requests.map(({ command }) => command)).toEqual(['swipe', 'screenshot']);
expect(requests.map(({ command }) => command)).toEqual(['gesture', 'screenshot']);
await swipe(1);
expect(requests.map(({ command }) => command)).toEqual([
'swipe',
'gesture',
'screenshot',
'snapshot',
'snapshot',
'swipe',
'gesture',
]);
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
});
@@ -74,11 +74,12 @@ test('delegates lifecycle and coordinate gestures through public daemon commands
expect.objectContaining({ command: 'snapshot' }),
expect.objectContaining({ command: 'snapshot' }),
expect.objectContaining({
command: 'swipe',
command: 'gesture',
positionals: [],
input: {
from: { x: 360, y: 400 },
to: { x: 40, y: 400 },
kind: 'pan',
origin: { x: 360, y: 400 },
delta: { x: -320, y: 0 },
durationMs: 240,
},
}),
@@ -117,16 +118,20 @@ test('uses the direct viewport without snapshot and pairs it with the nested ges
});
expect(requests.at(-1)).toMatchObject({
command: 'swipe',
command: 'gesture',
input: {
from: { x: 370, y: 420 },
to: { x: 50, y: 420 },
kind: 'pan',
origin: { x: 370, y: 420 },
delta: { x: -320, y: 0 },
durationMs: 300,
},
internal: { gestureViewport: viewport },
internal: {
gestureExecutionProfile: 'endpoint-hold',
gestureViewport: viewport,
},
});
expect(resolveGestureViewport).toHaveBeenCalledOnce();
expect(requests.map(({ command }) => command)).toEqual(['swipe']);
expect(requests.map(({ command }) => command)).toEqual(['gesture']);
});
test('uses an observation as the baseline for a later mutation barrier', async () => {
@@ -193,10 +198,10 @@ test('uses an observation as the baseline for a later mutation barrier', async (
expect(observation).toMatchObject({ matched: true });
expect(requests.map(({ command }) => command)).toEqual([
'swipe',
'gesture',
'snapshot',
'snapshot',
'swipe',
'gesture',
]);
expect(port.readMetrics?.()).toEqual({
hierarchyCaptures: 2,
@@ -252,10 +257,10 @@ test('settles a gesture before dispatching another gesture', async () => {
await swipe(1);
expect(requests.map(({ command }) => command)).toEqual([
'swipe',
'gesture',
'snapshot',
'snapshot',
'swipe',
'gesture',
]);
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
});
@@ -1,4 +1,8 @@
import assert from 'node:assert/strict';
import { describe, expect, test } from 'vitest';
import { buildGesturePlan } from '../../../contracts/gesture-plan.ts';
import { readGesturePayload } from '../../../contracts/gesture-input.ts';
import { normalizePublicGesture } from '../../../contracts/gesture-normalization.ts';
import {
projectMaestroPublicOperation,
type MaestroPublicOperation,
@@ -95,11 +99,19 @@ describe('Maestro public operation projection', () => {
viewport: { x: 0, y: 0, width: 100, height: 200 },
},
expected: {
command: 'swipe',
command: 'gesture',
positionals: [],
input: { from: { x: 90, y: 50 }, to: { x: 10, y: 50 }, durationMs: 400 },
input: {
kind: 'pan',
origin: { x: 90, y: 50 },
delta: { x: -80, y: 0 },
durationMs: 400,
},
flags: { postGestureStabilization: false },
internal: { gestureViewport: { x: 0, y: 0, width: 100, height: 200 } },
internal: {
gestureExecutionProfile: 'endpoint-hold',
gestureViewport: { x: 0, y: 0, width: 100, height: 200 },
},
},
},
{
@@ -163,15 +175,40 @@ describe('Maestro public operation projection', () => {
expect(projectMaestroPublicOperation(operation)).toEqual(expected);
});
test('omits optional swipe and scroll fields', () => {
test('preserves endpoint-hold for swipes without a viewport and omits scroll duration', () => {
expect(
projectMaestroPublicOperation({
kind: 'swipe',
gesture: { from: { x: 1, y: 2 }, to: { x: 3, y: 4 }, durationMs: 5 },
}),
).not.toHaveProperty('internal');
).toMatchObject({
command: 'gesture',
internal: { gestureExecutionProfile: 'endpoint-hold' },
});
expect(projectMaestroPublicOperation({ kind: 'scroll', direction: 'up' })).not.toHaveProperty(
'input',
);
});
test('a 400 ms Maestro swipe reaches plan execution with endpoint-hold profile', () => {
const viewport = { x: 0, y: 0, width: 400, height: 800 };
const projected = projectMaestroPublicOperation({
kind: 'swipe',
gesture: { from: { x: 360, y: 430 }, to: { x: 40, y: 430 }, durationMs: 400 },
viewport,
});
const input = readGesturePayload(projected.input);
const normalized = normalizePublicGesture(input);
if (normalized.gesture.intent === 'pan' && projected.internal?.gestureExecutionProfile) {
normalized.gesture.executionProfile = projected.internal.gestureExecutionProfile;
}
const plan = buildGesturePlan(
normalized.gesture,
projected.internal?.gestureViewport ?? viewport,
);
assert.equal(plan.topology, 'single');
assert.equal(plan.intent, 'pan');
assert.equal(plan.executionProfile, 'endpoint-hold');
assert.equal(plan.durationMs, 400);
});
});
+2 -1
View File
@@ -201,7 +201,7 @@ function createDaemonMaestroRuntimeParts(options: CreateDaemonMaestroRuntimeOper
);
},
gesture: async (input, context) => {
await invokeMutation(
const data = await invokeMutation(
{
kind: 'swipe',
gesture: input,
@@ -210,6 +210,7 @@ function createDaemonMaestroRuntimeParts(options: CreateDaemonMaestroRuntimeOper
context,
'deferred',
);
return data ? { data } : undefined;
},
inputText: async (input, context) => await typeTextAndSettle(input.text, context),
eraseText: async (input, context) =>
@@ -158,12 +158,21 @@ function projectPointClick(
function projectSwipe(
operation: Extract<MaestroInputOperation, { kind: 'swipe' }>,
): ProjectedMaestroPublicOperation {
const { from, to, durationMs } = operation.gesture;
return {
command: 'swipe',
command: 'gesture',
positionals: [],
input: operation.gesture,
input: {
kind: 'pan',
origin: from,
delta: { x: to.x - from.x, y: to.y - from.y },
durationMs,
},
flags: { postGestureStabilization: false },
...(operation.viewport ? { internal: { gestureViewport: operation.viewport } } : {}),
internal: {
gestureExecutionProfile: 'endpoint-hold',
...(operation.viewport ? { gestureViewport: operation.viewport } : {}),
},
};
}
+7 -1
View File
@@ -97,6 +97,8 @@ export type MaestroRuntimeResult = {
observation?: MaestroObservation;
outputEnv?: Record<string, string>;
artifactPaths?: string[];
/** Daemon response data for gesture/profile evidence in replay traces. */
data?: Record<string, unknown>;
};
export type MaestroRuntimeMetrics = {
@@ -122,7 +124,11 @@ export type MaestroEngineEvent = {
export type MaestroEngineObserver = {
commandStarted?(event: MaestroEngineEvent): void;
commandCompleted?(
event: MaestroEngineEvent & { durationMs: number; runtimeMetrics?: MaestroRuntimeMetrics },
event: MaestroEngineEvent & {
durationMs: number;
runtimeMetrics?: MaestroRuntimeMetrics;
data?: Record<string, unknown>;
},
): void;
commandFailed?(
event: MaestroEngineEvent & {
+2 -1
View File
@@ -75,13 +75,14 @@ async function executeObservedStep(
};
notifyMaestroObserver(() => state.options.observer?.commandStarted?.(event));
try {
await executeMaestroReplayPlanStep(step, state);
const result = await executeMaestroReplayPlanStep(step, state);
checkpointMaestroCancellation(state.options.signal);
notifyMaestroObserver(() =>
state.options.observer?.commandCompleted?.({
...event,
durationMs: now() - startedAt,
...runtimeMetricsDelta(metricsBefore, state.port.readMetrics?.()),
...(result?.data ? { data: result.data } : {}),
}),
);
} catch (error) {
@@ -21,6 +21,7 @@ import type {
MaestroObservationCondition,
MaestroRuntimeCommand,
MaestroRuntimePort,
MaestroRuntimeResult,
} from './engine-types.ts';
import type {
MaestroReplayPlan,
@@ -50,37 +51,41 @@ type PlanStepFailure = {
export async function executeMaestroReplayPlanStep(
step: MaestroReplayPlanStep,
state: MaestroReplayPlanExecutionState,
): Promise<void> {
): Promise<MaestroRuntimeResult | undefined> {
checkpointMaestroCancellation(state.options.signal);
try {
await withStepScopes(step, state.context, async () => {
await executeStep(step, state);
return await withStepScopes(step, state.context, async () => {
return await executeStep(step, state);
});
} catch (error) {
throw asMaestroReplayPlanStepFailure(error, step);
}
}
async function executeStep(step: MaestroReplayPlanStep, state: MaestroReplayPlanExecutionState) {
async function executeStep(
step: MaestroReplayPlanStep,
state: MaestroReplayPlanExecutionState,
): Promise<MaestroRuntimeResult | undefined> {
if (step.kind === 'command') {
await executeOptionalCommand(step.command, step.appId, state);
return;
return await executeOptionalCommand(step.command, step.appId, state);
}
await executeOpaqueStep(step, state);
return undefined;
}
async function executeOptionalCommand(
command: MaestroRuntimeCommand,
appId: string | undefined,
state: MaestroReplayPlanExecutionState,
): Promise<void> {
): Promise<MaestroRuntimeResult | undefined> {
try {
await executeCommand(command, appId, state);
return await executeCommand(command, appId, state);
} catch (error) {
checkpointMaestroCancellation(state.options.signal);
if (!isOptionalCommand(command) || !isMaestroTestFailure(error)) throw error;
state.warnings.push(formatOptionalWarning(command, error));
state.skipped += 1;
return undefined;
}
}
@@ -98,7 +103,7 @@ async function executeCommand(
rawCommand: MaestroRuntimeCommand,
appId: string | undefined,
state: MaestroReplayPlanExecutionState,
): Promise<void> {
): Promise<MaestroRuntimeResult | undefined> {
const command = resolveCommand(rawCommand, state.context);
switch (command.kind) {
case 'assertVisible':
@@ -108,7 +113,7 @@ async function executeCommand(
state,
);
state.executed += 1;
return;
return undefined;
case 'assertNotVisible':
await requireObservation(
{ kind: 'notVisible', selector: command.target },
@@ -116,7 +121,7 @@ async function executeCommand(
state,
);
state.executed += 1;
return;
return undefined;
case 'extendedWaitUntil':
await requireObservation(
readExtendedWaitCondition(command),
@@ -124,9 +129,9 @@ async function executeCommand(
state,
);
state.executed += 1;
return;
return undefined;
default:
await executeRuntimeCommand(command, appId, state);
return await executeRuntimeCommand(command, appId, state);
}
}
@@ -134,7 +139,7 @@ async function executeRuntimeCommand(
command: MaestroRuntimeCommand,
appId: string | undefined,
state: MaestroReplayPlanExecutionState,
): Promise<void> {
): Promise<MaestroRuntimeResult> {
const request = stripUndefined({
command,
env: state.context.values,
@@ -150,6 +155,7 @@ async function executeRuntimeCommand(
if (result.outputEnv) state.context.merge(result.outputEnv);
result.artifactPaths?.forEach((entry) => state.artifacts.add(entry));
state.executed += 1;
return result;
}
async function executeOpaqueStep(
@@ -430,6 +430,7 @@ function resultWithArtifacts(
...optionalObservation(observation ?? operationObservation),
...optionalOutputEnv(result?.outputEnv),
...optionalArtifactPaths(defaultArtifacts, result?.artifactPaths),
...optionalData(result?.data),
};
}
@@ -464,6 +465,12 @@ function optionalArtifactPaths(
return artifactPaths.length > 0 ? { artifactPaths } : {};
}
function optionalData(
data: Record<string, unknown> | undefined,
): Partial<Pick<MaestroRuntimeResult, 'data'>> {
return data && Object.keys(data).length > 0 ? { data: { ...data } } : {};
}
async function resolveInputTarget(
authored: MaestroGestureTarget,
query: Pick<MaestroTargetQuery, 'purpose' | 'timeoutMs' | 'index' | 'childOf'>,
+2
View File
@@ -91,6 +91,8 @@ export type MaestroRuntimeOperationResult = {
readonly observation?: MaestroObservation;
readonly outputEnv?: Record<string, string>;
readonly artifactPaths?: readonly string[];
/** Daemon response data surfaced for replay-trace evidence. */
readonly data?: Record<string, unknown>;
};
export type MaestroRuntimeOperation<TInput> = (
+9 -21
View File
@@ -10,13 +10,6 @@ test('structured gesture input rejects durations outside the planner range', ()
delta: { x: 30, y: 40 },
durationMs: 0,
},
{
kind: 'fling',
direction: 'left',
origin: { x: 10, y: 20 },
durationMs: 10_001,
},
{ kind: 'swipe', preset: 'left', durationMs: 0 },
{
kind: 'transform',
origin: { x: 10, y: 20 },
@@ -43,19 +36,14 @@ test('structured gesture input rejects durations outside the planner range', ()
}
});
test('deprecated rotate velocity accepts finite non-zero compatibility values', () => {
for (const velocity of [-2.5, -1, 0.25, 3]) {
assert.deepEqual(readGesturePayload({ kind: 'rotate', degrees: 45, velocity }), {
kind: 'rotate',
degrees: 45,
origin: undefined,
velocity,
});
}
test('rotate does not accept velocity', () => {
assert.deepEqual(readGesturePayload({ kind: 'rotate', degrees: 45 }), {
kind: 'rotate',
degrees: 45,
origin: undefined,
});
for (const velocity of [0, Number.NaN, Number.POSITIVE_INFINITY]) {
assert.throws(() => readGesturePayload({ kind: 'rotate', degrees: 45, velocity }), {
code: 'INVALID_ARGS',
});
}
assert.throws(() => readGesturePayload({ kind: 'rotate', degrees: 45, velocity: 1 }), {
code: 'INVALID_ARGS',
});
});
+18 -15
View File
@@ -28,13 +28,11 @@ export type FlingGesturePayload = {
direction: ScrollDirection;
origin: Point;
distance?: number;
durationMs?: number;
};
export type SwipeGesturePayload = {
kind: 'swipe';
preset: SwipePreset;
durationMs?: number;
};
export type PinchGesturePayload = {
@@ -47,7 +45,6 @@ export type RotateGesturePayload = {
kind: 'rotate';
degrees: number;
origin?: Point;
velocity?: number;
};
export type TransformGesturePayload = {
@@ -85,19 +82,29 @@ export function readGesturePayload(input: unknown): GesturePayload {
throw new AppError('INVALID_ARGS', 'pointerCount is supported only for gesture pan');
}
if (kind === 'fling') {
if (record.durationMs !== undefined) {
throw new AppError(
'INVALID_ARGS',
'gesture fling does not accept durationMs; use gesture pan for timed movement',
);
}
return {
kind,
direction: readEnum(record, 'direction', SCROLL_DIRECTIONS),
origin: readPoint(record, 'origin'),
distance: readOptionalInteger(record, 'distance', { min: 0 }),
durationMs: readOptionalGestureDuration(record),
};
}
if (kind === 'swipe') {
if (record.durationMs !== undefined) {
throw new AppError(
'INVALID_ARGS',
'gesture swipe does not accept durationMs; use gesture pan for timed movement',
);
}
return {
kind,
preset: readEnum(record, 'preset', SWIPE_PRESETS),
durationMs: readOptionalGestureDuration(record),
};
}
if (kind === 'pinch') {
@@ -108,11 +115,16 @@ export function readGesturePayload(input: unknown): GesturePayload {
};
}
if (kind === 'rotate') {
if (record.velocity !== undefined) {
throw new AppError(
'INVALID_ARGS',
'gesture rotate does not accept velocity; rotation pacing derives from degrees',
);
}
return {
kind,
degrees: readNumber(record, 'degrees'),
origin: readOptionalPoint(record, 'origin'),
velocity: readOptionalRotateVelocity(record),
};
}
return {
@@ -125,15 +137,6 @@ export function readGesturePayload(input: unknown): GesturePayload {
};
}
function readOptionalRotateVelocity(record: Record<string, unknown>): number | undefined {
if (record.velocity === undefined) return undefined;
const velocity = readNumber(record, 'velocity');
if (velocity === 0) {
throw new AppError('INVALID_ARGS', 'Expected velocity to be a non-zero finite number.');
}
return velocity;
}
function readOptionalGestureDuration(record: Record<string, unknown>): number | undefined {
const durationMs = readOptionalInteger(record, 'durationMs');
if (durationMs === undefined) return undefined;
+12 -65
View File
@@ -20,7 +20,7 @@ test('CLI and .ad positionals normalize at one explicit syntax seam', () => {
test('CLI and .ad coordinate swipes normalize at one explicit syntax seam', () => {
assert.deepEqual(
swipePayloadFromPositionals(['10', '20', '30', '40', '300'], {
swipePayloadFromPositionals(['10', '20', '30', '40'], {
count: 2,
pauseMs: 5,
pattern: 'ping-pong',
@@ -28,7 +28,6 @@ test('CLI and .ad coordinate swipes normalize at one explicit syntax seam', () =
{
from: { x: 10, y: 20 },
to: { x: 30, y: 40 },
durationMs: 300,
count: 2,
pauseMs: 5,
pattern: 'ping-pong',
@@ -48,25 +47,14 @@ test('gesture recording codec round-trips structured requests', () => {
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
});
test('gesture recording codec preserves timed fling duration when distance is omitted', () => {
test('gesture recording codec round-trips fling with distance', () => {
const payload = {
kind: 'fling' as const,
direction: 'left' as const,
origin: { x: 10, y: 20 },
durationMs: 600,
distance: 180,
};
const positionals = gesturePayloadToPositionals(payload);
assert.deepEqual(positionals, ['fling', 'left', '10', '20', '180', '600']);
assert.deepEqual(normalizePublicGesture(gesturePayloadFromPositionals(positionals)), {
gesture: {
intent: 'pan',
origin: { x: 10, y: 20 },
delta: { x: -180, y: 0 },
durationMs: 600,
executionProfile: 'endpoint-hold',
},
deprecations: [{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' }],
});
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
});
test('pinch and rotate syntax rejects a partial origin', () => {
@@ -78,71 +66,39 @@ test('pinch and rotate syntax rejects a partial origin', () => {
});
});
test('rotate serialization omits behaviorless velocity when no origin can delimit it', () => {
assert.deepEqual(gesturePayloadToPositionals({ kind: 'rotate', degrees: 35, velocity: 2 }), [
'rotate',
'35',
]);
});
test('swipe is fling sugar unless legacy duration preserves swipe timing', () => {
test('swipe is fling sugar', () => {
assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left' }), {
gesture: { intent: 'fling', preset: 'left' },
deprecations: [],
});
assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left', durationMs: 400 }), {
gesture: {
intent: 'pan',
preset: 'left',
durationMs: 400,
executionProfile: 'endpoint-hold',
},
deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }],
});
});
test('coordinate swipe duration preserves swipe timing and endpoints', () => {
test('coordinate swipe is a quick fling', () => {
assert.deepEqual(
normalizePublicSwipeMotion({
from: { x: 360, y: 400 },
to: { x: 40, y: 400 },
durationMs: 500,
}),
{
gesture: {
intent: 'pan',
origin: { x: 360, y: 400 },
delta: { x: -320, y: 0 },
durationMs: 500,
executionProfile: 'endpoint-hold',
},
deprecations: [
{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' },
],
gesture: { intent: 'fling', from: { x: 360, y: 400 }, to: { x: 40, y: 400 } },
},
);
});
test('duration-bearing fling is an explicit compatibility alias for pan', () => {
test('fling stays a fling', () => {
assert.deepEqual(
normalizePublicGesture({
kind: 'fling',
direction: 'left',
origin: { x: 200, y: 300 },
distance: 80,
durationMs: 500,
}),
{
gesture: {
intent: 'pan',
intent: 'fling',
direction: 'left',
origin: { x: 200, y: 300 },
delta: { x: -80, y: 0 },
durationMs: 500,
executionProfile: 'endpoint-hold',
distance: 80,
},
deprecations: [
{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' },
],
},
);
});
@@ -153,16 +109,7 @@ test('multi-touch aliases become constraints on transform motion', () => {
origin: undefined,
scale: 1.5,
});
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45, velocity: 2 }), {
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45 }), {
gesture: { intent: 'rotate', origin: undefined, degrees: -45 },
deprecations: [
{ rule: 'rotate-velocity', replacement: 'Rotation pacing is derived from degrees.' },
],
});
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45, velocity: -1.5 }), {
gesture: { intent: 'rotate', origin: undefined, degrees: -45 },
deprecations: [
{ rule: 'rotate-velocity', replacement: 'Rotation pacing is derived from degrees.' },
],
});
});
+71 -101
View File
@@ -1,31 +1,37 @@
import type { Point } from '../kernel/snapshot.ts';
import { AppError } from '../kernel/errors.ts';
import { gestureDirectionDelta } from './scroll-gesture.ts';
import { readGesturePayload, type GesturePayload } from './gesture-input.ts';
import { GESTURE_FLING_DEFAULT_DISTANCE } from './gesture-plan.ts';
import type { GestureSemanticInput } from './gesture-plan-types.ts';
export type GestureCompatibilityRule = 'swipe-duration' | 'fling-duration' | 'rotate-velocity';
export type GestureDeprecation = {
rule: GestureCompatibilityRule;
replacement: string;
};
export type NormalizedPublicGesture = {
gesture: GestureSemanticInput;
deprecations: GestureDeprecation[];
};
export type SwipePayload = {
from: Point;
to: Point;
durationMs?: number;
count?: number;
pauseMs?: number;
pattern?: 'one-way' | 'ping-pong';
};
function assertPositionalCount(
args: readonly string[],
max: number,
usageMessage: string,
): asserts args is { length: number } & typeof args {
if (args.length > max) {
throw new AppError('INVALID_ARGS', usageMessage);
}
}
function readOriginDelta(args: readonly string[]): { origin: Point; delta: Point } {
return {
origin: { x: Number(args[0]), y: Number(args[1]) },
delta: { x: Number(args[2]), y: Number(args[3]) },
};
}
/** The explicit parser for the public CLI and `.ad` gesture syntax. */
// fallow-ignore-next-line complexity
export function gesturePayloadFromPositionals(
@@ -35,50 +41,77 @@ export function gesturePayloadFromPositionals(
const kind = positionals[0];
const args = positionals.slice(1);
switch (kind) {
case 'pan':
case 'pan': {
assertPositionalCount(
args,
5,
'gesture pan accepts at most 5 arguments: x y dx dy [durationMs]',
);
const { origin, delta } = readOriginDelta(args);
return readGesturePayload({
kind,
origin: { x: Number(args[0]), y: Number(args[1]) },
delta: { x: Number(args[2]), y: Number(args[3]) },
origin,
delta,
pointerCount,
durationMs: optionalPositionNumber(args[4]),
});
case 'fling':
}
case 'fling': {
assertPositionalCount(
args,
4,
'gesture fling accepts at most 4 arguments: direction x y [distance]; for timed movement use gesture pan',
);
return readGesturePayload({
kind,
direction: args[0],
origin: { x: Number(args[1]), y: Number(args[2]) },
distance: optionalPositionNumber(args[3]),
durationMs: optionalPositionNumber(args[4]),
});
case 'swipe':
}
case 'swipe': {
assertPositionalCount(
args,
1,
'gesture swipe accepts 1 argument: preset; for timed movement use gesture pan',
);
return readGesturePayload({
kind,
preset: args[0],
durationMs: optionalPositionNumber(args[1]),
});
case 'pinch':
}
case 'pinch': {
assertPositionalCount(args, 3, 'gesture pinch accepts at most 3 arguments: scale [x] [y]');
return readGesturePayload({
kind,
scale: Number(args[0]),
origin: optionalOrigin(args[1], args[2]),
});
case 'rotate':
}
case 'rotate': {
assertPositionalCount(args, 3, 'gesture rotate accepts at most 3 arguments: degrees [x] [y]');
return readGesturePayload({
kind,
degrees: Number(args[0]),
origin: optionalOrigin(args[1], args[2]),
velocity: optionalPositionNumber(args[3]),
});
case 'transform':
}
case 'transform': {
assertPositionalCount(
args,
7,
'gesture transform accepts at most 7 arguments: x y dx dy scale degrees [durationMs]',
);
const { origin, delta } = readOriginDelta(args);
return readGesturePayload({
kind,
origin: { x: Number(args[0]), y: Number(args[1]) },
delta: { x: Number(args[2]), y: Number(args[3]) },
origin,
delta,
scale: Number(args[4]),
degrees: Number(args[5]),
durationMs: optionalPositionNumber(args[6]),
});
}
default:
return readGesturePayload({ kind });
}
@@ -97,25 +130,14 @@ export function gesturePayloadToPositionals(input: GesturePayload): string[] {
input.durationMs,
]);
case 'fling':
// `.ad` positionals cannot encode an empty distance slot. Preserve the
// planner default when a deprecated duration must occupy the next slot.
return compact([
input.kind,
input.direction,
input.origin.x,
input.origin.y,
input.durationMs === undefined
? input.distance
: (input.distance ?? GESTURE_FLING_DEFAULT_DISTANCE),
input.durationMs,
]);
return compact([input.kind, input.direction, input.origin.x, input.origin.y, input.distance]);
case 'swipe':
return compact([input.kind, input.preset, input.durationMs]);
return [input.kind, input.preset];
case 'pinch':
return compact([input.kind, input.scale, input.origin?.x, input.origin?.y]);
case 'rotate':
return input.origin
? compact([input.kind, input.degrees, input.origin.x, input.origin.y, input.velocity])
? compact([input.kind, input.degrees, input.origin.x, input.origin.y])
: [input.kind, String(input.degrees)];
case 'transform':
return compact([
@@ -134,19 +156,24 @@ export function gesturePayloadToPositionals(input: GesturePayload): string[] {
/** Parses the public CLI and `.ad` coordinate-swipe syntax. */
export function swipePayloadFromPositionals(
positionals: string[],
options: Omit<SwipePayload, 'from' | 'to' | 'durationMs'> = {},
options: Omit<SwipePayload, 'from' | 'to'> = {},
): SwipePayload {
if (positionals.length > 4) {
throw new AppError(
'INVALID_ARGS',
'swipe accepts 4 arguments: x1 y1 x2 y2; for timed movement use gesture pan',
);
}
return {
from: { x: Number(positionals[0]), y: Number(positionals[1]) },
to: { x: Number(positionals[2]), y: Number(positionals[3]) },
...(positionals[4] === undefined ? {} : { durationMs: Number(positionals[4]) }),
...(options.count === undefined ? {} : { count: options.count }),
...(options.pauseMs === undefined ? {} : { pauseMs: options.pauseMs }),
...(options.pattern === undefined ? {} : { pattern: options.pattern }),
};
}
/** The only public/deprecated gesture interpretation point. */
/** The only public gesture interpretation point. */
export function normalizePublicGesture(input: GesturePayload): NormalizedPublicGesture {
switch (input.kind) {
case 'pan':
@@ -158,26 +185,8 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
pointerCount: input.pointerCount,
durationMs: input.durationMs,
},
deprecations: [],
};
case 'fling': {
if (input.durationMs !== undefined) {
return {
gesture: {
intent: 'pan',
origin: input.origin,
delta: gestureDirectionDelta(
input.direction,
input.distance ?? GESTURE_FLING_DEFAULT_DISTANCE,
),
durationMs: input.durationMs,
executionProfile: 'endpoint-hold',
},
deprecations: [
{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' },
],
};
}
case 'fling':
return {
gesture: {
intent: 'fling',
@@ -185,40 +194,16 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
origin: input.origin,
distance: input.distance,
},
deprecations: [],
};
}
case 'swipe':
return input.durationMs === undefined
? { gesture: { intent: 'fling', preset: input.preset }, deprecations: [] }
: {
gesture: {
intent: 'pan',
preset: input.preset,
durationMs: input.durationMs,
executionProfile: 'endpoint-hold',
},
deprecations: [
{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' },
],
};
return { gesture: { intent: 'fling', preset: input.preset } };
case 'pinch':
return {
gesture: { intent: 'pinch', origin: input.origin, scale: input.scale },
deprecations: [],
};
case 'rotate':
return {
gesture: { intent: 'rotate', origin: input.origin, degrees: input.degrees },
deprecations:
input.velocity === undefined
? []
: [
{
rule: 'rotate-velocity',
replacement: 'Rotation pacing is derived from degrees.',
},
],
};
case 'transform':
return {
@@ -230,7 +215,6 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
degrees: input.degrees,
durationMs: input.durationMs,
},
deprecations: [],
};
}
}
@@ -238,23 +222,9 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
export function normalizePublicSwipeMotion(input: {
from: Point;
to: Point;
durationMs?: number;
}): NormalizedPublicGesture {
if (input.durationMs === undefined) {
return {
gesture: { intent: 'fling', from: input.from, to: input.to },
deprecations: [],
};
}
return {
gesture: {
intent: 'pan',
origin: input.from,
delta: { x: input.to.x - input.from.x, y: input.to.y - input.from.y },
durationMs: input.durationMs,
executionProfile: 'endpoint-hold',
},
deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }],
gesture: { intent: 'fling', from: input.from, to: input.to },
};
}
-6
View File
@@ -28,12 +28,6 @@ export type GestureSemanticInput =
durationMs?: number;
executionProfile?: GestureExecutionProfile;
}
| {
intent: 'pan';
preset: SwipePreset;
durationMs: number;
executionProfile?: GestureExecutionProfile;
}
| { intent: 'pinch'; origin?: Point; scale: number }
| { intent: 'rotate'; origin?: Point; degrees: number }
| {
+1 -13
View File
@@ -20,7 +20,7 @@ export * from './gesture-plan-types.ts';
export const GESTURE_SAMPLE_INTERVAL_MS = 16;
export const GESTURE_INITIAL_ANGLE_DEGREES = -90;
export const GESTURE_FLING_DEFAULT_DISTANCE = 180;
const GESTURE_FLING_DEFAULT_DISTANCE = 180;
const GESTURE_INITIAL_SPAN_RATIO = 0.25;
const GESTURE_PINCH_INITIAL_SPAN_RATIO = 0.4;
@@ -182,18 +182,6 @@ function buildPanPlan(
DEFAULT_PAN_DURATION_MS,
'gesture pan durationMs',
);
if ('preset' in input) {
const { from, to } = presetGestureEndpoints(input.preset, viewport);
return buildSinglePointerPlan(
'pan',
from,
to,
durationMs,
viewport,
input.executionProfile ?? 'timed-pan',
profile,
);
}
if ((input.pointerCount ?? 1) === 1) {
const start = finitePoint(input.origin, 'gesture pan origin');
const delta = finitePoint(input.delta, 'gesture pan delta');
+7 -7
View File
@@ -49,14 +49,14 @@ describe('single-pointer plans', () => {
test('planner resolves swipe presets inside the active viewport', () => {
const viewport = { x: 20, y: 30, width: 400, height: 800 };
const fling = buildGesturePlan({ intent: 'fling', preset: 'left' }, viewport);
const pan = buildGesturePlan({ intent: 'pan', preset: 'right', durationMs: 400 }, viewport);
const leftFling = buildGesturePlan({ intent: 'fling', preset: 'left' }, viewport);
const rightFling = buildGesturePlan({ intent: 'fling', preset: 'right' }, viewport);
assert.deepEqual(fling.pointers[0].samples[0]?.point, { x: 360, y: 430 });
assert.deepEqual(fling.pointers[0].samples.at(-1)?.point, { x: 80, y: 430 });
assert.equal(pan.durationMs, 400);
assert.deepEqual(pan.pointers[0].samples[0]?.point, { x: 80, y: 430 });
assert.deepEqual(pan.pointers[0].samples.at(-1)?.point, { x: 360, y: 430 });
assert.deepEqual(leftFling.pointers[0].samples[0]?.point, { x: 360, y: 430 });
assert.deepEqual(leftFling.pointers[0].samples.at(-1)?.point, { x: 80, y: 430 });
assert.equal(rightFling.durationMs, 100);
assert.deepEqual(rightFling.pointers[0].samples[0]?.point, { x: 80, y: 430 });
assert.deepEqual(rightFling.pointers[0].samples.at(-1)?.point, { x: 360, y: 430 });
});
test('single-pointer pan preserves its execution profile', () => {
+1 -1
View File
@@ -152,7 +152,7 @@ export function requireGestureSupported(input: GestureSemanticInput, device: Dev
throw unsupportedGesture(input, gesturePlatformMessage(input, device));
}
// Linux can preserve public coordinate/preset swipe through its drag primitive, but cannot
// honor the velocity semantics authored by `gesture fling`.
// honor the speed semantics authored by `gesture fling`.
if (input.intent === 'fling' && 'direction' in input && device.platform === 'linux') {
throw unsupportedGesture(input, 'gesture fling is not supported on Linux');
}
@@ -114,7 +114,7 @@ test('swipe rejects repetition inputs that can monopolize the request', async ()
message: 'Expected pauseMs to be at most 10000.',
},
{
input: { count: 7, durationMs: 10_000 },
input: { count: 7, pauseMs: 10_000 },
message: 'Swipe series must fit within 60000ms.',
},
];
@@ -513,7 +513,7 @@ test('.ad replay normalizes resolved gesture and swipe syntax into structured da
script: [
'env X=10',
'gesture pan ${X} 20 30 40 500 --pointer-count 2',
'swipe ${X} 20 30 40 300 --count 2 --pause-ms 5 --pattern ping-pong',
'swipe ${X} 20 30 40 --count 2 --pause-ms 5 --pattern ping-pong',
'',
].join('\n'),
});
@@ -543,7 +543,6 @@ test('.ad replay normalizes resolved gesture and swipe syntax into structured da
input: {
from: { x: 10, y: 20 },
to: { x: 30, y: 40 },
durationMs: 300,
count: 2,
pauseMs: 5,
pattern: 'ping-pong',
@@ -1606,7 +1605,15 @@ test('runReplayScriptFile resolves Maestro swipe.label from a labeled element re
calls.map((call) => [call.command, call.input]),
[
['snapshot', undefined],
['swipe', { from: { x: 110, y: 250 }, to: { x: 110, y: 40 }, durationMs: 400 }],
[
'gesture',
{
kind: 'pan',
origin: { x: 110, y: 250 },
delta: { x: 0, y: -210 },
durationMs: 400,
},
],
],
);
});
@@ -1664,7 +1671,15 @@ test('runReplayScriptFile keeps Maestro swipe.label anchored to the matched labe
calls.map((call) => [call.command, call.input]),
[
['snapshot', undefined],
['swipe', { from: { x: 100, y: 124 }, to: { x: 100, y: 14 }, durationMs: 400 }],
[
'gesture',
{
kind: 'pan',
origin: { x: 100, y: 124 },
delta: { x: 0, y: -110 },
durationMs: 400,
},
],
],
);
});
@@ -1717,16 +1732,25 @@ test('runReplayScriptFile resolves Maestro screen swipes from the direct viewpor
calls.map((call) => [call.command, call.input]),
[
[
'swipe',
'gesture',
{
from: { x: 340, y: 400 },
to: { x: 60, y: 400 },
kind: 'pan',
origin: { x: 340, y: 400 },
delta: { x: -280, y: 0 },
durationMs: 300,
},
],
['snapshot', undefined],
['snapshot', undefined],
['swipe', { from: { x: 360, y: 400 }, to: { x: 40, y: 400 }, durationMs: 300 }],
[
'gesture',
{
kind: 'pan',
origin: { x: 360, y: 400 },
delta: { x: -320, y: 0 },
durationMs: 300,
},
],
],
);
});
@@ -1779,16 +1803,25 @@ test('runReplayScriptFile delegates Android directional swipes and preserves per
calls.map((call) => [call.command, call.input]),
[
[
'swipe',
'gesture',
{
from: { x: 340, y: 400 },
to: { x: 60, y: 400 },
kind: 'pan',
origin: { x: 340, y: 400 },
delta: { x: -280, y: 0 },
durationMs: 300,
},
],
['snapshot', undefined],
['snapshot', undefined],
['swipe', { from: { x: 360, y: 400 }, to: { x: 40, y: 400 }, durationMs: 300 }],
[
'gesture',
{
kind: 'pan',
origin: { x: 360, y: 400 },
delta: { x: -320, y: 0 },
durationMs: 300,
},
],
],
);
});
+35 -20
View File
@@ -3,7 +3,6 @@ import {
gesturePayloadToPositionals,
normalizePublicGesture,
normalizePublicSwipeMotion,
type GestureDeprecation,
type SwipePayload,
} from '../../contracts/gesture-normalization.ts';
import { requireGestureSupported } from '../../core/capabilities.ts';
@@ -16,6 +15,7 @@ import {
import { AppError, normalizeError } from '../../kernel/errors.ts';
import { readOptionalInteger } from '../../kernel/input-validation.ts';
import type { Point } from '../../kernel/snapshot.ts';
import type { GestureSemanticInput } from '../../contracts/gesture-plan-types.ts';
import { isActiveProviderDevice } from '../../provider-device-runtime.ts';
import { sleep } from '../../utils/timeouts.ts';
import type { DaemonResponse, SessionState } from '../types.ts';
@@ -46,6 +46,9 @@ export async function dispatchGestureViaRuntime(
return await dispatchGestureInteraction(params, 'gesture', async (session) => {
const input = readGesturePayload(params.req.input);
const normalized = normalizePublicGesture(input);
if (normalized.gesture.intent === 'pan' && params.req.internal?.gestureExecutionProfile) {
normalized.gesture.executionProfile = params.req.internal.gestureExecutionProfile;
}
requireGestureSupported(normalized.gesture, session.device);
const result = await createGestureRuntime(params).interactions.gesture({
session: params.sessionName,
@@ -55,7 +58,9 @@ export async function dispatchGestureViaRuntime(
return {
positionals: gesturePayloadToPositionals(input),
flags: gestureReplayFlags(input, params.req.flags),
responseData: gestureResponseData(result, normalized.deprecations),
responseData: gestureResponseData(result, {
executionProfile: resolveExecutionProfile(normalized.gesture),
}),
...(input.kind === 'pinch' ? { recordingResultExtra: { scale: input.scale } } : {}),
};
});
@@ -75,7 +80,7 @@ export async function dispatchSwipeViaRuntime(
return {
positionals: swipeReplayPositionals(input),
flags: params.req.flags,
responseData: gestureResponseData(result, result.deprecations ?? [], {
responseData: gestureResponseData(result, {
from: input.from,
to: input.to,
x1: input.from.x,
@@ -84,6 +89,7 @@ export async function dispatchSwipeViaRuntime(
y2: input.to.y,
effectiveDurationMs: result.durationMs,
timingMode: 'direct',
executionProfile: 'endpoint-hold',
count,
pauseMs,
pattern,
@@ -147,11 +153,18 @@ async function dispatchGestureInteraction(
}
}
function resolveExecutionProfile(gesture: GestureSemanticInput): string | undefined {
if (gesture.intent === 'fling') return 'endpoint-hold';
if (gesture.intent === 'pan') return gesture.executionProfile ?? 'timed-pan';
return undefined;
}
function gestureResponseData(
result: GestureRuntimeResult,
deprecations: readonly GestureDeprecation[],
extra: Record<string, unknown> = {},
): Record<string, unknown> {
const executionProfile =
typeof extra.executionProfile === 'string' ? extra.executionProfile : undefined;
return {
kind: result.kind,
durationMs: result.durationMs,
@@ -160,7 +173,14 @@ function gestureResponseData(
to: result.to,
...(result.backendResult ?? {}),
...extra,
...(deprecations.length > 0 ? { deprecations } : {}),
...(executionProfile
? {
timing: {
executionProfile,
gestureDurationMs: result.durationMs,
},
}
: {}),
message: result.message,
};
}
@@ -170,6 +190,12 @@ function readSwipeInput(input: unknown): SwipePayload {
throw new AppError('INVALID_ARGS', 'swipe requires structured object input');
}
const record = input as Record<string, unknown>;
if (record.durationMs !== undefined) {
throw new AppError(
'INVALID_ARGS',
'swipe does not accept durationMs; use gesture pan for timed movement',
);
}
const pattern = record.pattern;
if (pattern !== undefined && pattern !== 'one-way' && pattern !== 'ping-pong') {
throw new AppError('INVALID_ARGS', 'swipe pattern must be one-way or ping-pong');
@@ -177,7 +203,6 @@ function readSwipeInput(input: unknown): SwipePayload {
const payload: SwipePayload = {
from: readSwipePoint(record.from, 'swipe from'),
to: readSwipePoint(record.to, 'swipe to'),
durationMs: readOptionalInteger(record, 'durationMs', { min: 16, max: 10_000 }),
count: readOptionalInteger(record, 'count', { min: 1, max: SWIPE_REPETITION_MAX }),
pauseMs: readOptionalInteger(record, 'pauseMs', { min: 0, max: SWIPE_PAUSE_MAX_MS }),
pattern,
@@ -189,7 +214,7 @@ function readSwipeInput(input: unknown): SwipePayload {
function assertSwipeSeriesFitsRequest(input: SwipePayload): void {
const count = input.count ?? 1;
const pauseMs = input.pauseMs ?? 0;
const gestureDurationMs = input.durationMs ?? GESTURE_FLING_DURATION_MS;
const gestureDurationMs = GESTURE_FLING_DURATION_MS;
const scheduledDurationMs = count * gestureDurationMs + Math.max(0, count - 1) * pauseMs;
if (scheduledDurationMs <= SWIPE_SERIES_MAX_SCHEDULED_DURATION_MS) return;
throw new AppError(
@@ -200,7 +225,7 @@ function assertSwipeSeriesFitsRequest(input: SwipePayload): void {
pauseMs,
gestureDurationMs,
scheduledDurationMs,
hint: 'Reduce --count, --pause-ms, or the deprecated swipe duration.',
hint: 'Reduce --count or --pause-ms.',
},
);
}
@@ -220,13 +245,7 @@ function readSwipePoint(value: unknown, field: string): Point {
}
function swipeReplayPositionals(input: SwipePayload): string[] {
return [
String(input.from.x),
String(input.from.y),
String(input.to.x),
String(input.to.y),
...(input.durationMs === undefined ? [] : [String(input.durationMs)]),
];
return [String(input.from.x), String(input.from.y), String(input.to.x), String(input.to.y)];
}
async function runSwipeRepetitions(
@@ -238,7 +257,6 @@ async function runSwipeRepetitions(
pattern: 'one-way' | 'ping-pong',
) {
let result: Awaited<ReturnType<typeof runtime.interactions.gesture>> | undefined;
const deprecations = normalizePublicSwipeMotion(input).deprecations;
for (let index = 0; index < count; index += 1) {
const normalized = normalizePublicSwipeMotion(swipeMotionAtIndex(input, pattern, index));
result = await runtime.interactions.gesture({
@@ -249,10 +267,7 @@ async function runSwipeRepetitions(
if (pauseMs > 0 && index + 1 < count) await sleep(pauseMs);
}
if (!result) throw new Error('Swipe orchestration did not execute a gesture.');
return {
...result,
...(deprecations.length > 0 ? { deprecations } : {}),
};
return result;
}
function swipeMotionAtIndex(
@@ -59,17 +59,20 @@ function traceStopEvent(
event: MaestroEngineEvent & {
durationMs: number;
runtimeMetrics?: MaestroRuntimeMetrics;
data?: Record<string, unknown>;
error?: unknown;
},
): MaestroEngineEvent & {
durationMs: number;
runtimeMetrics?: MaestroRuntimeMetrics;
data?: Record<string, unknown>;
error?: unknown;
} {
return stripUndefined({
...(start ?? event),
durationMs: event.durationMs,
runtimeMetrics: event.runtimeMetrics,
data: event.data,
error: event.error,
});
}
@@ -116,10 +119,12 @@ function appendMaestroTraceStop(
event: MaestroEngineEvent & {
durationMs: number;
runtimeMetrics?: MaestroRuntimeMetrics;
data?: Record<string, unknown>;
error?: unknown;
},
ok: boolean,
): void {
const resultTiming = maestroResultTiming(event);
appendReplayTraceEvent(tracePath, {
type: 'replay_action_stop',
ts: new Date().toISOString(),
@@ -132,7 +137,29 @@ function appendMaestroTraceStop(
command: formatMaestroCommandProgress(event.command).command,
ok,
durationMs: event.durationMs,
...(event.runtimeMetrics ? { resultTiming: event.runtimeMetrics } : {}),
...(Object.keys(resultTiming).length > 0 ? { resultTiming } : {}),
...(!ok && event.error instanceof AppError ? { errorCode: event.error.code } : {}),
});
}
function maestroResultTiming(
event: MaestroEngineEvent & {
runtimeMetrics?: MaestroRuntimeMetrics;
data?: Record<string, unknown>;
},
): Record<string, unknown> {
const timing = isPlainRecord(event.data?.timing) ? event.data.timing : undefined;
return stripUndefined({
...(event.runtimeMetrics ?? {}),
...(typeof timing?.executionProfile === 'string'
? { executionProfile: timing.executionProfile }
: {}),
...(typeof timing?.gestureDurationMs === 'number'
? { gestureDurationMs: timing.gestureDurationMs }
: {}),
});
}
function isPlainRecord(value: unknown): value is Record<string, unknown> {
return !!value && typeof value === 'object' && !Array.isArray(value);
}
+3
View File
@@ -18,6 +18,7 @@ import type { RecordingScope } from '../contracts/recording-scope.ts';
import type { DeviceInfo, Platform, PlatformSelector } from '../kernel/device.ts';
import type { ExecBackgroundResult, ExecResult } from '../utils/exec.ts';
import type { Rect, SnapshotState } from '../kernel/snapshot.ts';
import type { GestureExecutionProfile } from '../contracts/gesture-plan-types.ts';
// Type-only import; erased at runtime. ref-frame.ts imports SessionState from
// here, so this back-edge must stay type-only to avoid a runtime cycle.
import type { RefFrameScope, RefFrameState } from './ref-frame.ts';
@@ -65,6 +66,8 @@ type DaemonRequestInternal = {
closeAppOnly?: boolean;
/** Provider-owned viewport already resolved while normalizing a nested gesture command. */
gestureViewport?: Rect;
/** Maestro-compat execution profile for timed coordinate swipes projected to `gesture pan`. */
gestureExecutionProfile?: GestureExecutionProfile;
/**
* ADR 0012 step 4 post-resolution guard: the verified target member's
* normalized local identity AND structural denotation (document order +
+3 -6
View File
@@ -170,16 +170,13 @@ test('MCP gesture tool exposes and forwards two-finger pan topology', async () =
]);
});
test('MCP gesture compatibility metadata accepts numeric rotate velocity', () => {
test('MCP gesture metadata exposes pan/transform duration', () => {
const gesture = listCommandTools().find((tool) => tool.name === 'gesture');
assert.ok(gesture);
assert.deepEqual(gesture.inputSchema.properties?.velocity, {
type: 'number',
description: 'Deprecated: rotation pacing is derived from degrees; must be non-zero.',
});
assert.equal(gesture.inputSchema.properties?.velocity, undefined);
assert.deepEqual(gesture.inputSchema.properties?.durationMs, {
type: 'integer',
description: 'Pan/transform duration. Deprecated on swipe/fling; timed movement is a pan.',
description: 'Pan/transform duration.',
minimum: 16,
maximum: 10_000,
});
@@ -99,9 +99,11 @@ test('Provider-backed integration Android touch provider handles multi-touch ges
...world.selection,
});
await client.interactions.swipe({
from: { x: 340, y: 400 },
to: { x: 60, y: 400 },
await client.interactions.pan({
x: 340,
y: 400,
dx: -280,
dy: 0,
durationMs: 300,
...world.selection,
});
@@ -121,11 +121,12 @@ test('Provider-backed integration Linux desktop flow uses semantic desktop and i
positionals: ['42', '84', '1'],
},
{
name: 'swipe coordinates',
command: 'swipe',
name: 'pan coordinates',
command: 'gesture',
input: {
from: { x: 10, y: 20 },
to: { x: 30, y: 40 },
kind: 'pan',
origin: { x: 10, y: 20 },
delta: { x: 20, y: 20 },
durationMs: 16,
},
expectData: {
+3 -7
View File
@@ -358,16 +358,12 @@ On Android, `fill` also verifies text and treats IME-owned capture as a terminal
Android text entry is owned by `agent-device`: provider-native injection when available, then chunk-safe ASCII shell input. Do not switch to raw `adb`, clipboard, or paste as an agent fallback. If non-ASCII is unsupported in the current backend, report the tool/device gap.
`click --button secondary` is the desktop context-menu flow on macOS.
`click --button middle` is reserved for future runner support and currently returns an explicit unsupported-operation error on macOS.
`swipe` is a quick, fixed-duration directional throw. Its historical optional `durationMs` remains
accepted for compatibility but normalizes to a pan and reports a deprecation; use `gesture pan` for
deliberate timed movement.
`swipe` is a quick, fixed-duration directional throw. Use `gesture pan` for deliberate timed movement.
Repeated coordinate swipes accept at most 200 repetitions and 10000ms pauses, and their combined
gesture/pause schedule must fit within 60000ms.
`gesture pan` accepts `x y dx dy [durationMs]` for deliberate drags. It uses one pointer by default. Add `--pointer-count 2` for a parallel two-finger pan with constant contact span and angle; this shares the bounded two-contact synthesizer used by transform while retaining pan intent. Android preserves the requested travel duration; iOS uses XCTest drag primitives for one-pointer pan and private XCTest synthesis for two-pointer pan.
`gesture fling` accepts `up|down|left|right x y [distance]` for fast directional throws. Its
historical duration is accepted as a deprecated alias to pan.
`gesture rotate` accepts `degrees [x] [y]`; the degree sign controls direction. Its historical
velocity is accepted but deprecated because pacing is derived from the requested rotation.
`gesture fling` accepts `up|down|left|right x y [distance]` for fast directional throws.
`gesture rotate` accepts `degrees [x] [y]`; the degree sign controls direction. Pacing is derived from the requested rotation.
`gesture transform` accepts `x y dx dy scale degrees [durationMs]` for one combined two-finger pan/zoom/rotate gesture on Android and iOS simulators. Pinch, rotate, two-finger pan, and transform use the same viewport-aware pointer planning; impossible paths fail before injection instead of clamping or distorting the requested motion.
On iOS simulators it uses private XCTest synthesis for a continuous two-finger pan/scale/rotation path, so verify app-level metrics instead of assuming the requested values map exactly to recognizer output.
On Android, `gesture transform` injects a geometric two-finger path. App recognizers may report non-exact pan, scale, and rotation values, so verify qualitative state such as `pan changed yes`, `pinch changed yes`, and `rotate changed yes` unless the app explicitly promises exact centroid metrics. If exact app-state values matter, prefer isolated `gesture pan`, `gesture pinch`, or `gesture rotate` commands.