mirror of
https://github.com/callstack/agent-device.git
synced 2026-09-14 20:06:34 +08:00
* feat!: remove deprecated gesture duration and rotate velocity inputs (#1218, #1216) Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fix: address CI failures - remove dead export, dedupe positional validation, migrate linux-desktop swipe test to pan Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * fixup! preserve Maestro swipe endpoint-hold execution profile via internal seam Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * docs(adr): describe Maestro endpoint-hold internal seam in ADR 0013/0015 Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> * feat: surface Maestro swipe executionProfile in replay trace and assert endpoint-hold in differential Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com> --------- Co-authored-by: Michał Pierzchała <thymikee@gmail.com> Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
This commit is contained in:
committed by
GitHub
parent
3fed8acda5
commit
6d99914f49
@@ -2,6 +2,13 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
- Breaking: removed deprecated gesture duration and rotate velocity inputs (#1218).
|
||||
- `swipe x1 y1 x2 y2` no longer accepts a trailing `durationMs` positional; use `gesture pan x1 y1 (x2-x1) (y2-y1) durationMs` for deliberate timed drags.
|
||||
- Maestro `swipe` operations with a duration continue to normalize to `gesture pan` with the `endpoint-hold` execution profile, preserving the Maestro-compatible fast-swipe-then-hold behavior on iOS.
|
||||
- `gesture fling direction x y` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
|
||||
- `gesture swipe preset` no longer accepts a trailing `durationMs` positional; use `gesture pan` for timed movement.
|
||||
- `gesture rotate degrees [x] [y]` no longer accepts a trailing `velocity` positional; rotation pacing is derived from `degrees`.
|
||||
- MCP/Node schemas no longer advertise `velocity` or `durationMs` on `swipe`/`fling`/`gesture swipe`; `durationMs` remains on `gesture pan` and `gesture transform`.
|
||||
- Breaking: the deprecated `rotate` CLI command alias has been removed. Use `orientation` instead; invoking `rotate` now fails with `rotate was renamed to orientation; for the two-finger gesture use: gesture rotate`.
|
||||
- Breaking (ADR 0014, session ref-frame lifetime): a mutation through an `@ref` now expires the session's ref frame, so a later ref mutation without a fresh observation fails closed with a typed `details.reason` (`ref_frame_expired`, `ref_generation_mismatch`, `plain_ref_requires_complete_frame`, or `ref_not_issued`) instead of acting on a possibly-navigated screen. A ref-oriented sequence that performs several mutations must re-`snapshot` between them, consume an honestly issued `--settle` ref in pinned `@eN~s<gen>` form, or use selectors. Enforcement applies on every platform, not just iOS. Legacy hand-written `.ad` scripts that reuse several bare refs from one snapshot must capture between mutations or use selectors.
|
||||
- Ref reads resolve against the authorized ref frame's source tree, so an internal read-only capture (including Android freshness) can no longer retarget an admitted `@ref` by positional coincidence. Read-only ref consumers keep the structured staleness warning while the frame retains the ref's evidence.
|
||||
|
||||
@@ -22,9 +22,13 @@ the Android planned-touch executor.
|
||||
## Decision
|
||||
|
||||
Public gesture inputs normalize once in `src/contracts/gesture-normalization.ts`. This is the
|
||||
explicit compatibility boundary: convenience APIs and deprecated arguments become canonical
|
||||
semantic intent before entering the runtime. The private daemon wire is free to evolve with that
|
||||
model; compatibility is owed at CLI, Node.js, and MCP.
|
||||
explicit public compatibility boundary: canonical semantic intent is produced before entering the
|
||||
runtime. Deprecated arguments that have been removed (timed `swipe`, timed `gesture fling`,
|
||||
`gesture rotate` `velocity`) are rejected with actionable `INVALID_ARGS` messages rather than
|
||||
silently reinterpreted. The private daemon wire remains free to carry compatibility-only hints
|
||||
that do not appear on public surfaces; Maestro timed swipes use
|
||||
`internal.gestureExecutionProfile: 'endpoint-hold'` to preserve iOS fast-swipe-then-hold behavior
|
||||
while still routing through the canonical `pan` input.
|
||||
|
||||
The runtime plans canonical intent in `src/contracts/gesture-plan.ts`. Contact topology is separate
|
||||
from motion:
|
||||
@@ -32,12 +36,17 @@ from motion:
|
||||
- one contact: pan or fling with a complete pointer trajectory and an explicit execution profile;
|
||||
- two contacts: pan, pinch, rotate, or transform with two complete, synchronized trajectories.
|
||||
|
||||
`swipe` without a duration is public sugar for a fixed-duration fling. Its historical optional
|
||||
duration normalizes to pan intent with an endpoint-hold execution profile and reports a deprecation
|
||||
toward explicit pan. Maestro-authored swipes follow the same normalization and materialize
|
||||
Maestro's 400 ms default when duration is omitted. A genuine pan uses the timed-pan profile, so
|
||||
compatibility aliases retain their release behavior without becoming a new semantic intent. The
|
||||
same deprecation-to-pan rule applies to the historical fling duration. Pinch fixes translation and rotation at zero; rotate fixes
|
||||
`swipe` without a duration remains public sugar for a fixed-duration fling. Timed public forms
|
||||
(`swipe x1 y1 x2 y2 durationMs`, `gesture fling direction x y distance durationMs`,
|
||||
`gesture swipe preset durationMs`, and `gesture rotate degrees x y velocity`) are rejected; callers
|
||||
must use `gesture pan` for deliberate timed translation and `gesture rotate` without `velocity`.
|
||||
Maestro-authored swipes normalize to the canonical `pan` input (origin, delta, durationMs) and carry
|
||||
the `endpoint-hold` execution profile through a daemon-internal compatibility seam
|
||||
(`internal.gestureExecutionProfile`). This preserves the iOS fast-swipe-then-hold behavior that
|
||||
matches Maestro's XCTest driver, without exposing the profile on the public command surface. Maestro
|
||||
materializes its 400 ms default when duration is omitted. A genuine public `pan` uses the `timed-pan`
|
||||
profile, so compatibility-only paths retain their release behavior without becoming a new semantic
|
||||
intent. Pinch fixes translation and rotation at zero; rotate fixes
|
||||
translation at zero and scale at one; two-finger pan fixes scale at one and rotation at zero;
|
||||
transform can apply all three
|
||||
components atomically. Intent remains on the plan even when aliases share an executor.
|
||||
|
||||
@@ -21,7 +21,11 @@ Android compatibility is materially faster than native Maestro in the pager and
|
||||
that advantage is a product constraint, not expendable migration headroom.
|
||||
|
||||
ADR 0013 separately owns public gesture normalization, contact topology, trajectory planning, and
|
||||
native injection. Maestro's supported gesture surface is single-pointer swipe only.
|
||||
native injection. Maestro's supported gesture surface is single-pointer swipe only. Maestro swipes
|
||||
normalize to ADR 0013's canonical `pan` input (origin, delta, durationMs) and carry the
|
||||
`endpoint-hold` execution profile through a daemon-internal compatibility seam
|
||||
(`internal.gestureExecutionProfile`), preserving iOS fast-swipe-then-hold behavior without exposing
|
||||
the profile on the public command surface.
|
||||
|
||||
## Decision
|
||||
|
||||
@@ -55,13 +59,17 @@ The engine does not implement platform input. Absolute and percentage swipes pre
|
||||
endpoints without a hierarchy capture. Directional horizontal swipes reuse ADR 0013's shared in-page
|
||||
preset geometry so an iOS right swipe does not become an interactive-back gesture; vertical presets
|
||||
retain Maestro's platform geometry. All viewport-relative swipes resolve the cheapest fresh interaction
|
||||
viewport available so ADR 0013 can validate every planned sample.
|
||||
viewport available so ADR 0013 can validate every planned sample. Maestro swipes pair that resolved
|
||||
viewport with `internal.gestureExecutionProfile: 'endpoint-hold'` on the nested public `gesture` request
|
||||
so ADR 0013 produces the iOS fast-swipe-then-hold profile without a public `executionProfile` field.
|
||||
When normalization already resolves a viewport, the adapter pairs it with the nested public gesture
|
||||
request as daemon-internal metadata. ADR 0013 planning consumes that exact frame instead of probing
|
||||
the platform a second time.
|
||||
|
||||
Target-relative swipes reuse the target-resolution observation. The resulting typed single-pointer
|
||||
motion enters ADR 0013 after public compatibility normalization. Maestro code cannot construct or
|
||||
execute two-pointer pan, pinch, rotate, transform, or physical pointer trajectories.
|
||||
motion enters ADR 0013 as the canonical `pan` input, with the `endpoint-hold` execution profile carried
|
||||
as daemon-internal metadata. Maestro code cannot construct or execute two-pointer pan, pinch, rotate,
|
||||
transform, or physical pointer trajectories.
|
||||
|
||||
Simple successful target queries return their match, visibility decision, candidate count, and
|
||||
observation generation in one response. The daemon may retain the provider snapshot behind that
|
||||
|
||||
@@ -152,3 +152,48 @@ test('no device scenario claims to prove percent truncation (bug class 1)', () =
|
||||
'a 1px truncation delta is not app-observable; keep bug class 1 in runtime-port-geometry.test.ts',
|
||||
);
|
||||
});
|
||||
|
||||
// --- gestureExecutionProfile: proves the iOS endpoint-hold delivery path ---
|
||||
|
||||
const PROFILE_INVARIANT: Invariant = {
|
||||
kind: 'gestureExecutionProfile',
|
||||
command: 'swipe',
|
||||
profile: 'endpoint-hold',
|
||||
because: 'test',
|
||||
};
|
||||
|
||||
const stopWithProfile = (command: string, executionProfile: string) => ({
|
||||
type: 'replay_action_stop',
|
||||
step: 1,
|
||||
command,
|
||||
ok: true,
|
||||
durationMs: 300,
|
||||
resultTiming: { executionProfile },
|
||||
});
|
||||
|
||||
test('a swipe with endpoint-hold in resultTiming holds the gesture execution profile invariant', () => {
|
||||
const result = evaluateInvariant([stopWithProfile('swipe', 'endpoint-hold')], PROFILE_INVARIANT);
|
||||
assert.equal(result.status, 'held');
|
||||
});
|
||||
|
||||
test('a swipe with timed-pan violates the endpoint-hold invariant', () => {
|
||||
const result = evaluateInvariant([stopWithProfile('swipe', 'timed-pan')], PROFILE_INVARIANT);
|
||||
assert.equal(result.status, 'violated');
|
||||
assert.match(result.detail, /timed-pan/);
|
||||
});
|
||||
|
||||
test('a trace with no executionProfile reports no-data', () => {
|
||||
const result = evaluateInvariant([stop('swipe', 300)], PROFILE_INVARIANT);
|
||||
assert.equal(result.status, 'no-data');
|
||||
});
|
||||
|
||||
test('percent-swipe asserts the endpoint-hold execution profile in its engine-side invariant', () => {
|
||||
const scenario = DIFFERENTIAL_SCENARIOS.find((s) => s.id === 'percent-swipe');
|
||||
const invariant = scenario?.engineInvariants?.[0];
|
||||
assert.ok(invariant, 'percent-swipe must carry an engine-side invariant');
|
||||
assert.equal(invariant?.kind, 'gestureExecutionProfile');
|
||||
assert.equal(
|
||||
invariant?.kind === 'gestureExecutionProfile' ? invariant.profile : undefined,
|
||||
'endpoint-hold',
|
||||
);
|
||||
});
|
||||
|
||||
@@ -41,6 +41,12 @@ export type Invariant =
|
||||
metric: 'tapRetries' | 'hierarchyCaptures' | 'screenshotCaptures';
|
||||
min: number;
|
||||
because: string;
|
||||
}
|
||||
| {
|
||||
kind: 'gestureExecutionProfile';
|
||||
command: string;
|
||||
profile: 'endpoint-hold' | 'timed-pan';
|
||||
because: string;
|
||||
};
|
||||
|
||||
export type InvariantResult = {
|
||||
@@ -107,6 +113,32 @@ export function evaluateInvariant(events: TraceEvent[], invariant: Invariant): I
|
||||
};
|
||||
}
|
||||
|
||||
if (invariant.kind === 'gestureExecutionProfile') {
|
||||
const profiles = steps
|
||||
.map((step) => step.resultTiming?.executionProfile)
|
||||
.filter((value): value is string => typeof value === 'string');
|
||||
if (profiles.length === 0) {
|
||||
return {
|
||||
invariant,
|
||||
status: 'no-data',
|
||||
detail: `no ${invariant.command} step recorded an executionProfile`,
|
||||
};
|
||||
}
|
||||
const firstMismatch = profiles.find((profile) => profile !== invariant.profile);
|
||||
if (firstMismatch !== undefined) {
|
||||
return {
|
||||
invariant,
|
||||
status: 'violated',
|
||||
detail: `${invariant.command} executionProfile was ${firstMismatch} (expected ${invariant.profile}): ${invariant.because}`,
|
||||
};
|
||||
}
|
||||
return {
|
||||
invariant,
|
||||
status: 'held',
|
||||
detail: `${invariant.command} executionProfile is ${invariant.profile} on ${profiles.length} step(s)`,
|
||||
};
|
||||
}
|
||||
|
||||
const timed = steps.filter((step) => typeof step.durationMs === 'number');
|
||||
if (timed.length === 0) {
|
||||
return {
|
||||
|
||||
@@ -137,6 +137,15 @@ export const DIFFERENTIAL_SCENARIOS: DifferentialScenario[] = [
|
||||
comparesAcrossEngines:
|
||||
'A percentage-endpoint swipe completes on both engines. It does NOT compare resolved pixel endpoints — no metric records them and a 1px delta is not app-observable.',
|
||||
expect: 'pass',
|
||||
engineInvariants: [
|
||||
{
|
||||
kind: 'gestureExecutionProfile',
|
||||
command: 'swipe',
|
||||
profile: 'endpoint-hold',
|
||||
because:
|
||||
'Maestro timed swipes are normalized to pan with the endpoint-hold execution profile, which the iOS runner delivers as a fast 100ms move followed by a hold.',
|
||||
},
|
||||
],
|
||||
divergenceMeans: 'The swipe behaves differently enough on one engine to fail the flow.',
|
||||
},
|
||||
// PARKED: tap-retry-if-no-change (flow kept at differential/flows/, invariant
|
||||
|
||||
@@ -263,7 +263,7 @@ test('command usage shows command flags without global flags', async () => {
|
||||
const help = await usageForCommand('swipe');
|
||||
if (help === null) throw new Error('Expected command help text');
|
||||
assert.match(help, /Quick coordinate fling with optional repeat pattern/);
|
||||
assert.match(help, /duration positional is accepted as a deprecated alias to pan/);
|
||||
assert.doesNotMatch(help, /duration positional is accepted as a deprecated alias to pan/);
|
||||
assert.match(help, /Command flags:/);
|
||||
assert.match(help, /--pattern one-way\|ping-pong/);
|
||||
assert.doesNotMatch(help, /Global flags:/);
|
||||
|
||||
@@ -282,7 +282,7 @@ Read-only and waits:
|
||||
Ambiguous find: add --first or --last. If info is not visible/exposed, report that gap instead of typing/searching/navigating to reveal it.
|
||||
|
||||
Navigation and gestures:
|
||||
Use scroll for lists; swipe for quick coordinate gestures/carousels; gesture pan for deliberate timed drags; gesture fling for fast directional throws. Historical swipe/fling duration arguments remain compatibility aliases to pan and are deprecated.
|
||||
Use scroll for lists; swipe for quick coordinate gestures/carousels; gesture pan for deliberate timed drags; gesture fling for fast directional throws.
|
||||
For fast macOS desktop list traversal, prefer fixed pixel wheel steps and batch them when no snapshot is needed between passes:
|
||||
agent-device scroll down --pixels 200 --duration-ms 50 --platform macos
|
||||
agent-device batch --steps '[{"command":"scroll","input":{"direction":"down","pixels":200,"durationMs":50}},{"command":"scroll","input":{"direction":"down","pixels":200,"durationMs":50}}]' --platform macos
|
||||
|
||||
@@ -752,8 +752,6 @@ export type LongPressOptions = DeviceCommandBaseOptions &
|
||||
export type SwipeOptions = DeviceCommandBaseOptions & {
|
||||
from: { x: number; y: number };
|
||||
to: { x: number; y: number };
|
||||
/** @deprecated Timed movement is a pan. Omit this field for swipe/fling semantics. */
|
||||
durationMs?: number;
|
||||
count?: number;
|
||||
pauseMs?: number;
|
||||
pattern?: SwipePattern;
|
||||
@@ -773,14 +771,10 @@ export type FlingOptions = DeviceCommandBaseOptions & {
|
||||
x: number;
|
||||
y: number;
|
||||
distance?: number;
|
||||
/** @deprecated Timed movement is a pan. Omit this field for fling semantics. */
|
||||
durationMs?: number;
|
||||
};
|
||||
|
||||
export type SwipeGestureOptions = DeviceCommandBaseOptions & {
|
||||
preset: SwipePreset;
|
||||
/** @deprecated Timed movement is a pan. Omit this field for swipe/fling semantics. */
|
||||
durationMs?: number;
|
||||
};
|
||||
|
||||
export type FocusOptions = DeviceCommandBaseOptions & {
|
||||
@@ -819,8 +813,6 @@ export type RotateGestureOptions = DeviceCommandBaseOptions & {
|
||||
degrees: number;
|
||||
x?: number;
|
||||
y?: number;
|
||||
/** @deprecated Rotation pacing is derived from degrees. */
|
||||
velocity?: number;
|
||||
};
|
||||
|
||||
export type TransformGestureOptions = DeviceCommandBaseOptions & TransformGestureParams;
|
||||
|
||||
@@ -69,7 +69,6 @@ export type CommandInput = Omit<InternalRequestOptions, 'batchSteps' | 'target'>
|
||||
update?: boolean;
|
||||
url?: string;
|
||||
value?: string;
|
||||
velocity?: number;
|
||||
x?: number;
|
||||
y?: number;
|
||||
} & Record<string, unknown>;
|
||||
|
||||
@@ -107,16 +107,15 @@ const interactionCliSchemas = {
|
||||
allowedFlags: [...postActionObservationCliFlags('longpress'), ...SELECTOR_SNAPSHOT_FLAGS],
|
||||
},
|
||||
swipe: {
|
||||
helpDescription:
|
||||
'Quick coordinate fling with optional repeat pattern. The historical duration positional is accepted as a deprecated alias to pan.',
|
||||
positionalArgs: ['x1', 'y1', 'x2', 'y2', 'durationMs?'],
|
||||
helpDescription: 'Quick coordinate fling with optional repeat pattern.',
|
||||
positionalArgs: ['x1', 'y1', 'x2', 'y2'],
|
||||
allowedFlags: ['count', 'pauseMs', 'pattern'],
|
||||
},
|
||||
gesture: {
|
||||
usageOverride: 'gesture <pan|fling|swipe|pinch|rotate|transform> ...',
|
||||
listUsageOverride: 'gesture <pan|fling|swipe|pinch|rotate|transform> ...',
|
||||
helpDescription:
|
||||
'Run touch gestures: pan <x> <y> <dx> <dy> [durationMs], fling <up|down|left|right> <x> <y> [distance], swipe <left|right|left-edge|right-edge>, pinch <scale> [x] [y], rotate <degrees> [x] [y], or transform <x> <y> <dx> <dy> <scale> <degrees> [durationMs]. Historical swipe/fling duration and rotate velocity arguments remain deprecated compatibility aliases. For command plans, output only command lines. Android transform verification should use all app-observable effects, for example wait text "pan changed yes", wait text "pinch changed yes", and wait text "rotate changed yes", not exact transform values.',
|
||||
'Run touch gestures: pan <x> <y> <dx> <dy> [durationMs], fling <up|down|left|right> <x> <y> [distance], swipe <left|right|left-edge|right-edge>, pinch <scale> [x] [y], rotate <degrees> [x] [y], or transform <x> <y> <dx> <dy> <scale> <degrees> [durationMs]. For command plans, output only command lines. Android transform verification should use all app-observable effects, for example wait text "pan changed yes", wait text "pinch changed yes", and wait text "rotate changed yes", not exact transform values.',
|
||||
summary: 'Run pan, fling, swipe, pinch, rotate, or transform gestures',
|
||||
positionalArgs: ['pan|fling|swipe|pinch|rotate|transform', 'args?'],
|
||||
allowsExtraPositionals: true,
|
||||
@@ -453,7 +452,6 @@ function toFlingOptions(input: FlingInput): FlingOptions {
|
||||
x: input.origin.x,
|
||||
y: input.origin.y,
|
||||
distance: input.distance,
|
||||
durationMs: input.durationMs,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -461,7 +459,6 @@ function toSwipeGestureOptions(input: SwipeGestureInput): SwipeGestureOptions {
|
||||
return {
|
||||
...commonToClientOptions(input),
|
||||
preset: input.preset,
|
||||
durationMs: input.durationMs,
|
||||
};
|
||||
}
|
||||
|
||||
@@ -480,7 +477,6 @@ function toRotateOptions(input: RotateInput): RotateGestureOptions {
|
||||
degrees: input.degrees,
|
||||
x: input.origin?.x,
|
||||
y: input.origin?.y,
|
||||
velocity: input.velocity,
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -5,7 +5,6 @@ test('swipe writes only typed daemon input', () => {
|
||||
const request = interactionDaemonWriters.swipe({
|
||||
from: { x: 10, y: 20 },
|
||||
to: { x: 30, y: 40 },
|
||||
durationMs: 300,
|
||||
count: 2,
|
||||
pauseMs: 10,
|
||||
pattern: 'ping-pong',
|
||||
@@ -15,7 +14,6 @@ test('swipe writes only typed daemon input', () => {
|
||||
expect(request.input).toEqual({
|
||||
from: { x: 10, y: 20 },
|
||||
to: { x: 30, y: 40 },
|
||||
durationMs: 300,
|
||||
count: 2,
|
||||
pauseMs: 10,
|
||||
pattern: 'ping-pong',
|
||||
|
||||
@@ -127,7 +127,6 @@ export const interactionDaemonWriters = {
|
||||
request(PUBLIC_COMMANDS.swipe, [], input, {
|
||||
from: input.from,
|
||||
to: input.to,
|
||||
durationMs: input.durationMs,
|
||||
count: input.count,
|
||||
pauseMs: input.pauseMs,
|
||||
pattern: input.pattern,
|
||||
|
||||
@@ -139,10 +139,6 @@ const longPressFields = {
|
||||
const swipeFields = {
|
||||
from: requiredField(pointField('Swipe start point.')),
|
||||
to: requiredField(pointField('Swipe end point.')),
|
||||
durationMs: integerField('Deprecated: timed movement is a pan; omit for swipe.', {
|
||||
min: 16,
|
||||
max: 10_000,
|
||||
}),
|
||||
count: integerField('Number of swipe repetitions.', { min: 1, max: SWIPE_REPETITION_MAX }),
|
||||
pauseMs: integerField('Pause between repeated swipes.', { min: 0, max: SWIPE_PAUSE_MAX_MS }),
|
||||
pattern: enumField(SWIPE_PATTERNS),
|
||||
@@ -221,11 +217,7 @@ const gestureFields = {
|
||||
distance: integerField('Fling distance.', { min: 0 }),
|
||||
scale: numberField('Pinch or transform scale.'),
|
||||
degrees: numberField('Rotation in degrees.'),
|
||||
velocity: numberField('Deprecated: rotation pacing is derived from degrees; must be non-zero.'),
|
||||
durationMs: integerField(
|
||||
'Pan/transform duration. Deprecated on swipe/fling; timed movement is a pan.',
|
||||
{ min: 16, max: 10_000 },
|
||||
),
|
||||
durationMs: integerField('Pan/transform duration.', { min: 16, max: 10_000 }),
|
||||
pointerCount: integerField('Pan touch pointer count (1 or 2).', { min: 1, max: 2 }),
|
||||
};
|
||||
|
||||
|
||||
@@ -76,7 +76,7 @@ test.each([
|
||||
'click',
|
||||
'snapshot',
|
||||
'snapshot',
|
||||
'swipe',
|
||||
'gesture',
|
||||
]);
|
||||
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
|
||||
},
|
||||
@@ -90,7 +90,7 @@ test('preserves mutation ordering after a failed dispatch is continued', async (
|
||||
baseReq: makeBaseRequest({ flags: { platform: 'ios', replayBackend: 'maestro' } }),
|
||||
invoke: async (request) => {
|
||||
requests.push(request);
|
||||
if (request.command === 'swipe' && swipeAttempts++ === 0) {
|
||||
if (request.command === 'gesture' && swipeAttempts++ === 0) {
|
||||
throw new Error('dispatch failed after delivery became possible');
|
||||
}
|
||||
return request.command === 'snapshot'
|
||||
@@ -132,10 +132,10 @@ test('preserves mutation ordering after a failed dispatch is continued', async (
|
||||
await swipe(1);
|
||||
|
||||
expect(requests.map(({ command }) => command)).toEqual([
|
||||
'swipe',
|
||||
'gesture',
|
||||
'snapshot',
|
||||
'snapshot',
|
||||
'swipe',
|
||||
'gesture',
|
||||
]);
|
||||
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
|
||||
});
|
||||
@@ -189,15 +189,15 @@ test('defers a pending mutation boundary across non-mutating runtime commands',
|
||||
env: {},
|
||||
invalidateObservation() {},
|
||||
});
|
||||
expect(requests.map(({ command }) => command)).toEqual(['swipe', 'screenshot']);
|
||||
expect(requests.map(({ command }) => command)).toEqual(['gesture', 'screenshot']);
|
||||
|
||||
await swipe(1);
|
||||
expect(requests.map(({ command }) => command)).toEqual([
|
||||
'swipe',
|
||||
'gesture',
|
||||
'screenshot',
|
||||
'snapshot',
|
||||
'snapshot',
|
||||
'swipe',
|
||||
'gesture',
|
||||
]);
|
||||
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
|
||||
});
|
||||
|
||||
@@ -74,11 +74,12 @@ test('delegates lifecycle and coordinate gestures through public daemon commands
|
||||
expect.objectContaining({ command: 'snapshot' }),
|
||||
expect.objectContaining({ command: 'snapshot' }),
|
||||
expect.objectContaining({
|
||||
command: 'swipe',
|
||||
command: 'gesture',
|
||||
positionals: [],
|
||||
input: {
|
||||
from: { x: 360, y: 400 },
|
||||
to: { x: 40, y: 400 },
|
||||
kind: 'pan',
|
||||
origin: { x: 360, y: 400 },
|
||||
delta: { x: -320, y: 0 },
|
||||
durationMs: 240,
|
||||
},
|
||||
}),
|
||||
@@ -117,16 +118,20 @@ test('uses the direct viewport without snapshot and pairs it with the nested ges
|
||||
});
|
||||
|
||||
expect(requests.at(-1)).toMatchObject({
|
||||
command: 'swipe',
|
||||
command: 'gesture',
|
||||
input: {
|
||||
from: { x: 370, y: 420 },
|
||||
to: { x: 50, y: 420 },
|
||||
kind: 'pan',
|
||||
origin: { x: 370, y: 420 },
|
||||
delta: { x: -320, y: 0 },
|
||||
durationMs: 300,
|
||||
},
|
||||
internal: { gestureViewport: viewport },
|
||||
internal: {
|
||||
gestureExecutionProfile: 'endpoint-hold',
|
||||
gestureViewport: viewport,
|
||||
},
|
||||
});
|
||||
expect(resolveGestureViewport).toHaveBeenCalledOnce();
|
||||
expect(requests.map(({ command }) => command)).toEqual(['swipe']);
|
||||
expect(requests.map(({ command }) => command)).toEqual(['gesture']);
|
||||
});
|
||||
|
||||
test('uses an observation as the baseline for a later mutation barrier', async () => {
|
||||
@@ -193,10 +198,10 @@ test('uses an observation as the baseline for a later mutation barrier', async (
|
||||
|
||||
expect(observation).toMatchObject({ matched: true });
|
||||
expect(requests.map(({ command }) => command)).toEqual([
|
||||
'swipe',
|
||||
'gesture',
|
||||
'snapshot',
|
||||
'snapshot',
|
||||
'swipe',
|
||||
'gesture',
|
||||
]);
|
||||
expect(port.readMetrics?.()).toEqual({
|
||||
hierarchyCaptures: 2,
|
||||
@@ -252,10 +257,10 @@ test('settles a gesture before dispatching another gesture', async () => {
|
||||
await swipe(1);
|
||||
|
||||
expect(requests.map(({ command }) => command)).toEqual([
|
||||
'swipe',
|
||||
'gesture',
|
||||
'snapshot',
|
||||
'snapshot',
|
||||
'swipe',
|
||||
'gesture',
|
||||
]);
|
||||
expect(clock.value).toBe(MAESTRO_OBSERVATION_POLL_MS);
|
||||
});
|
||||
|
||||
@@ -1,4 +1,8 @@
|
||||
import assert from 'node:assert/strict';
|
||||
import { describe, expect, test } from 'vitest';
|
||||
import { buildGesturePlan } from '../../../contracts/gesture-plan.ts';
|
||||
import { readGesturePayload } from '../../../contracts/gesture-input.ts';
|
||||
import { normalizePublicGesture } from '../../../contracts/gesture-normalization.ts';
|
||||
import {
|
||||
projectMaestroPublicOperation,
|
||||
type MaestroPublicOperation,
|
||||
@@ -95,11 +99,19 @@ describe('Maestro public operation projection', () => {
|
||||
viewport: { x: 0, y: 0, width: 100, height: 200 },
|
||||
},
|
||||
expected: {
|
||||
command: 'swipe',
|
||||
command: 'gesture',
|
||||
positionals: [],
|
||||
input: { from: { x: 90, y: 50 }, to: { x: 10, y: 50 }, durationMs: 400 },
|
||||
input: {
|
||||
kind: 'pan',
|
||||
origin: { x: 90, y: 50 },
|
||||
delta: { x: -80, y: 0 },
|
||||
durationMs: 400,
|
||||
},
|
||||
flags: { postGestureStabilization: false },
|
||||
internal: { gestureViewport: { x: 0, y: 0, width: 100, height: 200 } },
|
||||
internal: {
|
||||
gestureExecutionProfile: 'endpoint-hold',
|
||||
gestureViewport: { x: 0, y: 0, width: 100, height: 200 },
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -163,15 +175,40 @@ describe('Maestro public operation projection', () => {
|
||||
expect(projectMaestroPublicOperation(operation)).toEqual(expected);
|
||||
});
|
||||
|
||||
test('omits optional swipe and scroll fields', () => {
|
||||
test('preserves endpoint-hold for swipes without a viewport and omits scroll duration', () => {
|
||||
expect(
|
||||
projectMaestroPublicOperation({
|
||||
kind: 'swipe',
|
||||
gesture: { from: { x: 1, y: 2 }, to: { x: 3, y: 4 }, durationMs: 5 },
|
||||
}),
|
||||
).not.toHaveProperty('internal');
|
||||
).toMatchObject({
|
||||
command: 'gesture',
|
||||
internal: { gestureExecutionProfile: 'endpoint-hold' },
|
||||
});
|
||||
expect(projectMaestroPublicOperation({ kind: 'scroll', direction: 'up' })).not.toHaveProperty(
|
||||
'input',
|
||||
);
|
||||
});
|
||||
|
||||
test('a 400 ms Maestro swipe reaches plan execution with endpoint-hold profile', () => {
|
||||
const viewport = { x: 0, y: 0, width: 400, height: 800 };
|
||||
const projected = projectMaestroPublicOperation({
|
||||
kind: 'swipe',
|
||||
gesture: { from: { x: 360, y: 430 }, to: { x: 40, y: 430 }, durationMs: 400 },
|
||||
viewport,
|
||||
});
|
||||
const input = readGesturePayload(projected.input);
|
||||
const normalized = normalizePublicGesture(input);
|
||||
if (normalized.gesture.intent === 'pan' && projected.internal?.gestureExecutionProfile) {
|
||||
normalized.gesture.executionProfile = projected.internal.gestureExecutionProfile;
|
||||
}
|
||||
const plan = buildGesturePlan(
|
||||
normalized.gesture,
|
||||
projected.internal?.gestureViewport ?? viewport,
|
||||
);
|
||||
assert.equal(plan.topology, 'single');
|
||||
assert.equal(plan.intent, 'pan');
|
||||
assert.equal(plan.executionProfile, 'endpoint-hold');
|
||||
assert.equal(plan.durationMs, 400);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -201,7 +201,7 @@ function createDaemonMaestroRuntimeParts(options: CreateDaemonMaestroRuntimeOper
|
||||
);
|
||||
},
|
||||
gesture: async (input, context) => {
|
||||
await invokeMutation(
|
||||
const data = await invokeMutation(
|
||||
{
|
||||
kind: 'swipe',
|
||||
gesture: input,
|
||||
@@ -210,6 +210,7 @@ function createDaemonMaestroRuntimeParts(options: CreateDaemonMaestroRuntimeOper
|
||||
context,
|
||||
'deferred',
|
||||
);
|
||||
return data ? { data } : undefined;
|
||||
},
|
||||
inputText: async (input, context) => await typeTextAndSettle(input.text, context),
|
||||
eraseText: async (input, context) =>
|
||||
|
||||
@@ -158,12 +158,21 @@ function projectPointClick(
|
||||
function projectSwipe(
|
||||
operation: Extract<MaestroInputOperation, { kind: 'swipe' }>,
|
||||
): ProjectedMaestroPublicOperation {
|
||||
const { from, to, durationMs } = operation.gesture;
|
||||
return {
|
||||
command: 'swipe',
|
||||
command: 'gesture',
|
||||
positionals: [],
|
||||
input: operation.gesture,
|
||||
input: {
|
||||
kind: 'pan',
|
||||
origin: from,
|
||||
delta: { x: to.x - from.x, y: to.y - from.y },
|
||||
durationMs,
|
||||
},
|
||||
flags: { postGestureStabilization: false },
|
||||
...(operation.viewport ? { internal: { gestureViewport: operation.viewport } } : {}),
|
||||
internal: {
|
||||
gestureExecutionProfile: 'endpoint-hold',
|
||||
...(operation.viewport ? { gestureViewport: operation.viewport } : {}),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -97,6 +97,8 @@ export type MaestroRuntimeResult = {
|
||||
observation?: MaestroObservation;
|
||||
outputEnv?: Record<string, string>;
|
||||
artifactPaths?: string[];
|
||||
/** Daemon response data for gesture/profile evidence in replay traces. */
|
||||
data?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
export type MaestroRuntimeMetrics = {
|
||||
@@ -122,7 +124,11 @@ export type MaestroEngineEvent = {
|
||||
export type MaestroEngineObserver = {
|
||||
commandStarted?(event: MaestroEngineEvent): void;
|
||||
commandCompleted?(
|
||||
event: MaestroEngineEvent & { durationMs: number; runtimeMetrics?: MaestroRuntimeMetrics },
|
||||
event: MaestroEngineEvent & {
|
||||
durationMs: number;
|
||||
runtimeMetrics?: MaestroRuntimeMetrics;
|
||||
data?: Record<string, unknown>;
|
||||
},
|
||||
): void;
|
||||
commandFailed?(
|
||||
event: MaestroEngineEvent & {
|
||||
|
||||
@@ -75,13 +75,14 @@ async function executeObservedStep(
|
||||
};
|
||||
notifyMaestroObserver(() => state.options.observer?.commandStarted?.(event));
|
||||
try {
|
||||
await executeMaestroReplayPlanStep(step, state);
|
||||
const result = await executeMaestroReplayPlanStep(step, state);
|
||||
checkpointMaestroCancellation(state.options.signal);
|
||||
notifyMaestroObserver(() =>
|
||||
state.options.observer?.commandCompleted?.({
|
||||
...event,
|
||||
durationMs: now() - startedAt,
|
||||
...runtimeMetricsDelta(metricsBefore, state.port.readMetrics?.()),
|
||||
...(result?.data ? { data: result.data } : {}),
|
||||
}),
|
||||
);
|
||||
} catch (error) {
|
||||
|
||||
@@ -21,6 +21,7 @@ import type {
|
||||
MaestroObservationCondition,
|
||||
MaestroRuntimeCommand,
|
||||
MaestroRuntimePort,
|
||||
MaestroRuntimeResult,
|
||||
} from './engine-types.ts';
|
||||
import type {
|
||||
MaestroReplayPlan,
|
||||
@@ -50,37 +51,41 @@ type PlanStepFailure = {
|
||||
export async function executeMaestroReplayPlanStep(
|
||||
step: MaestroReplayPlanStep,
|
||||
state: MaestroReplayPlanExecutionState,
|
||||
): Promise<void> {
|
||||
): Promise<MaestroRuntimeResult | undefined> {
|
||||
checkpointMaestroCancellation(state.options.signal);
|
||||
try {
|
||||
await withStepScopes(step, state.context, async () => {
|
||||
await executeStep(step, state);
|
||||
return await withStepScopes(step, state.context, async () => {
|
||||
return await executeStep(step, state);
|
||||
});
|
||||
} catch (error) {
|
||||
throw asMaestroReplayPlanStepFailure(error, step);
|
||||
}
|
||||
}
|
||||
|
||||
async function executeStep(step: MaestroReplayPlanStep, state: MaestroReplayPlanExecutionState) {
|
||||
async function executeStep(
|
||||
step: MaestroReplayPlanStep,
|
||||
state: MaestroReplayPlanExecutionState,
|
||||
): Promise<MaestroRuntimeResult | undefined> {
|
||||
if (step.kind === 'command') {
|
||||
await executeOptionalCommand(step.command, step.appId, state);
|
||||
return;
|
||||
return await executeOptionalCommand(step.command, step.appId, state);
|
||||
}
|
||||
await executeOpaqueStep(step, state);
|
||||
return undefined;
|
||||
}
|
||||
|
||||
async function executeOptionalCommand(
|
||||
command: MaestroRuntimeCommand,
|
||||
appId: string | undefined,
|
||||
state: MaestroReplayPlanExecutionState,
|
||||
): Promise<void> {
|
||||
): Promise<MaestroRuntimeResult | undefined> {
|
||||
try {
|
||||
await executeCommand(command, appId, state);
|
||||
return await executeCommand(command, appId, state);
|
||||
} catch (error) {
|
||||
checkpointMaestroCancellation(state.options.signal);
|
||||
if (!isOptionalCommand(command) || !isMaestroTestFailure(error)) throw error;
|
||||
state.warnings.push(formatOptionalWarning(command, error));
|
||||
state.skipped += 1;
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -98,7 +103,7 @@ async function executeCommand(
|
||||
rawCommand: MaestroRuntimeCommand,
|
||||
appId: string | undefined,
|
||||
state: MaestroReplayPlanExecutionState,
|
||||
): Promise<void> {
|
||||
): Promise<MaestroRuntimeResult | undefined> {
|
||||
const command = resolveCommand(rawCommand, state.context);
|
||||
switch (command.kind) {
|
||||
case 'assertVisible':
|
||||
@@ -108,7 +113,7 @@ async function executeCommand(
|
||||
state,
|
||||
);
|
||||
state.executed += 1;
|
||||
return;
|
||||
return undefined;
|
||||
case 'assertNotVisible':
|
||||
await requireObservation(
|
||||
{ kind: 'notVisible', selector: command.target },
|
||||
@@ -116,7 +121,7 @@ async function executeCommand(
|
||||
state,
|
||||
);
|
||||
state.executed += 1;
|
||||
return;
|
||||
return undefined;
|
||||
case 'extendedWaitUntil':
|
||||
await requireObservation(
|
||||
readExtendedWaitCondition(command),
|
||||
@@ -124,9 +129,9 @@ async function executeCommand(
|
||||
state,
|
||||
);
|
||||
state.executed += 1;
|
||||
return;
|
||||
return undefined;
|
||||
default:
|
||||
await executeRuntimeCommand(command, appId, state);
|
||||
return await executeRuntimeCommand(command, appId, state);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -134,7 +139,7 @@ async function executeRuntimeCommand(
|
||||
command: MaestroRuntimeCommand,
|
||||
appId: string | undefined,
|
||||
state: MaestroReplayPlanExecutionState,
|
||||
): Promise<void> {
|
||||
): Promise<MaestroRuntimeResult> {
|
||||
const request = stripUndefined({
|
||||
command,
|
||||
env: state.context.values,
|
||||
@@ -150,6 +155,7 @@ async function executeRuntimeCommand(
|
||||
if (result.outputEnv) state.context.merge(result.outputEnv);
|
||||
result.artifactPaths?.forEach((entry) => state.artifacts.add(entry));
|
||||
state.executed += 1;
|
||||
return result;
|
||||
}
|
||||
|
||||
async function executeOpaqueStep(
|
||||
|
||||
@@ -430,6 +430,7 @@ function resultWithArtifacts(
|
||||
...optionalObservation(observation ?? operationObservation),
|
||||
...optionalOutputEnv(result?.outputEnv),
|
||||
...optionalArtifactPaths(defaultArtifacts, result?.artifactPaths),
|
||||
...optionalData(result?.data),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -464,6 +465,12 @@ function optionalArtifactPaths(
|
||||
return artifactPaths.length > 0 ? { artifactPaths } : {};
|
||||
}
|
||||
|
||||
function optionalData(
|
||||
data: Record<string, unknown> | undefined,
|
||||
): Partial<Pick<MaestroRuntimeResult, 'data'>> {
|
||||
return data && Object.keys(data).length > 0 ? { data: { ...data } } : {};
|
||||
}
|
||||
|
||||
async function resolveInputTarget(
|
||||
authored: MaestroGestureTarget,
|
||||
query: Pick<MaestroTargetQuery, 'purpose' | 'timeoutMs' | 'index' | 'childOf'>,
|
||||
|
||||
@@ -91,6 +91,8 @@ export type MaestroRuntimeOperationResult = {
|
||||
readonly observation?: MaestroObservation;
|
||||
readonly outputEnv?: Record<string, string>;
|
||||
readonly artifactPaths?: readonly string[];
|
||||
/** Daemon response data surfaced for replay-trace evidence. */
|
||||
readonly data?: Record<string, unknown>;
|
||||
};
|
||||
|
||||
export type MaestroRuntimeOperation<TInput> = (
|
||||
|
||||
@@ -10,13 +10,6 @@ test('structured gesture input rejects durations outside the planner range', ()
|
||||
delta: { x: 30, y: 40 },
|
||||
durationMs: 0,
|
||||
},
|
||||
{
|
||||
kind: 'fling',
|
||||
direction: 'left',
|
||||
origin: { x: 10, y: 20 },
|
||||
durationMs: 10_001,
|
||||
},
|
||||
{ kind: 'swipe', preset: 'left', durationMs: 0 },
|
||||
{
|
||||
kind: 'transform',
|
||||
origin: { x: 10, y: 20 },
|
||||
@@ -43,19 +36,14 @@ test('structured gesture input rejects durations outside the planner range', ()
|
||||
}
|
||||
});
|
||||
|
||||
test('deprecated rotate velocity accepts finite non-zero compatibility values', () => {
|
||||
for (const velocity of [-2.5, -1, 0.25, 3]) {
|
||||
assert.deepEqual(readGesturePayload({ kind: 'rotate', degrees: 45, velocity }), {
|
||||
kind: 'rotate',
|
||||
degrees: 45,
|
||||
origin: undefined,
|
||||
velocity,
|
||||
});
|
||||
}
|
||||
test('rotate does not accept velocity', () => {
|
||||
assert.deepEqual(readGesturePayload({ kind: 'rotate', degrees: 45 }), {
|
||||
kind: 'rotate',
|
||||
degrees: 45,
|
||||
origin: undefined,
|
||||
});
|
||||
|
||||
for (const velocity of [0, Number.NaN, Number.POSITIVE_INFINITY]) {
|
||||
assert.throws(() => readGesturePayload({ kind: 'rotate', degrees: 45, velocity }), {
|
||||
code: 'INVALID_ARGS',
|
||||
});
|
||||
}
|
||||
assert.throws(() => readGesturePayload({ kind: 'rotate', degrees: 45, velocity: 1 }), {
|
||||
code: 'INVALID_ARGS',
|
||||
});
|
||||
});
|
||||
|
||||
@@ -28,13 +28,11 @@ export type FlingGesturePayload = {
|
||||
direction: ScrollDirection;
|
||||
origin: Point;
|
||||
distance?: number;
|
||||
durationMs?: number;
|
||||
};
|
||||
|
||||
export type SwipeGesturePayload = {
|
||||
kind: 'swipe';
|
||||
preset: SwipePreset;
|
||||
durationMs?: number;
|
||||
};
|
||||
|
||||
export type PinchGesturePayload = {
|
||||
@@ -47,7 +45,6 @@ export type RotateGesturePayload = {
|
||||
kind: 'rotate';
|
||||
degrees: number;
|
||||
origin?: Point;
|
||||
velocity?: number;
|
||||
};
|
||||
|
||||
export type TransformGesturePayload = {
|
||||
@@ -85,19 +82,29 @@ export function readGesturePayload(input: unknown): GesturePayload {
|
||||
throw new AppError('INVALID_ARGS', 'pointerCount is supported only for gesture pan');
|
||||
}
|
||||
if (kind === 'fling') {
|
||||
if (record.durationMs !== undefined) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'gesture fling does not accept durationMs; use gesture pan for timed movement',
|
||||
);
|
||||
}
|
||||
return {
|
||||
kind,
|
||||
direction: readEnum(record, 'direction', SCROLL_DIRECTIONS),
|
||||
origin: readPoint(record, 'origin'),
|
||||
distance: readOptionalInteger(record, 'distance', { min: 0 }),
|
||||
durationMs: readOptionalGestureDuration(record),
|
||||
};
|
||||
}
|
||||
if (kind === 'swipe') {
|
||||
if (record.durationMs !== undefined) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'gesture swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
);
|
||||
}
|
||||
return {
|
||||
kind,
|
||||
preset: readEnum(record, 'preset', SWIPE_PRESETS),
|
||||
durationMs: readOptionalGestureDuration(record),
|
||||
};
|
||||
}
|
||||
if (kind === 'pinch') {
|
||||
@@ -108,11 +115,16 @@ export function readGesturePayload(input: unknown): GesturePayload {
|
||||
};
|
||||
}
|
||||
if (kind === 'rotate') {
|
||||
if (record.velocity !== undefined) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'gesture rotate does not accept velocity; rotation pacing derives from degrees',
|
||||
);
|
||||
}
|
||||
return {
|
||||
kind,
|
||||
degrees: readNumber(record, 'degrees'),
|
||||
origin: readOptionalPoint(record, 'origin'),
|
||||
velocity: readOptionalRotateVelocity(record),
|
||||
};
|
||||
}
|
||||
return {
|
||||
@@ -125,15 +137,6 @@ export function readGesturePayload(input: unknown): GesturePayload {
|
||||
};
|
||||
}
|
||||
|
||||
function readOptionalRotateVelocity(record: Record<string, unknown>): number | undefined {
|
||||
if (record.velocity === undefined) return undefined;
|
||||
const velocity = readNumber(record, 'velocity');
|
||||
if (velocity === 0) {
|
||||
throw new AppError('INVALID_ARGS', 'Expected velocity to be a non-zero finite number.');
|
||||
}
|
||||
return velocity;
|
||||
}
|
||||
|
||||
function readOptionalGestureDuration(record: Record<string, unknown>): number | undefined {
|
||||
const durationMs = readOptionalInteger(record, 'durationMs');
|
||||
if (durationMs === undefined) return undefined;
|
||||
|
||||
@@ -20,7 +20,7 @@ test('CLI and .ad positionals normalize at one explicit syntax seam', () => {
|
||||
|
||||
test('CLI and .ad coordinate swipes normalize at one explicit syntax seam', () => {
|
||||
assert.deepEqual(
|
||||
swipePayloadFromPositionals(['10', '20', '30', '40', '300'], {
|
||||
swipePayloadFromPositionals(['10', '20', '30', '40'], {
|
||||
count: 2,
|
||||
pauseMs: 5,
|
||||
pattern: 'ping-pong',
|
||||
@@ -28,7 +28,6 @@ test('CLI and .ad coordinate swipes normalize at one explicit syntax seam', () =
|
||||
{
|
||||
from: { x: 10, y: 20 },
|
||||
to: { x: 30, y: 40 },
|
||||
durationMs: 300,
|
||||
count: 2,
|
||||
pauseMs: 5,
|
||||
pattern: 'ping-pong',
|
||||
@@ -48,25 +47,14 @@ test('gesture recording codec round-trips structured requests', () => {
|
||||
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
|
||||
});
|
||||
|
||||
test('gesture recording codec preserves timed fling duration when distance is omitted', () => {
|
||||
test('gesture recording codec round-trips fling with distance', () => {
|
||||
const payload = {
|
||||
kind: 'fling' as const,
|
||||
direction: 'left' as const,
|
||||
origin: { x: 10, y: 20 },
|
||||
durationMs: 600,
|
||||
distance: 180,
|
||||
};
|
||||
const positionals = gesturePayloadToPositionals(payload);
|
||||
assert.deepEqual(positionals, ['fling', 'left', '10', '20', '180', '600']);
|
||||
assert.deepEqual(normalizePublicGesture(gesturePayloadFromPositionals(positionals)), {
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
origin: { x: 10, y: 20 },
|
||||
delta: { x: -180, y: 0 },
|
||||
durationMs: 600,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' }],
|
||||
});
|
||||
assert.deepEqual(gesturePayloadFromPositionals(gesturePayloadToPositionals(payload)), payload);
|
||||
});
|
||||
|
||||
test('pinch and rotate syntax rejects a partial origin', () => {
|
||||
@@ -78,71 +66,39 @@ test('pinch and rotate syntax rejects a partial origin', () => {
|
||||
});
|
||||
});
|
||||
|
||||
test('rotate serialization omits behaviorless velocity when no origin can delimit it', () => {
|
||||
assert.deepEqual(gesturePayloadToPositionals({ kind: 'rotate', degrees: 35, velocity: 2 }), [
|
||||
'rotate',
|
||||
'35',
|
||||
]);
|
||||
});
|
||||
|
||||
test('swipe is fling sugar unless legacy duration preserves swipe timing', () => {
|
||||
test('swipe is fling sugar', () => {
|
||||
assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left' }), {
|
||||
gesture: { intent: 'fling', preset: 'left' },
|
||||
deprecations: [],
|
||||
});
|
||||
assert.deepEqual(normalizePublicGesture({ kind: 'swipe', preset: 'left', durationMs: 400 }), {
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
preset: 'left',
|
||||
durationMs: 400,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }],
|
||||
});
|
||||
});
|
||||
|
||||
test('coordinate swipe duration preserves swipe timing and endpoints', () => {
|
||||
test('coordinate swipe is a quick fling', () => {
|
||||
assert.deepEqual(
|
||||
normalizePublicSwipeMotion({
|
||||
from: { x: 360, y: 400 },
|
||||
to: { x: 40, y: 400 },
|
||||
durationMs: 500,
|
||||
}),
|
||||
{
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
origin: { x: 360, y: 400 },
|
||||
delta: { x: -320, y: 0 },
|
||||
durationMs: 500,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [
|
||||
{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' },
|
||||
],
|
||||
gesture: { intent: 'fling', from: { x: 360, y: 400 }, to: { x: 40, y: 400 } },
|
||||
},
|
||||
);
|
||||
});
|
||||
|
||||
test('duration-bearing fling is an explicit compatibility alias for pan', () => {
|
||||
test('fling stays a fling', () => {
|
||||
assert.deepEqual(
|
||||
normalizePublicGesture({
|
||||
kind: 'fling',
|
||||
direction: 'left',
|
||||
origin: { x: 200, y: 300 },
|
||||
distance: 80,
|
||||
durationMs: 500,
|
||||
}),
|
||||
{
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
intent: 'fling',
|
||||
direction: 'left',
|
||||
origin: { x: 200, y: 300 },
|
||||
delta: { x: -80, y: 0 },
|
||||
durationMs: 500,
|
||||
executionProfile: 'endpoint-hold',
|
||||
distance: 80,
|
||||
},
|
||||
deprecations: [
|
||||
{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' },
|
||||
],
|
||||
},
|
||||
);
|
||||
});
|
||||
@@ -153,16 +109,7 @@ test('multi-touch aliases become constraints on transform motion', () => {
|
||||
origin: undefined,
|
||||
scale: 1.5,
|
||||
});
|
||||
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45, velocity: 2 }), {
|
||||
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45 }), {
|
||||
gesture: { intent: 'rotate', origin: undefined, degrees: -45 },
|
||||
deprecations: [
|
||||
{ rule: 'rotate-velocity', replacement: 'Rotation pacing is derived from degrees.' },
|
||||
],
|
||||
});
|
||||
assert.deepEqual(normalizePublicGesture({ kind: 'rotate', degrees: -45, velocity: -1.5 }), {
|
||||
gesture: { intent: 'rotate', origin: undefined, degrees: -45 },
|
||||
deprecations: [
|
||||
{ rule: 'rotate-velocity', replacement: 'Rotation pacing is derived from degrees.' },
|
||||
],
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,31 +1,37 @@
|
||||
import type { Point } from '../kernel/snapshot.ts';
|
||||
import { AppError } from '../kernel/errors.ts';
|
||||
import { gestureDirectionDelta } from './scroll-gesture.ts';
|
||||
import { readGesturePayload, type GesturePayload } from './gesture-input.ts';
|
||||
import { GESTURE_FLING_DEFAULT_DISTANCE } from './gesture-plan.ts';
|
||||
import type { GestureSemanticInput } from './gesture-plan-types.ts';
|
||||
|
||||
export type GestureCompatibilityRule = 'swipe-duration' | 'fling-duration' | 'rotate-velocity';
|
||||
|
||||
export type GestureDeprecation = {
|
||||
rule: GestureCompatibilityRule;
|
||||
replacement: string;
|
||||
};
|
||||
|
||||
export type NormalizedPublicGesture = {
|
||||
gesture: GestureSemanticInput;
|
||||
deprecations: GestureDeprecation[];
|
||||
};
|
||||
|
||||
export type SwipePayload = {
|
||||
from: Point;
|
||||
to: Point;
|
||||
durationMs?: number;
|
||||
count?: number;
|
||||
pauseMs?: number;
|
||||
pattern?: 'one-way' | 'ping-pong';
|
||||
};
|
||||
|
||||
function assertPositionalCount(
|
||||
args: readonly string[],
|
||||
max: number,
|
||||
usageMessage: string,
|
||||
): asserts args is { length: number } & typeof args {
|
||||
if (args.length > max) {
|
||||
throw new AppError('INVALID_ARGS', usageMessage);
|
||||
}
|
||||
}
|
||||
|
||||
function readOriginDelta(args: readonly string[]): { origin: Point; delta: Point } {
|
||||
return {
|
||||
origin: { x: Number(args[0]), y: Number(args[1]) },
|
||||
delta: { x: Number(args[2]), y: Number(args[3]) },
|
||||
};
|
||||
}
|
||||
|
||||
/** The explicit parser for the public CLI and `.ad` gesture syntax. */
|
||||
// fallow-ignore-next-line complexity
|
||||
export function gesturePayloadFromPositionals(
|
||||
@@ -35,50 +41,77 @@ export function gesturePayloadFromPositionals(
|
||||
const kind = positionals[0];
|
||||
const args = positionals.slice(1);
|
||||
switch (kind) {
|
||||
case 'pan':
|
||||
case 'pan': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
5,
|
||||
'gesture pan accepts at most 5 arguments: x y dx dy [durationMs]',
|
||||
);
|
||||
const { origin, delta } = readOriginDelta(args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
origin: { x: Number(args[0]), y: Number(args[1]) },
|
||||
delta: { x: Number(args[2]), y: Number(args[3]) },
|
||||
origin,
|
||||
delta,
|
||||
pointerCount,
|
||||
durationMs: optionalPositionNumber(args[4]),
|
||||
});
|
||||
case 'fling':
|
||||
}
|
||||
case 'fling': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
4,
|
||||
'gesture fling accepts at most 4 arguments: direction x y [distance]; for timed movement use gesture pan',
|
||||
);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
direction: args[0],
|
||||
origin: { x: Number(args[1]), y: Number(args[2]) },
|
||||
distance: optionalPositionNumber(args[3]),
|
||||
durationMs: optionalPositionNumber(args[4]),
|
||||
});
|
||||
case 'swipe':
|
||||
}
|
||||
case 'swipe': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
1,
|
||||
'gesture swipe accepts 1 argument: preset; for timed movement use gesture pan',
|
||||
);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
preset: args[0],
|
||||
durationMs: optionalPositionNumber(args[1]),
|
||||
});
|
||||
case 'pinch':
|
||||
}
|
||||
case 'pinch': {
|
||||
assertPositionalCount(args, 3, 'gesture pinch accepts at most 3 arguments: scale [x] [y]');
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
scale: Number(args[0]),
|
||||
origin: optionalOrigin(args[1], args[2]),
|
||||
});
|
||||
case 'rotate':
|
||||
}
|
||||
case 'rotate': {
|
||||
assertPositionalCount(args, 3, 'gesture rotate accepts at most 3 arguments: degrees [x] [y]');
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
degrees: Number(args[0]),
|
||||
origin: optionalOrigin(args[1], args[2]),
|
||||
velocity: optionalPositionNumber(args[3]),
|
||||
});
|
||||
case 'transform':
|
||||
}
|
||||
case 'transform': {
|
||||
assertPositionalCount(
|
||||
args,
|
||||
7,
|
||||
'gesture transform accepts at most 7 arguments: x y dx dy scale degrees [durationMs]',
|
||||
);
|
||||
const { origin, delta } = readOriginDelta(args);
|
||||
return readGesturePayload({
|
||||
kind,
|
||||
origin: { x: Number(args[0]), y: Number(args[1]) },
|
||||
delta: { x: Number(args[2]), y: Number(args[3]) },
|
||||
origin,
|
||||
delta,
|
||||
scale: Number(args[4]),
|
||||
degrees: Number(args[5]),
|
||||
durationMs: optionalPositionNumber(args[6]),
|
||||
});
|
||||
}
|
||||
default:
|
||||
return readGesturePayload({ kind });
|
||||
}
|
||||
@@ -97,25 +130,14 @@ export function gesturePayloadToPositionals(input: GesturePayload): string[] {
|
||||
input.durationMs,
|
||||
]);
|
||||
case 'fling':
|
||||
// `.ad` positionals cannot encode an empty distance slot. Preserve the
|
||||
// planner default when a deprecated duration must occupy the next slot.
|
||||
return compact([
|
||||
input.kind,
|
||||
input.direction,
|
||||
input.origin.x,
|
||||
input.origin.y,
|
||||
input.durationMs === undefined
|
||||
? input.distance
|
||||
: (input.distance ?? GESTURE_FLING_DEFAULT_DISTANCE),
|
||||
input.durationMs,
|
||||
]);
|
||||
return compact([input.kind, input.direction, input.origin.x, input.origin.y, input.distance]);
|
||||
case 'swipe':
|
||||
return compact([input.kind, input.preset, input.durationMs]);
|
||||
return [input.kind, input.preset];
|
||||
case 'pinch':
|
||||
return compact([input.kind, input.scale, input.origin?.x, input.origin?.y]);
|
||||
case 'rotate':
|
||||
return input.origin
|
||||
? compact([input.kind, input.degrees, input.origin.x, input.origin.y, input.velocity])
|
||||
? compact([input.kind, input.degrees, input.origin.x, input.origin.y])
|
||||
: [input.kind, String(input.degrees)];
|
||||
case 'transform':
|
||||
return compact([
|
||||
@@ -134,19 +156,24 @@ export function gesturePayloadToPositionals(input: GesturePayload): string[] {
|
||||
/** Parses the public CLI and `.ad` coordinate-swipe syntax. */
|
||||
export function swipePayloadFromPositionals(
|
||||
positionals: string[],
|
||||
options: Omit<SwipePayload, 'from' | 'to' | 'durationMs'> = {},
|
||||
options: Omit<SwipePayload, 'from' | 'to'> = {},
|
||||
): SwipePayload {
|
||||
if (positionals.length > 4) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'swipe accepts 4 arguments: x1 y1 x2 y2; for timed movement use gesture pan',
|
||||
);
|
||||
}
|
||||
return {
|
||||
from: { x: Number(positionals[0]), y: Number(positionals[1]) },
|
||||
to: { x: Number(positionals[2]), y: Number(positionals[3]) },
|
||||
...(positionals[4] === undefined ? {} : { durationMs: Number(positionals[4]) }),
|
||||
...(options.count === undefined ? {} : { count: options.count }),
|
||||
...(options.pauseMs === undefined ? {} : { pauseMs: options.pauseMs }),
|
||||
...(options.pattern === undefined ? {} : { pattern: options.pattern }),
|
||||
};
|
||||
}
|
||||
|
||||
/** The only public/deprecated gesture interpretation point. */
|
||||
/** The only public gesture interpretation point. */
|
||||
export function normalizePublicGesture(input: GesturePayload): NormalizedPublicGesture {
|
||||
switch (input.kind) {
|
||||
case 'pan':
|
||||
@@ -158,26 +185,8 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
|
||||
pointerCount: input.pointerCount,
|
||||
durationMs: input.durationMs,
|
||||
},
|
||||
deprecations: [],
|
||||
};
|
||||
case 'fling': {
|
||||
if (input.durationMs !== undefined) {
|
||||
return {
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
origin: input.origin,
|
||||
delta: gestureDirectionDelta(
|
||||
input.direction,
|
||||
input.distance ?? GESTURE_FLING_DEFAULT_DISTANCE,
|
||||
),
|
||||
durationMs: input.durationMs,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [
|
||||
{ rule: 'fling-duration', replacement: 'Use gesture pan for timed movement.' },
|
||||
],
|
||||
};
|
||||
}
|
||||
case 'fling':
|
||||
return {
|
||||
gesture: {
|
||||
intent: 'fling',
|
||||
@@ -185,40 +194,16 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
|
||||
origin: input.origin,
|
||||
distance: input.distance,
|
||||
},
|
||||
deprecations: [],
|
||||
};
|
||||
}
|
||||
case 'swipe':
|
||||
return input.durationMs === undefined
|
||||
? { gesture: { intent: 'fling', preset: input.preset }, deprecations: [] }
|
||||
: {
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
preset: input.preset,
|
||||
durationMs: input.durationMs,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [
|
||||
{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' },
|
||||
],
|
||||
};
|
||||
return { gesture: { intent: 'fling', preset: input.preset } };
|
||||
case 'pinch':
|
||||
return {
|
||||
gesture: { intent: 'pinch', origin: input.origin, scale: input.scale },
|
||||
deprecations: [],
|
||||
};
|
||||
case 'rotate':
|
||||
return {
|
||||
gesture: { intent: 'rotate', origin: input.origin, degrees: input.degrees },
|
||||
deprecations:
|
||||
input.velocity === undefined
|
||||
? []
|
||||
: [
|
||||
{
|
||||
rule: 'rotate-velocity',
|
||||
replacement: 'Rotation pacing is derived from degrees.',
|
||||
},
|
||||
],
|
||||
};
|
||||
case 'transform':
|
||||
return {
|
||||
@@ -230,7 +215,6 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
|
||||
degrees: input.degrees,
|
||||
durationMs: input.durationMs,
|
||||
},
|
||||
deprecations: [],
|
||||
};
|
||||
}
|
||||
}
|
||||
@@ -238,23 +222,9 @@ export function normalizePublicGesture(input: GesturePayload): NormalizedPublicG
|
||||
export function normalizePublicSwipeMotion(input: {
|
||||
from: Point;
|
||||
to: Point;
|
||||
durationMs?: number;
|
||||
}): NormalizedPublicGesture {
|
||||
if (input.durationMs === undefined) {
|
||||
return {
|
||||
gesture: { intent: 'fling', from: input.from, to: input.to },
|
||||
deprecations: [],
|
||||
};
|
||||
}
|
||||
return {
|
||||
gesture: {
|
||||
intent: 'pan',
|
||||
origin: input.from,
|
||||
delta: { x: input.to.x - input.from.x, y: input.to.y - input.from.y },
|
||||
durationMs: input.durationMs,
|
||||
executionProfile: 'endpoint-hold',
|
||||
},
|
||||
deprecations: [{ rule: 'swipe-duration', replacement: 'Use gesture pan for timed movement.' }],
|
||||
gesture: { intent: 'fling', from: input.from, to: input.to },
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
@@ -28,12 +28,6 @@ export type GestureSemanticInput =
|
||||
durationMs?: number;
|
||||
executionProfile?: GestureExecutionProfile;
|
||||
}
|
||||
| {
|
||||
intent: 'pan';
|
||||
preset: SwipePreset;
|
||||
durationMs: number;
|
||||
executionProfile?: GestureExecutionProfile;
|
||||
}
|
||||
| { intent: 'pinch'; origin?: Point; scale: number }
|
||||
| { intent: 'rotate'; origin?: Point; degrees: number }
|
||||
| {
|
||||
|
||||
@@ -20,7 +20,7 @@ export * from './gesture-plan-types.ts';
|
||||
|
||||
export const GESTURE_SAMPLE_INTERVAL_MS = 16;
|
||||
export const GESTURE_INITIAL_ANGLE_DEGREES = -90;
|
||||
export const GESTURE_FLING_DEFAULT_DISTANCE = 180;
|
||||
const GESTURE_FLING_DEFAULT_DISTANCE = 180;
|
||||
|
||||
const GESTURE_INITIAL_SPAN_RATIO = 0.25;
|
||||
const GESTURE_PINCH_INITIAL_SPAN_RATIO = 0.4;
|
||||
@@ -182,18 +182,6 @@ function buildPanPlan(
|
||||
DEFAULT_PAN_DURATION_MS,
|
||||
'gesture pan durationMs',
|
||||
);
|
||||
if ('preset' in input) {
|
||||
const { from, to } = presetGestureEndpoints(input.preset, viewport);
|
||||
return buildSinglePointerPlan(
|
||||
'pan',
|
||||
from,
|
||||
to,
|
||||
durationMs,
|
||||
viewport,
|
||||
input.executionProfile ?? 'timed-pan',
|
||||
profile,
|
||||
);
|
||||
}
|
||||
if ((input.pointerCount ?? 1) === 1) {
|
||||
const start = finitePoint(input.origin, 'gesture pan origin');
|
||||
const delta = finitePoint(input.delta, 'gesture pan delta');
|
||||
|
||||
@@ -49,14 +49,14 @@ describe('single-pointer plans', () => {
|
||||
|
||||
test('planner resolves swipe presets inside the active viewport', () => {
|
||||
const viewport = { x: 20, y: 30, width: 400, height: 800 };
|
||||
const fling = buildGesturePlan({ intent: 'fling', preset: 'left' }, viewport);
|
||||
const pan = buildGesturePlan({ intent: 'pan', preset: 'right', durationMs: 400 }, viewport);
|
||||
const leftFling = buildGesturePlan({ intent: 'fling', preset: 'left' }, viewport);
|
||||
const rightFling = buildGesturePlan({ intent: 'fling', preset: 'right' }, viewport);
|
||||
|
||||
assert.deepEqual(fling.pointers[0].samples[0]?.point, { x: 360, y: 430 });
|
||||
assert.deepEqual(fling.pointers[0].samples.at(-1)?.point, { x: 80, y: 430 });
|
||||
assert.equal(pan.durationMs, 400);
|
||||
assert.deepEqual(pan.pointers[0].samples[0]?.point, { x: 80, y: 430 });
|
||||
assert.deepEqual(pan.pointers[0].samples.at(-1)?.point, { x: 360, y: 430 });
|
||||
assert.deepEqual(leftFling.pointers[0].samples[0]?.point, { x: 360, y: 430 });
|
||||
assert.deepEqual(leftFling.pointers[0].samples.at(-1)?.point, { x: 80, y: 430 });
|
||||
assert.equal(rightFling.durationMs, 100);
|
||||
assert.deepEqual(rightFling.pointers[0].samples[0]?.point, { x: 80, y: 430 });
|
||||
assert.deepEqual(rightFling.pointers[0].samples.at(-1)?.point, { x: 360, y: 430 });
|
||||
});
|
||||
|
||||
test('single-pointer pan preserves its execution profile', () => {
|
||||
|
||||
@@ -152,7 +152,7 @@ export function requireGestureSupported(input: GestureSemanticInput, device: Dev
|
||||
throw unsupportedGesture(input, gesturePlatformMessage(input, device));
|
||||
}
|
||||
// Linux can preserve public coordinate/preset swipe through its drag primitive, but cannot
|
||||
// honor the velocity semantics authored by `gesture fling`.
|
||||
// honor the speed semantics authored by `gesture fling`.
|
||||
if (input.intent === 'fling' && 'direction' in input && device.platform === 'linux') {
|
||||
throw unsupportedGesture(input, 'gesture fling is not supported on Linux');
|
||||
}
|
||||
|
||||
@@ -114,7 +114,7 @@ test('swipe rejects repetition inputs that can monopolize the request', async ()
|
||||
message: 'Expected pauseMs to be at most 10000.',
|
||||
},
|
||||
{
|
||||
input: { count: 7, durationMs: 10_000 },
|
||||
input: { count: 7, pauseMs: 10_000 },
|
||||
message: 'Swipe series must fit within 60000ms.',
|
||||
},
|
||||
];
|
||||
|
||||
@@ -513,7 +513,7 @@ test('.ad replay normalizes resolved gesture and swipe syntax into structured da
|
||||
script: [
|
||||
'env X=10',
|
||||
'gesture pan ${X} 20 30 40 500 --pointer-count 2',
|
||||
'swipe ${X} 20 30 40 300 --count 2 --pause-ms 5 --pattern ping-pong',
|
||||
'swipe ${X} 20 30 40 --count 2 --pause-ms 5 --pattern ping-pong',
|
||||
'',
|
||||
].join('\n'),
|
||||
});
|
||||
@@ -543,7 +543,6 @@ test('.ad replay normalizes resolved gesture and swipe syntax into structured da
|
||||
input: {
|
||||
from: { x: 10, y: 20 },
|
||||
to: { x: 30, y: 40 },
|
||||
durationMs: 300,
|
||||
count: 2,
|
||||
pauseMs: 5,
|
||||
pattern: 'ping-pong',
|
||||
@@ -1606,7 +1605,15 @@ test('runReplayScriptFile resolves Maestro swipe.label from a labeled element re
|
||||
calls.map((call) => [call.command, call.input]),
|
||||
[
|
||||
['snapshot', undefined],
|
||||
['swipe', { from: { x: 110, y: 250 }, to: { x: 110, y: 40 }, durationMs: 400 }],
|
||||
[
|
||||
'gesture',
|
||||
{
|
||||
kind: 'pan',
|
||||
origin: { x: 110, y: 250 },
|
||||
delta: { x: 0, y: -210 },
|
||||
durationMs: 400,
|
||||
},
|
||||
],
|
||||
],
|
||||
);
|
||||
});
|
||||
@@ -1664,7 +1671,15 @@ test('runReplayScriptFile keeps Maestro swipe.label anchored to the matched labe
|
||||
calls.map((call) => [call.command, call.input]),
|
||||
[
|
||||
['snapshot', undefined],
|
||||
['swipe', { from: { x: 100, y: 124 }, to: { x: 100, y: 14 }, durationMs: 400 }],
|
||||
[
|
||||
'gesture',
|
||||
{
|
||||
kind: 'pan',
|
||||
origin: { x: 100, y: 124 },
|
||||
delta: { x: 0, y: -110 },
|
||||
durationMs: 400,
|
||||
},
|
||||
],
|
||||
],
|
||||
);
|
||||
});
|
||||
@@ -1717,16 +1732,25 @@ test('runReplayScriptFile resolves Maestro screen swipes from the direct viewpor
|
||||
calls.map((call) => [call.command, call.input]),
|
||||
[
|
||||
[
|
||||
'swipe',
|
||||
'gesture',
|
||||
{
|
||||
from: { x: 340, y: 400 },
|
||||
to: { x: 60, y: 400 },
|
||||
kind: 'pan',
|
||||
origin: { x: 340, y: 400 },
|
||||
delta: { x: -280, y: 0 },
|
||||
durationMs: 300,
|
||||
},
|
||||
],
|
||||
['snapshot', undefined],
|
||||
['snapshot', undefined],
|
||||
['swipe', { from: { x: 360, y: 400 }, to: { x: 40, y: 400 }, durationMs: 300 }],
|
||||
[
|
||||
'gesture',
|
||||
{
|
||||
kind: 'pan',
|
||||
origin: { x: 360, y: 400 },
|
||||
delta: { x: -320, y: 0 },
|
||||
durationMs: 300,
|
||||
},
|
||||
],
|
||||
],
|
||||
);
|
||||
});
|
||||
@@ -1779,16 +1803,25 @@ test('runReplayScriptFile delegates Android directional swipes and preserves per
|
||||
calls.map((call) => [call.command, call.input]),
|
||||
[
|
||||
[
|
||||
'swipe',
|
||||
'gesture',
|
||||
{
|
||||
from: { x: 340, y: 400 },
|
||||
to: { x: 60, y: 400 },
|
||||
kind: 'pan',
|
||||
origin: { x: 340, y: 400 },
|
||||
delta: { x: -280, y: 0 },
|
||||
durationMs: 300,
|
||||
},
|
||||
],
|
||||
['snapshot', undefined],
|
||||
['snapshot', undefined],
|
||||
['swipe', { from: { x: 360, y: 400 }, to: { x: 40, y: 400 }, durationMs: 300 }],
|
||||
[
|
||||
'gesture',
|
||||
{
|
||||
kind: 'pan',
|
||||
origin: { x: 360, y: 400 },
|
||||
delta: { x: -320, y: 0 },
|
||||
durationMs: 300,
|
||||
},
|
||||
],
|
||||
],
|
||||
);
|
||||
});
|
||||
|
||||
@@ -3,7 +3,6 @@ import {
|
||||
gesturePayloadToPositionals,
|
||||
normalizePublicGesture,
|
||||
normalizePublicSwipeMotion,
|
||||
type GestureDeprecation,
|
||||
type SwipePayload,
|
||||
} from '../../contracts/gesture-normalization.ts';
|
||||
import { requireGestureSupported } from '../../core/capabilities.ts';
|
||||
@@ -16,6 +15,7 @@ import {
|
||||
import { AppError, normalizeError } from '../../kernel/errors.ts';
|
||||
import { readOptionalInteger } from '../../kernel/input-validation.ts';
|
||||
import type { Point } from '../../kernel/snapshot.ts';
|
||||
import type { GestureSemanticInput } from '../../contracts/gesture-plan-types.ts';
|
||||
import { isActiveProviderDevice } from '../../provider-device-runtime.ts';
|
||||
import { sleep } from '../../utils/timeouts.ts';
|
||||
import type { DaemonResponse, SessionState } from '../types.ts';
|
||||
@@ -46,6 +46,9 @@ export async function dispatchGestureViaRuntime(
|
||||
return await dispatchGestureInteraction(params, 'gesture', async (session) => {
|
||||
const input = readGesturePayload(params.req.input);
|
||||
const normalized = normalizePublicGesture(input);
|
||||
if (normalized.gesture.intent === 'pan' && params.req.internal?.gestureExecutionProfile) {
|
||||
normalized.gesture.executionProfile = params.req.internal.gestureExecutionProfile;
|
||||
}
|
||||
requireGestureSupported(normalized.gesture, session.device);
|
||||
const result = await createGestureRuntime(params).interactions.gesture({
|
||||
session: params.sessionName,
|
||||
@@ -55,7 +58,9 @@ export async function dispatchGestureViaRuntime(
|
||||
return {
|
||||
positionals: gesturePayloadToPositionals(input),
|
||||
flags: gestureReplayFlags(input, params.req.flags),
|
||||
responseData: gestureResponseData(result, normalized.deprecations),
|
||||
responseData: gestureResponseData(result, {
|
||||
executionProfile: resolveExecutionProfile(normalized.gesture),
|
||||
}),
|
||||
...(input.kind === 'pinch' ? { recordingResultExtra: { scale: input.scale } } : {}),
|
||||
};
|
||||
});
|
||||
@@ -75,7 +80,7 @@ export async function dispatchSwipeViaRuntime(
|
||||
return {
|
||||
positionals: swipeReplayPositionals(input),
|
||||
flags: params.req.flags,
|
||||
responseData: gestureResponseData(result, result.deprecations ?? [], {
|
||||
responseData: gestureResponseData(result, {
|
||||
from: input.from,
|
||||
to: input.to,
|
||||
x1: input.from.x,
|
||||
@@ -84,6 +89,7 @@ export async function dispatchSwipeViaRuntime(
|
||||
y2: input.to.y,
|
||||
effectiveDurationMs: result.durationMs,
|
||||
timingMode: 'direct',
|
||||
executionProfile: 'endpoint-hold',
|
||||
count,
|
||||
pauseMs,
|
||||
pattern,
|
||||
@@ -147,11 +153,18 @@ async function dispatchGestureInteraction(
|
||||
}
|
||||
}
|
||||
|
||||
function resolveExecutionProfile(gesture: GestureSemanticInput): string | undefined {
|
||||
if (gesture.intent === 'fling') return 'endpoint-hold';
|
||||
if (gesture.intent === 'pan') return gesture.executionProfile ?? 'timed-pan';
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function gestureResponseData(
|
||||
result: GestureRuntimeResult,
|
||||
deprecations: readonly GestureDeprecation[],
|
||||
extra: Record<string, unknown> = {},
|
||||
): Record<string, unknown> {
|
||||
const executionProfile =
|
||||
typeof extra.executionProfile === 'string' ? extra.executionProfile : undefined;
|
||||
return {
|
||||
kind: result.kind,
|
||||
durationMs: result.durationMs,
|
||||
@@ -160,7 +173,14 @@ function gestureResponseData(
|
||||
to: result.to,
|
||||
...(result.backendResult ?? {}),
|
||||
...extra,
|
||||
...(deprecations.length > 0 ? { deprecations } : {}),
|
||||
...(executionProfile
|
||||
? {
|
||||
timing: {
|
||||
executionProfile,
|
||||
gestureDurationMs: result.durationMs,
|
||||
},
|
||||
}
|
||||
: {}),
|
||||
message: result.message,
|
||||
};
|
||||
}
|
||||
@@ -170,6 +190,12 @@ function readSwipeInput(input: unknown): SwipePayload {
|
||||
throw new AppError('INVALID_ARGS', 'swipe requires structured object input');
|
||||
}
|
||||
const record = input as Record<string, unknown>;
|
||||
if (record.durationMs !== undefined) {
|
||||
throw new AppError(
|
||||
'INVALID_ARGS',
|
||||
'swipe does not accept durationMs; use gesture pan for timed movement',
|
||||
);
|
||||
}
|
||||
const pattern = record.pattern;
|
||||
if (pattern !== undefined && pattern !== 'one-way' && pattern !== 'ping-pong') {
|
||||
throw new AppError('INVALID_ARGS', 'swipe pattern must be one-way or ping-pong');
|
||||
@@ -177,7 +203,6 @@ function readSwipeInput(input: unknown): SwipePayload {
|
||||
const payload: SwipePayload = {
|
||||
from: readSwipePoint(record.from, 'swipe from'),
|
||||
to: readSwipePoint(record.to, 'swipe to'),
|
||||
durationMs: readOptionalInteger(record, 'durationMs', { min: 16, max: 10_000 }),
|
||||
count: readOptionalInteger(record, 'count', { min: 1, max: SWIPE_REPETITION_MAX }),
|
||||
pauseMs: readOptionalInteger(record, 'pauseMs', { min: 0, max: SWIPE_PAUSE_MAX_MS }),
|
||||
pattern,
|
||||
@@ -189,7 +214,7 @@ function readSwipeInput(input: unknown): SwipePayload {
|
||||
function assertSwipeSeriesFitsRequest(input: SwipePayload): void {
|
||||
const count = input.count ?? 1;
|
||||
const pauseMs = input.pauseMs ?? 0;
|
||||
const gestureDurationMs = input.durationMs ?? GESTURE_FLING_DURATION_MS;
|
||||
const gestureDurationMs = GESTURE_FLING_DURATION_MS;
|
||||
const scheduledDurationMs = count * gestureDurationMs + Math.max(0, count - 1) * pauseMs;
|
||||
if (scheduledDurationMs <= SWIPE_SERIES_MAX_SCHEDULED_DURATION_MS) return;
|
||||
throw new AppError(
|
||||
@@ -200,7 +225,7 @@ function assertSwipeSeriesFitsRequest(input: SwipePayload): void {
|
||||
pauseMs,
|
||||
gestureDurationMs,
|
||||
scheduledDurationMs,
|
||||
hint: 'Reduce --count, --pause-ms, or the deprecated swipe duration.',
|
||||
hint: 'Reduce --count or --pause-ms.',
|
||||
},
|
||||
);
|
||||
}
|
||||
@@ -220,13 +245,7 @@ function readSwipePoint(value: unknown, field: string): Point {
|
||||
}
|
||||
|
||||
function swipeReplayPositionals(input: SwipePayload): string[] {
|
||||
return [
|
||||
String(input.from.x),
|
||||
String(input.from.y),
|
||||
String(input.to.x),
|
||||
String(input.to.y),
|
||||
...(input.durationMs === undefined ? [] : [String(input.durationMs)]),
|
||||
];
|
||||
return [String(input.from.x), String(input.from.y), String(input.to.x), String(input.to.y)];
|
||||
}
|
||||
|
||||
async function runSwipeRepetitions(
|
||||
@@ -238,7 +257,6 @@ async function runSwipeRepetitions(
|
||||
pattern: 'one-way' | 'ping-pong',
|
||||
) {
|
||||
let result: Awaited<ReturnType<typeof runtime.interactions.gesture>> | undefined;
|
||||
const deprecations = normalizePublicSwipeMotion(input).deprecations;
|
||||
for (let index = 0; index < count; index += 1) {
|
||||
const normalized = normalizePublicSwipeMotion(swipeMotionAtIndex(input, pattern, index));
|
||||
result = await runtime.interactions.gesture({
|
||||
@@ -249,10 +267,7 @@ async function runSwipeRepetitions(
|
||||
if (pauseMs > 0 && index + 1 < count) await sleep(pauseMs);
|
||||
}
|
||||
if (!result) throw new Error('Swipe orchestration did not execute a gesture.');
|
||||
return {
|
||||
...result,
|
||||
...(deprecations.length > 0 ? { deprecations } : {}),
|
||||
};
|
||||
return result;
|
||||
}
|
||||
|
||||
function swipeMotionAtIndex(
|
||||
|
||||
@@ -59,17 +59,20 @@ function traceStopEvent(
|
||||
event: MaestroEngineEvent & {
|
||||
durationMs: number;
|
||||
runtimeMetrics?: MaestroRuntimeMetrics;
|
||||
data?: Record<string, unknown>;
|
||||
error?: unknown;
|
||||
},
|
||||
): MaestroEngineEvent & {
|
||||
durationMs: number;
|
||||
runtimeMetrics?: MaestroRuntimeMetrics;
|
||||
data?: Record<string, unknown>;
|
||||
error?: unknown;
|
||||
} {
|
||||
return stripUndefined({
|
||||
...(start ?? event),
|
||||
durationMs: event.durationMs,
|
||||
runtimeMetrics: event.runtimeMetrics,
|
||||
data: event.data,
|
||||
error: event.error,
|
||||
});
|
||||
}
|
||||
@@ -116,10 +119,12 @@ function appendMaestroTraceStop(
|
||||
event: MaestroEngineEvent & {
|
||||
durationMs: number;
|
||||
runtimeMetrics?: MaestroRuntimeMetrics;
|
||||
data?: Record<string, unknown>;
|
||||
error?: unknown;
|
||||
},
|
||||
ok: boolean,
|
||||
): void {
|
||||
const resultTiming = maestroResultTiming(event);
|
||||
appendReplayTraceEvent(tracePath, {
|
||||
type: 'replay_action_stop',
|
||||
ts: new Date().toISOString(),
|
||||
@@ -132,7 +137,29 @@ function appendMaestroTraceStop(
|
||||
command: formatMaestroCommandProgress(event.command).command,
|
||||
ok,
|
||||
durationMs: event.durationMs,
|
||||
...(event.runtimeMetrics ? { resultTiming: event.runtimeMetrics } : {}),
|
||||
...(Object.keys(resultTiming).length > 0 ? { resultTiming } : {}),
|
||||
...(!ok && event.error instanceof AppError ? { errorCode: event.error.code } : {}),
|
||||
});
|
||||
}
|
||||
|
||||
function maestroResultTiming(
|
||||
event: MaestroEngineEvent & {
|
||||
runtimeMetrics?: MaestroRuntimeMetrics;
|
||||
data?: Record<string, unknown>;
|
||||
},
|
||||
): Record<string, unknown> {
|
||||
const timing = isPlainRecord(event.data?.timing) ? event.data.timing : undefined;
|
||||
return stripUndefined({
|
||||
...(event.runtimeMetrics ?? {}),
|
||||
...(typeof timing?.executionProfile === 'string'
|
||||
? { executionProfile: timing.executionProfile }
|
||||
: {}),
|
||||
...(typeof timing?.gestureDurationMs === 'number'
|
||||
? { gestureDurationMs: timing.gestureDurationMs }
|
||||
: {}),
|
||||
});
|
||||
}
|
||||
|
||||
function isPlainRecord(value: unknown): value is Record<string, unknown> {
|
||||
return !!value && typeof value === 'object' && !Array.isArray(value);
|
||||
}
|
||||
|
||||
@@ -18,6 +18,7 @@ import type { RecordingScope } from '../contracts/recording-scope.ts';
|
||||
import type { DeviceInfo, Platform, PlatformSelector } from '../kernel/device.ts';
|
||||
import type { ExecBackgroundResult, ExecResult } from '../utils/exec.ts';
|
||||
import type { Rect, SnapshotState } from '../kernel/snapshot.ts';
|
||||
import type { GestureExecutionProfile } from '../contracts/gesture-plan-types.ts';
|
||||
// Type-only import; erased at runtime. ref-frame.ts imports SessionState from
|
||||
// here, so this back-edge must stay type-only to avoid a runtime cycle.
|
||||
import type { RefFrameScope, RefFrameState } from './ref-frame.ts';
|
||||
@@ -65,6 +66,8 @@ type DaemonRequestInternal = {
|
||||
closeAppOnly?: boolean;
|
||||
/** Provider-owned viewport already resolved while normalizing a nested gesture command. */
|
||||
gestureViewport?: Rect;
|
||||
/** Maestro-compat execution profile for timed coordinate swipes projected to `gesture pan`. */
|
||||
gestureExecutionProfile?: GestureExecutionProfile;
|
||||
/**
|
||||
* ADR 0012 step 4 post-resolution guard: the verified target member's
|
||||
* normalized local identity AND structural denotation (document order +
|
||||
|
||||
@@ -170,16 +170,13 @@ test('MCP gesture tool exposes and forwards two-finger pan topology', async () =
|
||||
]);
|
||||
});
|
||||
|
||||
test('MCP gesture compatibility metadata accepts numeric rotate velocity', () => {
|
||||
test('MCP gesture metadata exposes pan/transform duration', () => {
|
||||
const gesture = listCommandTools().find((tool) => tool.name === 'gesture');
|
||||
assert.ok(gesture);
|
||||
assert.deepEqual(gesture.inputSchema.properties?.velocity, {
|
||||
type: 'number',
|
||||
description: 'Deprecated: rotation pacing is derived from degrees; must be non-zero.',
|
||||
});
|
||||
assert.equal(gesture.inputSchema.properties?.velocity, undefined);
|
||||
assert.deepEqual(gesture.inputSchema.properties?.durationMs, {
|
||||
type: 'integer',
|
||||
description: 'Pan/transform duration. Deprecated on swipe/fling; timed movement is a pan.',
|
||||
description: 'Pan/transform duration.',
|
||||
minimum: 16,
|
||||
maximum: 10_000,
|
||||
});
|
||||
|
||||
@@ -99,9 +99,11 @@ test('Provider-backed integration Android touch provider handles multi-touch ges
|
||||
...world.selection,
|
||||
});
|
||||
|
||||
await client.interactions.swipe({
|
||||
from: { x: 340, y: 400 },
|
||||
to: { x: 60, y: 400 },
|
||||
await client.interactions.pan({
|
||||
x: 340,
|
||||
y: 400,
|
||||
dx: -280,
|
||||
dy: 0,
|
||||
durationMs: 300,
|
||||
...world.selection,
|
||||
});
|
||||
|
||||
@@ -121,11 +121,12 @@ test('Provider-backed integration Linux desktop flow uses semantic desktop and i
|
||||
positionals: ['42', '84', '1'],
|
||||
},
|
||||
{
|
||||
name: 'swipe coordinates',
|
||||
command: 'swipe',
|
||||
name: 'pan coordinates',
|
||||
command: 'gesture',
|
||||
input: {
|
||||
from: { x: 10, y: 20 },
|
||||
to: { x: 30, y: 40 },
|
||||
kind: 'pan',
|
||||
origin: { x: 10, y: 20 },
|
||||
delta: { x: 20, y: 20 },
|
||||
durationMs: 16,
|
||||
},
|
||||
expectData: {
|
||||
|
||||
@@ -358,16 +358,12 @@ On Android, `fill` also verifies text and treats IME-owned capture as a terminal
|
||||
Android text entry is owned by `agent-device`: provider-native injection when available, then chunk-safe ASCII shell input. Do not switch to raw `adb`, clipboard, or paste as an agent fallback. If non-ASCII is unsupported in the current backend, report the tool/device gap.
|
||||
`click --button secondary` is the desktop context-menu flow on macOS.
|
||||
`click --button middle` is reserved for future runner support and currently returns an explicit unsupported-operation error on macOS.
|
||||
`swipe` is a quick, fixed-duration directional throw. Its historical optional `durationMs` remains
|
||||
accepted for compatibility but normalizes to a pan and reports a deprecation; use `gesture pan` for
|
||||
deliberate timed movement.
|
||||
`swipe` is a quick, fixed-duration directional throw. Use `gesture pan` for deliberate timed movement.
|
||||
Repeated coordinate swipes accept at most 200 repetitions and 10000ms pauses, and their combined
|
||||
gesture/pause schedule must fit within 60000ms.
|
||||
`gesture pan` accepts `x y dx dy [durationMs]` for deliberate drags. It uses one pointer by default. Add `--pointer-count 2` for a parallel two-finger pan with constant contact span and angle; this shares the bounded two-contact synthesizer used by transform while retaining pan intent. Android preserves the requested travel duration; iOS uses XCTest drag primitives for one-pointer pan and private XCTest synthesis for two-pointer pan.
|
||||
`gesture fling` accepts `up|down|left|right x y [distance]` for fast directional throws. Its
|
||||
historical duration is accepted as a deprecated alias to pan.
|
||||
`gesture rotate` accepts `degrees [x] [y]`; the degree sign controls direction. Its historical
|
||||
velocity is accepted but deprecated because pacing is derived from the requested rotation.
|
||||
`gesture fling` accepts `up|down|left|right x y [distance]` for fast directional throws.
|
||||
`gesture rotate` accepts `degrees [x] [y]`; the degree sign controls direction. Pacing is derived from the requested rotation.
|
||||
`gesture transform` accepts `x y dx dy scale degrees [durationMs]` for one combined two-finger pan/zoom/rotate gesture on Android and iOS simulators. Pinch, rotate, two-finger pan, and transform use the same viewport-aware pointer planning; impossible paths fail before injection instead of clamping or distorting the requested motion.
|
||||
On iOS simulators it uses private XCTest synthesis for a continuous two-finger pan/scale/rotation path, so verify app-level metrics instead of assuming the requested values map exactly to recognizer output.
|
||||
On Android, `gesture transform` injects a geometric two-finger path. App recognizers may report non-exact pan, scale, and rotation values, so verify qualitative state such as `pan changed yes`, `pinch changed yes`, and `rotate changed yes` unless the app explicitly promises exact centroid metrics. If exact app-state values matter, prefer isolated `gesture pan`, `gesture pinch`, or `gesture rotate` commands.
|
||||
|
||||
Reference in New Issue
Block a user