Michał Pierzchała
287cc18c29
* fix(gestures): fail pre-removal gesture forms at .ad parse time (#1216) #1315 removed the timed forms of `swipe`, `gesture fling`, and `gesture swipe` and `gesture rotate`'s `velocity`, but shipped without the migration guide, the repository sweep, or the parse-time error that issue #1216's own checklist gates a removal on. The sweep finds what that left behind: both `06-swipe-gestures.ad` integration fixtures still carried the 5-argument swipe and would fail at replay, two tests still asserted the removed shapes, and two branches still read the retired positional. Argument arity for every public gesture syntax now lives in one table keyed off the canonical `GESTURE_KINDS`, so a new kind cannot skip it and a form removed from the CLI is removed from `.ad` in the same edit. Both callers read it: the CLI argv parse, and a new `.ad` preflight. A stale script now fails when it is parsed — before the replay executes any device action — naming the line and computing its rewrite, instead of running up to that step and failing as a repairable divergence. The preflight checks arity only: `${VAR}` tokens resolve after planning, and interpolation never splits a token, so the count is decidable while the values are not. Deleting the dead duration read in `readSwipeGeometry` would have left `replay export` emitting no duration, handing Maestro's 400ms default to a gesture the script runs at 100ms, so the export now states `duration: 100`. `.ad` positional gesture parsing is NOT removed. Its only remaining callers are the CLI argv parse and the `.ad` line parse, both the current public syntax rather than a bridge to an older one, so there is nothing to migrate off. ADR 0013 records that and drops the "compatibility" framing that made it read as debt. Both migrated fixtures verified on real devices with the repo's own CLI: iOS simulator 34.9s, Android emulator 45.9s. * fix(gestures): reject removed swipe input at the Node/MCP boundary Review findings on d88c6ed8. P1: `interactionDaemonWriters.swipe` hand-projects five fields, so a JavaScript caller's `durationMs` was dropped before the daemon's `readSwipeInput` could reject it and a default-duration fling ran instead — the exact silent reinterpretation the guide promises does not happen. `gesture` was already safe because its writer runs `readGestureInput` -> `readGesturePayload`, which rejects the removed keys; `swipe` was the one surface with no reader of its own. The rejection now lives in contracts and is shared by the client writer and the daemon handler, so there is one rule and one message. The SDK regression covers all four removed keys and asserts the transport is never reached; reverting the writer call fails it on `swipe durationMs`. P2: the preflight's retired-slot test required a numeric token, so `swipe 197 650 197 300 ${DURATION}` fell back to bare usage text. An unresolved `${VAR}` now counts as the retired slot and is carried into the pan rewrite, while a stray flag or word stays a plain usage error. P2: the removal shipped in 0.20.0, not 0.21 — removal commit6d99914f4is contained in tag v0.20.0. The guide said 0.21 because the CHANGELOG still files it under `Unreleased`; the tag is the truth (headings lag several releases repo-wide, so that is pre-existing and left alone). The `.ad` grep recipe now matches variable-backed durations too. * docs(gestures): make the migration sweep and MCP claim accurate Re-review findings on 328bad8e (both migration-guide accuracy). The saved-script sweep matched only the five-argument `swipe` form; it missed timed `gesture fling`, timed `gesture swipe`, and `gesture rotate ... velocity`, so the repository-cleanliness step was incomplete. Provide one grep per retired form (number-or-`${VAR}` token), each verified to flag the removed shape and skip the valid one. The MCP section claimed the structured rejection carries the CLI's concrete replacement command. It does not: `readGesturePayload` and `assertNoRemovedSwipeInput` return a message that names the removed key and the replacement command (e.g. `gesture fling does not accept durationMs; use gesture pan for timed movement`) but not the fully-substituted coordinate rewrite, because the structured request carries no positional string to rewrite. Describe what the structured path actually returns. * docs(gestures): make the .ad sweep parser-aligned Re-review finding on4fd5d39: the documented sweep required literal single spaces and unquoted numeric tokens, but the `.ad` tokenizer separates on any whitespace (`/\s/`, so tabs too) and accepts double-quoted tokens, so tab-separated or quoted-duration stale lines the parser rejects were missed. Rewrite the patterns to follow the tokenizer: `[[:space:]]+` between tokens and a numeric slot that accepts a bare or double-quoted number (optionally signed) or `${VAR}`. Requiring a digit in the numeric slot keeps a trailing flag like `--count` from being read as the retired positional. Verified against a fixture of tab/space/quoted/quoted-var/negative encodings that the four greps flag exactly the lines `parseReplayScriptDetailed` rejects and none it accepts, and that the live repo sweeps clean. Also state plainly what a regex cannot promise: the parser is the authoritative gate — every retired form is rejected at parse time before any device action, so running the suite finds every stale line by construction and a missed grep can never reach execution. The grep stays a bulk pre-flight, and the "every affected line" claim is scoped to that.
agent-device
Let your coding agent verify its changes in the running app.
agent-device lets coding agents inspect, control, and verify apps on iOS, Android, tvOS, Android TV, Amazon Vega OS TV through the Vega Virtual Device (VVD), web, macOS, and Linux. Agents can read token-efficient accessibility snapshots where supported, find elements by ref or selector, run device actions, and save evidence for review. Initial Vega OS support is VVD-only and covers discovery, app lifecycle, and complete TV-remote control; physical Fire TV, capture, and selector backends remain unsupported.
Your coding agent or QA tool reads each result and chooses the next command. agent-device runs the command and saves evidence when asked.
agent-device uses the inspect-act-verify process from Vercel's agent-browser for mobile, TV, and desktop apps. Basic --platform web support runs agent-browser in the same session and replay system.
Quick start
Install the CLI and check setup. It requires Node.js 22.12 or newer; web automation requires Node.js 24 or newer. See Installation for target requirements.
npm install -g agent-device@latest
agent-device doctor
agent-device --version
agent-device help workflow
agent-device help tv
Run agent-device doctor yourself before handing the CLI to an agent. The installed CLI help defines current behavior. agent-device help workflow links to guides for debugging, replay, React Native profiling, and other tasks.
Add a contact in the built-in iOS Contacts app:
# Start a session.
agent-device open Contacts --platform ios
# Inspect the screen. The example below shows the output; refs vary.
agent-device snapshot -i
# @e2 [button] "Add"
# Use the ref and wait for the UI to settle.
agent-device press @e2 --settle
# The diff includes:
# + @e7 [text-field] "First name"
agent-device fill @e7 "Ada" --settle
# The next diff shows changed values and current refs:
# - @e7 [text-field] "First name"
# + @e14 [text-field] "Ada"
# = @e15 [text-field] "Last name"
# Capture evidence and close the session.
agent-device screenshot ./contact-form.png
agent-device close
Use refs only from the latest output. Do not assume an earlier @eN still identifies the same element. After a command with --settle, use the refs in its diff. Take another snapshot only if the diff omits what you need.
Snapshots use the app's accessibility tree. Clear labels, roles, and test IDs make agent runs more reliable. Use screenshots and videos as evidence or when accessibility data is poor. Use refs and selectors for actions and assertions when you can.
What agents can do
- Inspect app state through accessibility snapshots, refs, selectors, and React Native component trees.
- Act on visible UI by tapping or pressing elements, filling fields, scrolling, making gestures, waiting, asserting state, and handling alerts.
- Diagnose failures with screenshots, video, logs, traces, network data, performance samples, crash details, and React profiles.
- Repeat workflows by saving working steps as
.adscripts for local use or CI. Export strict Maestro YAML when needed.
See Commands for the commands and evidence each target supports.
Next steps
- Set up your agent: run the CLI from Cursor, Codex, Claude Code, Windsurf, or another agent terminal. See AI Agent Setup for skills, rules, MCP tools, and setup for each client.
- Try the sample app: clone the repo and run the bundled Expo test app. Quick Start covers a guided run with screenshots, replay, and performance data.
- Build repeatable tests: use Replay & E2E to repeat tests. Use Debugging & Profiling to find bugs.
Articles and videos
Articles
- Build an AI QA agent for Expo apps with EAS Workflows
- Agent Device: iOS & Android automation for AI agents
- Building mobile QA agents with Vercel Eve
- How we optimized Agent Device for mobile app automation
Videos
- Verifying mobile apps with agent-device
- Using agent-device in an AI coding workflow
- Cloud agents that test mobile apps on real devices
Where to run agent-device
| Path | Best for | Start with |
|---|---|---|
| Local | Trying commands and debugging apps on simulators, emulators, physical devices, macOS, and Linux. | Follow the Quick Start. |
| CI/CD | Automated pull request and merge validation with replay scripts and captured artifacts. | Try the EAS workflow template. GitHub Actions template coming soon. |
| Cloud / remote | Linux runners, managed devices, and remote jobs. | Use Agent Device Cloud, set a remote profile with Commands, or contact Callstack for team QA. |
How it works
agent-device keeps device state in sessions. It sends commands to XCTest on iOS and tvOS, ADB and the snapshot helper on Android, Vega CLI/VDA on the Vega Virtual Device, a local helper on macOS, and AT-SPI on Linux.
Node.js apps can use the typed client or public subpaths. agent-device/android-adb provides the Android ADB provider interface, helpers for logcat, the clipboard, the keyboard, and apps, and port reverse management.
FAQ
What is agent-device?
agent-device is a command-line tool that lets coding agents inspect, control, and verify apps and save evidence for review. It supports iOS, Android, TV, web, macOS, and Linux.
Does it work with React Native, Expo, Flutter, and native apps?
Yes. agent-device supports native iOS and Android apps, plus React Native, Expo, and Flutter apps on supported targets. The commands and evidence vary by target.
How is it different from Appium, Detox, or Maestro?
With agent-device, an agent reads app state and chooses each command at run time. Teams use Appium, Detox, and Maestro to write and maintain test suites. agent-device can complement them by saving its runs as .ad scripts or exporting them as strict Maestro YAML.
Can agent-device run in CI?
Yes. Record a run as an .ad script, replay it locally or in CI, and save screenshots, logs, and other artifacts for review. See Replay & E2E or start with the EAS workflow template.
Who uses agent-device?
Teams and developers at Callstack, JPMorgan Chase, Expensify, Shopify, Kindred, Total Wine & More, LegendList, HerLyfe, App & Flow, and others use agent-device.
Documentation
Contributing
See CONTRIBUTING.md.
Made at Callstack
agent-device is open source under the MIT license. Visit agent-device.dev or contact Callstack.