37 Commits

Author SHA1 Message Date
Yury Semikhatsky 655530f6d0 devops: restore npm publishing from GitHub Actions (#459)
## Summary
- Release publishing goes back to the GitHub Actions workflow with OIDC
trusted publishing and provenance.
- The ESRP pipeline stays available for manual on-demand runs only (no
automatic triggers).
2026-09-03 12:04:51 -07:00
Yury Semikhatsky 397ee39c83 chore: mark v0.1.19 (#458) 2026-09-01 08:38:23 -07:00
Yury Semikhatsky 58d9406780 chore: roll Playwright to 1.63.0-alpha-2026-08-31 (#457) 2026-09-01 08:36:57 -07:00
Yury Semikhatsky cbc09311c4 devops: publish all npm versions via ESRP pipeline (#456)
The ESRP pipeline now publishes @next with current timestamp on manual
runs and @latest from v* release tags. The GitHub Actions npm publish
workflow is removed.
2026-08-27 15:21:41 -07:00
Mert Can Altin 60cb176373 fix(cli): call defaultRegistryDirectory() when resolving the cache file (#454)
`registry.defaultRegistryDirectory` became a function upstream in
microsoft/playwright#41942 and arrived here with the roll to
1.63.0-alpha-2026-08-05. `cacheFile()` still used it as a string, so
`path.join()` threw, `readCache()`/`writeCache()` swallowed the
TypeError, and the update check was never cached: every single CLI
invocation fetched the npm registry and re-ran the installed-skill
check.

Locally that is ~480ms per command instead of ~125ms.

Tests always set PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST, so the default
branch was never exercised. Add a regression test that points HOME at a
temp directory and asserts the cache file is written.

Also hoist `cacheFile()` out of the try blocks so only I/O and parse
failures are swallowed there, instead of masking a path-computation bug
as "no cache".
2026-08-24 16:15:19 -06:00
dependabot[bot] 2f85a94b7b chore(deps): bump the github-actions group with 2 updates (#451) 2026-08-19 12:49:25 -07:00
Dan Fiedler 578643330f Pin GitHub Actions to full-length commit SHAs (#450) 2026-08-19 09:33:15 -07:00
Pavel Feldman 4f9d85a947 fix(cli): apply update-check policy to installed skill check (#447) 2026-08-13 11:39:25 -07:00
Yury Semikhatsky e2d8f311f3 devops: add ESRP pipeline for publishing npm alpha versions (#446) 2026-08-07 11:18:33 -07:00
Yury Semikhatsky ca196c2971 chore: mark v0.1.18 (#444) 2026-08-05 17:12:10 -07:00
Yury Semikhatsky 40783a0063 chore: roll Playwright to 1.63.0-alpha-2026-08-05 (#442) 2026-08-05 15:44:50 -07:00
Dmitry Gozman eee5a185c9 fix(skills): ignore line ending differences when checking installed skill (#439)
Fixes: https://github.com/microsoft/playwright/issues/41760
2026-07-15 17:22:37 +01:00
Yury Semikhatsky 793cfb3257 chore: mark v0.1.17 (#437) 2026-07-09 12:09:39 -07:00
Yury Semikhatsky 372ad83f96 chore: roll Playwright to 1.62.0-alpha-1783623505000 (#436) 2026-07-09 12:05:38 -07:00
Yury Semikhatsky 22d98afd7e chore: mark v0.1.16 (#435) 2026-07-09 11:29:49 -07:00
Yury Semikhatsky 72735e5705 chore: roll Playwright to 1.62.0-alpha-2026-07-08 (#434) 2026-07-09 11:09:14 -07:00
Pavel Feldman 74d9bf144a chore: mark v0.1.15 (#432) 2026-06-29 16:26:16 -07:00
Pavel Feldman 34bf2ada4d chore: roll Playwright to 1.62.0-alpha-2026-06-29 (#431) 2026-06-29 16:08:36 -07:00
Simon Knott 13639df120 test(skills): regress on skill-update notice (#429)
Add an integration test that installs skills, mutates the installed SKILL.md, and asserts the out-of-date warning is shown. Guards against the bundled-skill path resolution regressing if files move.
2026-06-29 13:49:28 +02:00
Pavel Feldman 9805da3991 feat(cli): warn when installed skill is out of date (#427) 2026-06-23 11:13:50 -07:00
Pavel Feldman 0857d71fe5 feat(cli): notify when a newer version is available on npm (#426) 2026-06-23 10:16:29 -07:00
Yury Semikhatsky 9b118a1a73 chore: mark v0.1.14 (#423) 2026-06-09 17:14:06 -07:00
Yury Semikhatsky 7845dfc927 chore: roll Playwright to 1.61.0-alpha-1781023400000 (#422) 2026-06-09 17:03:34 -07:00
Yury Semikhatsky 3a1bafc8b4 chore: mark v0.1.13 (#402) 2026-05-07 16:01:12 -07:00
Yury Semikhatsky fba4d994e1 chore: roll Playwright to 1.61.0-alpha-1778188671000 (#401) 2026-05-07 15:31:41 -07:00
Yury Semikhatsky 695107b8cd chore: mark v0.1.12 (#400) 2026-05-06 16:00:56 -07:00
Yury Semikhatsky a9785a501c chore: roll Playwright to 1.60.0-alpha-1778101408000 (#399) 2026-05-06 14:56:19 -07:00
Andrea Arturo Venti Fuentes fb2a027f20 fix(deps): declare playwright-core as direct dependency (#397) 2026-05-04 14:54:43 -07:00
Yury Semikhatsky 212b11d473 chore: mark v0.1.11 (#395)
* chore(ci): downgrade Node.js from 22 to 20

* chore: mark v0.1.11
2026-05-01 15:34:39 -07:00
Yury Semikhatsky ab6ab40a1c chore(ci): bump checkout/setup-node to Node 24 versions (#394)
* chore(ci): bump checkout/setup-node to versions running on Node.js 24

GitHub is forcing JavaScript actions to Node 24 by default on
2026-06-02 and removing Node 20 from runners on 2026-09-16.

- actions/checkout@v4 -> v5
- actions/setup-node@v4 -> v5

* chore(ci): bump CI Node.js from 20 to 22

Node.js 20 reached end-of-life in April 2026.

* chore(ci): pin Node.js to 24, drop bundled npm upgrade

Node 24 ships npm 11.12.1, which already satisfies the npm 11.5.1+
requirement for OIDC npm publishing.

* chore(ci): keep CI on Node.js 22 (oldest active LTS)

Use the oldest supported LTS in CI to catch any reliance on
newer-Node-only behavior. Publish job stays on 24 for the
bundled npm 11.
2026-05-01 15:17:15 -07:00
Yury Semikhatsky 6b909c5169 chore: roll Playwright to 1.60.0-alpha-1777669338000 (#393) 2026-05-01 14:57:09 -07:00
Yury Semikhatsky 8d95174a8f chore: mark v0.1.10 (#391) 2026-04-30 13:14:05 -07:00
Yury Semikhatsky cb97fa36dc chore: roll Playwright to 1.60.0-alpha-1777575331000 (#392) 2026-04-30 12:55:58 -07:00
Yury Semikhatsky 37c3028c36 chore: roll Playwright to 1.60.0-alpha-1777566615000 (#390) 2026-04-30 11:04:06 -07:00
Pavel Feldman ee24ded177 chore: mark v0.1.9 (#380) 2026-04-24 19:24:43 -07:00
Pavel Feldman 8428dd70d1 chore: roll Playwright to 1.60.0-alpha-1777077614000 (#381) 2026-04-24 17:59:11 -07:00
Pavel Feldman e0019254b5 chore: roll Playwright to 1.60.0-alpha-2026-04-24 (#379) 2026-04-24 17:13:19 -07:00
21 changed files with 938 additions and 84 deletions
+141
View File
@@ -0,0 +1,141 @@
# Publishes @playwright/cli via ESRP. Manual trigger only, regular publishing
# is done from GitHub Actions, see .github/workflows/publish.yml.
# Depending on the selected ref, a manual run publishes:
# - @next (alpha with current timestamp) from main
# - @latest from v* release tags
trigger: none
pr: none
resources:
repositories:
- repository: 1esPipelines
type: git
name: 1ESPipelineTemplates/1ESPipelineTemplates
ref: refs/tags/release
extends:
template: v1/1ES.Official.PipelineTemplate.yml@1esPipelines
parameters:
pool:
name: DevDivPlaywrightAzurePipelinesUbuntu2204
os: linux
sdl:
sourceAnalysisPool:
# SDL tools require windows, see https://aka.ms/AAo6v8e
name: DevDivPlaywrightAzurePipelinesWindows2022
os: windows
stages:
- stage: Stage
jobs:
- job: Build
displayName: "Build npm package"
templateContext:
outputs:
- output: pipelineArtifact
path: $(Build.ArtifactStagingDirectory)/esrp-build
artifact: esrp-build
steps:
- checkout: self
displayName: "Checkout code"
- task: Bash@3
displayName: "Check the branch is main or a v* tag"
inputs:
targetType: "inline"
script: |
if [[ "$BUILD_SOURCE_BRANCH" != "refs/heads/main" && "$BUILD_SOURCE_BRANCH" != refs/tags/v* ]]; then
echo "Can only publish from main or v* tags."
echo "Unexpected branch: $BUILD_SOURCE_BRANCH"
exit 1
fi
env:
BUILD_SOURCE_BRANCH: $(Build.SourceBranch)
- task: UseNode@1
inputs:
version: '24.x'
displayName: "Install Node.js"
- task: Bash@3
displayName: "setup .npmrc"
inputs:
targetType: "inline"
script: |
echo "registry=https://devdiv.pkgs.visualstudio.com/DevDiv/_packaging/DevDiv_PublicPackages/npm/registry/" >> .npmrc
- task: npmAuthenticate@0
displayName: "authenticate the private npm registry"
inputs:
workingFile: .npmrc
- script: npm ci
displayName: "npm ci"
- task: Bash@3
name: setVersion
displayName: "Set version and dist-tag"
inputs:
targetType: "inline"
script: |
set -e
BASE_VERSION=$(node -p "require('./package.json').version.split('-')[0]")
if [[ "$BUILD_SOURCE_BRANCH" == refs/tags/v* ]]; then
# Release version is already checked in, only publish what the tag points at.
NPM_DIST_TAG="latest"
if [[ "$BUILD_SOURCE_BRANCH" != "refs/tags/v$BASE_VERSION" ]]; then
echo "ERROR: version '$BASE_VERSION' does not match tag '$BUILD_SOURCE_BRANCH'"
exit 1
fi
else
NPM_DIST_TAG="next"
npm version "${BASE_VERSION}-alpha-$(date +%s)000" --no-git-tag-version
fi
echo "Publishing version $(node -p "require('./package.json').version") with dist-tag $NPM_DIST_TAG"
echo "##vso[task.setvariable variable=npmDistTag;isOutput=true]$NPM_DIST_TAG"
env:
BUILD_SOURCE_BRANCH: $(Build.SourceBranch)
- task: Bash@3
displayName: "Pack the package"
inputs:
targetType: "inline"
script: |
set -e
mkdir -p "$(Build.ArtifactStagingDirectory)/esrp-build"
npm pack --pack-destination="$(Build.ArtifactStagingDirectory)/esrp-build"
ls -la "$(Build.ArtifactStagingDirectory)/esrp-build"
- job: Publish
displayName: "ESRP Release to npm"
dependsOn: Build
variables:
npmDistTag: $[ dependencies.Build.outputs['setVersion.npmDistTag'] ]
templateContext:
type: releaseJob
isProduction: true
inputs:
- input: pipelineArtifact
artifactName: esrp-build
targetPath: $(Build.ArtifactStagingDirectory)/esrp-build
steps:
- checkout: none
- task: EsrpRelease@11
inputs:
connectedservicename: 'Playwright-ESRP-PME'
usemanagedidentity: true
keyvaultname: 'playwright-esrp-pme'
signcertname: 'ESRP-Release-Sign'
clientid: '13434a40-7de4-4c23-81a3-d843dc81c2c5'
intent: 'PackageDistribution'
contenttype: 'npm'
# npm dist-tag to publish with.
productstate: '$(npmDistTag)'
folderlocation: '$(Build.ArtifactStagingDirectory)/esrp-build'
waitforreleasecompletion: true
owners: 'yurys@microsoft.com'
approvers: 'yurys@microsoft.com'
serviceendpointurl: 'https://api.esrp.microsoft.com'
mainpublisher: 'Playwright'
domaintenantid: '975f013f-7f24-47e8-a7d3-abc4752bf346'
displayName: 'ESRP Release to npm'
+2 -1
View File
@@ -6,8 +6,9 @@
`npm info playwright@next version`
2. **Update Playwright packages** in `package.json`:
- Update `playwright` (dependency) and `@playwright/test` (devDependency) to the target version.
- Update `playwright` and `playwright-core` (dependencies) and `@playwright/test` (devDependency) to the target version.
- Run `npm install` to update `package-lock.json`.
- Verify with `npm ls playwright-core` that the root `playwright-core` actually rolled — it is pinned directly in `package.json` and is what `playwright-cli.js` runs; forgetting it leaves the CLI on the old version while tests pass against the new `@playwright/test`.
3. **Run the update script** to sync skills and README:
```bash
+11
View File
@@ -0,0 +1,11 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
groups:
github-actions:
patterns: ["*"]
schedule:
interval: "weekly"
cooldown:
default-days: 7
+2 -2
View File
@@ -14,9 +14,9 @@ jobs:
os: [ubuntu-latest, macos-15, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Use Node.js 20
uses: actions/setup-node@v4
uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: '20'
cache: 'npm'
+3 -5
View File
@@ -12,12 +12,10 @@ jobs:
contents: read
id-token: write # Required for OIDC npm publishing
steps:
- uses: actions/checkout@v5
- uses: actions/setup-node@v5
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
with:
node-version: 20
node-version: 24
registry-url: https://registry.npmjs.org/
# Ensure npm 11.5.1 or later is installed (for OIDC npm publishing)
- run: npm install -g npm@latest
- run: npm ci
- run: npm publish
+3
View File
@@ -4,3 +4,6 @@ node_modules/
/.playwright-cli/
# Ignore self-skill which is a build artifact
.claude/skills/playwright-cli/
.npmrc
# Playwright CLI output (may contain credentials)
.playwright-cli/
+1
View File
@@ -2,4 +2,5 @@
!README.md
!LICENSE
!playwright-cli.js
!skillCheck.js
!skills/**
+36 -6
View File
@@ -139,6 +139,8 @@ playwright-cli dblclick <ref> [button] # perform double click on a web page
playwright-cli fill <ref> <text> # fill text into editable element
playwright-cli fill <ref> <text> --submit # fill and press Enter
playwright-cli drag <startRef> <endRef> # perform drag and drop between two elements
playwright-cli drop <ref> --path=<file> # drop files onto an element (from outside the page)
playwright-cli drop <ref> --data="k=v" # drop data onto an element
playwright-cli hover <ref> # hover over element on page
playwright-cli select <ref> <val> # select an option in a dropdown
playwright-cli upload <file> # upload one or multiple files
@@ -148,6 +150,8 @@ playwright-cli snapshot # capture page snapshot to obtain elemen
playwright-cli snapshot --filename=f # save snapshot to specific file
playwright-cli snapshot <ref> # snapshot a specific element
playwright-cli snapshot --depth=N # limit snapshot depth for efficiency
playwright-cli find <text> # search the snapshot for text, returns matching nodes
playwright-cli find --regex <pattern> # search the snapshot with a regexp
playwright-cli eval <func> [ref] # evaluate javascript expression on page or element
playwright-cli dialog-accept [prompt] # accept a dialog
playwright-cli dialog-dismiss # dismiss a dialog
@@ -184,6 +188,7 @@ playwright-cli mousewheel <dx> <dy> # scroll mouse wheel
```bash
playwright-cli screenshot [ref] # screenshot of the current page or element
playwright-cli screenshot --filename=f # save screenshot with specific filename
playwright-cli screenshot --hires # capture at full device pixel ratio
playwright-cli pdf # save page as pdf
playwright-cli pdf --filename=page.pdf # save pdf with specific filename
```
@@ -237,23 +242,38 @@ playwright-cli unroute [pattern] # remove route(s)
```bash
playwright-cli console [min-level] # list console messages
playwright-cli network # list all network requests since loading the page
playwright-cli requests # list all network requests since loading the page
playwright-cli request <index> # show details for a specific request
playwright-cli run-code <code> # run playwright code snippet
playwright-cli run-code --filename=f # run playwright code from a file
playwright-cli tracing-start # start trace recording
playwright-cli tracing-stop # stop trace recording
playwright-cli recording-start # record user actions in the browser
playwright-cli recording-stop # stop recording, print actions as Playwright code
playwright-cli video-start [filename] # start video recording
playwright-cli video-chapter <title> # add a chapter marker to the video
playwright-cli video-show-actions # annotate each action with a callout in the video
playwright-cli video-hide-actions # stop annotating actions in the video
playwright-cli video-stop # stop video recording
playwright-cli show # open the visual dashboard
playwright-cli show --annotate # launch dashboard for UI review / design feedback
playwright-cli generate-locator <ref> # generate a playwright locator for an element
playwright-cli highlight <ref> # show a persistent highlight overlay
playwright-cli highlight <ref> --style= # highlight with a custom CSS style
playwright-cli highlight <ref> --hide # hide highlight on a specific element
playwright-cli highlight --hide # hide all page highlights
```
### Open parameters
```bash
playwright-cli open --browser=chrome # use specific browser
playwright-cli attach --extension # connect via browser extension
playwright-cli open --mobile # emulate a generic mobile device
playwright-cli open --device="iPhone 15" # emulate a specific device
playwright-cli attach --extension=chrome # connect via Playwright Extension
playwright-cli attach --cdp=chrome # attach to running Chrome/Edge by channel
playwright-cli attach --cdp=<url> # attach via CDP endpoint
playwright-cli detach # detach an attached session, leaves the external browser running
playwright-cli open --persistent # use persistent profile
playwright-cli open --profile=<path> # use custom profile directory
playwright-cli open --config=file.json # use config file
@@ -289,6 +309,16 @@ playwright-cli snapshot "#main"
# limit snapshot depth for efficiency, take a partial snapshot afterwards
playwright-cli snapshot --depth=4
playwright-cli snapshot e34
# include each element's bounding box as [box=x,y,width,height]
playwright-cli snapshot --boxes
# search a large snapshot instead of capturing it all — returns matching nodes
# with 3 lines of context around each match (like grep -C)
playwright-cli find "Add to cart"
playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}"
# wrap the regexp in slashes to add flags, e.g. /i for case-insensitive
playwright-cli find --regex "/sign (in|up)/i"
```
### Targeting elements
@@ -329,13 +359,13 @@ playwright-cli kill-all # forcefully kill all browser processes
### Local installation
If global `playwright-cli` command is not available, try a local version via `npx playwright-cli`:
If global `playwright-cli` command is not available, try a local version via `npx playwright cli`:
```bash
npx --no-install playwright-cli --version
npx --no-install playwright --version
```
When local version is available, use `npx playwright-cli` in all commands. Otherwise, install `playwright-cli` as a global command:
When local version is available, use `npx playwright cli` in all commands. Otherwise, install `playwright-cli` as a global command:
```bash
npm install -g @playwright/cli@latest
@@ -543,7 +573,7 @@ The installed skill includes detailed reference guides for common tasks:
* **Running Playwright code** — execute arbitrary Playwright scripts
* **Browser session management** — manage multiple browser sessions
* **Storage state (cookies, localStorage)** — persist and restore browser state
* **Test generation** — generate Playwright tests from interactions
* **Test generation (plan / generate / heal)** — generate Playwright tests from a spec or interactions
* **Tracing** — record and inspect execution traces
* **Video recording** — capture browser session videos
* **Inspecting element attributes** — get element id, class, or any attribute not visible in the snapshot
+19 -35
View File
@@ -1,21 +1,22 @@
{
"name": "@playwright/cli",
"version": "0.1.8",
"version": "0.1.19",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@playwright/cli",
"version": "0.1.8",
"version": "0.1.19",
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.60.0-alpha-2026-04-14"
"playwright": "1.63.0-alpha-2026-08-31",
"playwright-core": "1.63.0-alpha-2026-08-31"
},
"bin": {
"playwright-cli": "playwright-cli.js"
},
"devDependencies": {
"@playwright/test": "1.60.0-alpha-2026-04-14",
"@playwright/test": "1.63.0-alpha-2026-08-31",
"@types/node": "^25.2.1"
},
"engines": {
@@ -23,19 +24,19 @@
}
},
"node_modules/@playwright/test": {
"version": "1.60.0-alpha-2026-04-14",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.60.0-alpha-2026-04-14.tgz",
"integrity": "sha512-pUgrMj7b+Fq/lfuja6Gce3XObUfgwKzhe9tGBVmMOKC2sfxVulzKzMEBwYccV7ccK2F+xdXraSJ6+Nq17G8+ug==",
"version": "1.63.0-alpha-2026-08-31",
"resolved": "https://registry.npmjs.org/@playwright/test/-/test-1.63.0-alpha-2026-08-31.tgz",
"integrity": "sha512-sNAYSkzbTC67x/v4ZekJ5ZzBFhSaHGc5t8+/3lqZI9LSvtYWVC+L2bHDbfFIhVXBjtl+1v+rg9Fdx+Rx3w2E8g==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"playwright": "1.60.0-alpha-2026-04-14"
"playwright": "1.63.0-alpha-2026-08-31"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
"node": ">=20"
}
},
"node_modules/@types/node": {
@@ -48,48 +49,31 @@
"undici-types": "~7.16.0"
}
},
"node_modules/fsevents": {
"version": "2.3.2",
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
"hasInstallScript": true,
"license": "MIT",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
}
},
"node_modules/playwright": {
"version": "1.60.0-alpha-2026-04-14",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.60.0-alpha-2026-04-14.tgz",
"integrity": "sha512-wrYNgIOWpw/vaC4TpFHx9mVYF7kM6bYyRbStIHfxfqHDZvj1rNcYKSiIBNSUA9qm4bye4PvPEDSpe3RqjYFX6A==",
"version": "1.63.0-alpha-2026-08-31",
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.63.0-alpha-2026-08-31.tgz",
"integrity": "sha512-3XAsuznfu8jBVJ4QxdGvBkt0+b8ZFwuwJYyOfiIw5ZjUOrNLNRhKxzLzLuydou3gJ9c6eMwVqgzdiOwhy54Kzw==",
"license": "Apache-2.0",
"dependencies": {
"playwright-core": "1.60.0-alpha-2026-04-14"
"playwright-core": "1.63.0-alpha-2026-08-31"
},
"bin": {
"playwright": "cli.js"
},
"engines": {
"node": ">=18"
},
"optionalDependencies": {
"fsevents": "2.3.2"
"node": ">=20"
}
},
"node_modules/playwright-core": {
"version": "1.60.0-alpha-2026-04-14",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.60.0-alpha-2026-04-14.tgz",
"integrity": "sha512-Wkuhxvhg7YnJXJo+UomP4XlOhdH+AL2QdzJyLuHzqKEIxFkKqtSKF5TTfLt5gyB0UhqEO6knMpr5hULo3W6OEw==",
"version": "1.63.0-alpha-2026-08-31",
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.63.0-alpha-2026-08-31.tgz",
"integrity": "sha512-1ek0Lyr12h6jcs/WTcNoVtzZkQp7D/90PsMuBW/Rm6h3AsWAbzpqj0geMv8+8Tzzr9CSUYvg9kznrVZINQMXXw==",
"license": "Apache-2.0",
"bin": {
"playwright-core": "cli.js"
},
"engines": {
"node": ">=18"
"node": ">=20"
}
},
"node_modules/undici-types": {
+4 -3
View File
@@ -1,6 +1,6 @@
{
"name": "@playwright/cli",
"version": "0.1.8",
"version": "0.1.19",
"description": "Playwright CLI",
"repository": {
"type": "git",
@@ -18,11 +18,12 @@
"test": "playwright test"
},
"devDependencies": {
"@playwright/test": "1.60.0-alpha-2026-04-14",
"@playwright/test": "1.63.0-alpha-2026-08-31",
"@types/node": "^25.2.1"
},
"dependencies": {
"playwright": "1.60.0-alpha-2026-04-14"
"playwright": "1.63.0-alpha-2026-08-31",
"playwright-core": "1.63.0-alpha-2026-08-31"
},
"bin": {
"playwright-cli": "playwright-cli.js"
+95 -1
View File
@@ -15,7 +15,101 @@
* limitations under the License.
*/
// @ts-check
const fs = require('fs');
const path = require('path');
const { program } = require('playwright-core/lib/tools/cli-client/program');
const coreBundle = require('playwright-core/lib/coreBundle');
const { tools, registry } = coreBundle;
const { checkInstalledSkills, frame } = require('./skillCheck');
const packageJson = require('./package.json');
program({ embedderVersion: packageJson.version });
const ONE_DAY_MS = 24 * 60 * 60 * 1000;
main();
async function main() {
await checkForUpdates().catch(() => {});
program({ embedderVersion: packageJson.version });
}
async function checkForUpdates() {
if (process.env.NO_UPDATE_NOTIFIER || process.env.CI)
return;
const cache = readCache();
const stale = !cache || (Date.now() - cache.lastCheck) > ONE_DAY_MS;
if (!stale)
return;
writeCache({ lastCheck: Date.now() });
const command = process.argv.slice(2).find(arg => !arg.startsWith('-'));
if (command !== 'install')
checkInstalledSkills();
const latest = await fetchLatestVersion();
if (latest && tools.compareSemver(latest, packageJson.version) > 0)
printNotice(packageJson.version, latest);
}
async function fetchLatestVersion() {
try {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 1500);
try {
const res = await fetch(`https://registry.npmjs.org/${packageJson.name}/latest`, { signal: controller.signal });
if (!res.ok)
return undefined;
const json = await res.json();
return typeof json.version === 'string' ? json.version : undefined;
} finally {
clearTimeout(timeout);
}
} catch {
return undefined;
}
}
/**
*
* @param {string} current
* @param {string} latest
*/
function printNotice(current, latest) {
process.stderr.write('\n' + frame([
`Update available for ${packageJson.name}: ${current}${latest}`,
`Run \`npm install -g ${packageJson.name}@latest\` (global) or`,
`\`npm install --save-dev ${packageJson.name}@latest\` (local) to update.`,
]) + '\n');
}
function cacheFile() {
const dir = process.env.PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST || registry.defaultRegistryDirectory();
return path.join(dir, 'cli-update-check.json');
}
function readCache() {
const file = cacheFile();
try {
const data = JSON.parse(fs.readFileSync(file, 'utf8'));
if (typeof data.lastCheck === 'number')
return data;
} catch {
}
return undefined;
}
/**
* @param {*} data
*/
function writeCache(data) {
const file = cacheFile();
try {
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, JSON.stringify(data));
} catch {
}
}
+79
View File
@@ -0,0 +1,79 @@
/**
* Copyright (c) Microsoft Corporation.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
// @ts-check
const fs = require('fs');
const path = require('path');
function bundledSkillFile() {
const corePath = require.resolve('playwright-core/package.json');
return path.join(path.dirname(corePath), 'lib', 'tools', 'skills', 'playwright-cli', 'SKILL.md');
}
function installedSkillTargets() {
const cwd = process.cwd();
return [
{ dir: path.join(cwd, '.claude', 'skills', 'playwright-cli'), command: 'playwright-cli install --skills' },
{ dir: path.join(cwd, '.agents', 'skills', 'playwright-cli'), command: 'playwright-cli install --skills=agents' },
];
}
/**
* @param {string} file
* @returns
*/
function readSkill(file) {
// Normalize line endings, they could be affected by git or editor settings.
return fs.existsSync(file) ? fs.readFileSync(file, 'utf8').replace(/\r\n/g, '\n') : null;
}
/**
* @param {string[]} lines
* @returns {string}
*/
function frame(lines) {
const width = Math.max(...lines.map(line => line.length));
const top = '╔' + '═'.repeat(width + 2) + '╗';
const bottom = '╚' + '═'.repeat(width + 2) + '╝';
const body = lines.map(line => `${line.padEnd(width)}`);
return [top, ...body, bottom].join('\n') + '\n';
}
function checkInstalledSkills() {
try {
const bundled = readSkill(bundledSkillFile());
if (!bundled)
return;
for (const target of installedSkillTargets()) {
const installed = readSkill(path.join(target.dir, 'SKILL.md'));
if (installed === null)
continue;
if (installed !== bundled) {
process.stderr.write(frame([
`The playwright-cli skill at '${path.relative(process.cwd(), target.dir)}'`,
`does not match the tool version.`,
``,
`Run \`${target.command}\``,
`to install the up-to-date skill.`,
]));
}
}
} catch {
}
}
module.exports = { checkInstalledSkills, frame };
+82 -8
View File
@@ -38,12 +38,20 @@ playwright-cli dblclick e7
# --submit presses Enter after filling the element
playwright-cli fill e5 "user@example.com" --submit
playwright-cli drag e2 e8
# drop files or data onto an element (from outside the page)
playwright-cli drop e4 --path=./image.png
playwright-cli drop e4 --data="text/plain=hello world"
playwright-cli hover e4
playwright-cli select e9 "option-value"
playwright-cli upload ./document.pdf
playwright-cli check e12
playwright-cli uncheck e12
playwright-cli snapshot
# search the snapshot for text or a regexp, returns matching nodes with surrounding context
playwright-cli find "Sign in"
playwright-cli find --regex "Sign (in|up)"
# wrap the regexp in slashes to add flags, e.g. /i for case-insensitive
playwright-cli find --regex "/sign (in|up)/i"
playwright-cli eval "document.title"
playwright-cli eval "el => el.textContent" e5
# get element id, class, or any attribute not visible in the snapshot
@@ -90,6 +98,7 @@ playwright-cli mousewheel 0 100
playwright-cli screenshot
playwright-cli screenshot e5
playwright-cli screenshot --filename=page.png
playwright-cli screenshot --hires
playwright-cli pdf --filename=page.pdf
```
@@ -150,14 +159,37 @@ playwright-cli unroute
```bash
playwright-cli console
playwright-cli console warning
playwright-cli network
playwright-cli requests
playwright-cli request 5
playwright-cli run-code "async page => await page.context().grantPermissions(['geolocation'])"
playwright-cli run-code --filename=script.js
playwright-cli tracing-start
playwright-cli tracing-stop
# record user actions in the browser, print them as Playwright code on stop
playwright-cli recording-start
playwright-cli recording-stop
playwright-cli video-start video.webm
playwright-cli video-chapter "Chapter Title" --description="Details" --duration=2000
playwright-cli video-stop
# annotate each subsequent action (click, type, ...) with a callout naming the action and highlighting the target
playwright-cli video-show-actions --duration=600 --position=top-right
playwright-cli video-hide-actions
# launch the dashboard for UI review / design feedback — user annotates the page, you receive the annotated screenshot, snapshot, and notes
playwright-cli show --annotate
# generate a Playwright locator for an element from its ref or selector
playwright-cli generate-locator e5 --raw
# show a persistent highlight overlay for an element, optionally with a custom style
playwright-cli highlight e5
playwright-cli highlight e5 --style="outline: 3px dashed red"
# hide a single element highlight, or all page highlights when no target is given
playwright-cli highlight e5 --hide
playwright-cli highlight --hide
```
## Raw output
@@ -175,6 +207,11 @@ TOKEN=$(playwright-cli --raw cookie-get session_id)
playwright-cli --raw localstorage-get theme
```
For structured output wrapping every reply as JSON, pass --json
```bash
playwright-cli list --json
```
## Open parameters
```bash
# Use specific browser when creating session
@@ -183,13 +220,19 @@ playwright-cli open --browser=firefox
playwright-cli open --browser=webkit
playwright-cli open --browser=msedge
# Emulate a generic mobile device (Pixel 10 for Chromium, iPhone 17 for WebKit).
# Prefer this when a mobile layout is acceptable: mobile pages are usually
# lighter, so snapshots are smaller and cheaper.
playwright-cli open --mobile
playwright-cli open --device="iPhone 15"
# Use persistent profile (by default profile is in-memory)
playwright-cli open --persistent
# Use persistent profile with custom directory
playwright-cli open --profile=/path/to/profile
# Connect to browser via extension
playwright-cli attach --extension
# Connect to browser via Playwright Extension
playwright-cli attach --extension=chrome
# Connect to a running Chrome or Edge by channel name
playwright-cli attach --cdp=chrome
@@ -203,10 +246,24 @@ playwright-cli open --config=my-config.json
# Close the browser
playwright-cli close
# Detach from an attached browser (leaves the external browser running)
playwright-cli -s=msedge detach
# Delete user data for the default session
playwright-cli delete-data
```
## URLs with `&` on Windows
On Windows, `cmd.exe` and PowerShell treat `&` as a command separator, so URLs with multiple query parameters get truncated before `playwright-cli` runs. Escape `&` with `^&` in `cmd.exe`, or use `--%` in PowerShell:
```batch
playwright-cli goto "https://example.com/?a=1^&b=2"
```
```powershell
playwright-cli --% goto "https://example.com/?a=1&b=2"
```
## Snapshots
After each command, playwright-cli provides a snapshot of the current browser state.
@@ -235,6 +292,14 @@ playwright-cli snapshot "#main"
# limit snapshot depth for efficiency, take a partial snapshot afterwards
playwright-cli snapshot --depth=4
playwright-cli snapshot e34
# include each element's bounding box as [box=x,y,width,height]
playwright-cli snapshot --boxes
# search a large snapshot instead of capturing it all — returns matching nodes
# with 3 lines of context around each match (like grep -C)
playwright-cli find "Add to cart"
playwright-cli find --regex "\\$[0-9]+\\.[0-9]{2}"
```
## Targeting elements
@@ -282,13 +347,13 @@ playwright-cli kill-all
## Installation
If global `playwright-cli` command is not available, try a local version via `npx playwright-cli`:
If global `playwright-cli` command is not available, try a local version via `npx playwright cli`:
```bash
npx --no-install playwright-cli --version
npx --no-install playwright --version
```
When local version is available, use `npx playwright-cli` in all commands. Otherwise, install `playwright-cli` as a global command:
When local version is available, use `npx playwright cli` in all commands. Otherwise, install `playwright-cli` as a global command:
```bash
npm install -g @playwright/cli@latest
@@ -325,7 +390,7 @@ playwright-cli open https://example.com
playwright-cli click e4
playwright-cli fill e7 "test"
playwright-cli console
playwright-cli network
playwright-cli requests
playwright-cli close
```
@@ -338,6 +403,15 @@ playwright-cli tracing-stop
playwright-cli close
```
## Example: Interactive session
Ask the user for UI review or design feedback. The user draws boxes on the live page and types comments; you receive the annotated screenshot, the snapshot of the marked region, and the user's notes. Use this whenever the user asks for "UI review", "design feedback", or to "ask the user what they think / want / mean":
```bash
playwright-cli open https://example.com
playwright-cli show --annotate
```
## Specific tasks
* **Running and Debugging Playwright tests** [references/playwright-tests.md](references/playwright-tests.md)
@@ -345,7 +419,7 @@ playwright-cli close
* **Running Playwright code** [references/running-code.md](references/running-code.md)
* **Browser session management** [references/session-management.md](references/session-management.md)
* **Storage state (cookies, localStorage)** [references/storage-state.md](references/storage-state.md)
* **Test generation** [references/test-generation.md](references/test-generation.md)
* **Test generation (plan / generate / heal)** [references/test-generation.md](references/test-generation.md)
* **Tracing** [references/tracing.md](references/tracing.md)
* **Video recording** [references/video-recording.md](references/video-recording.md)
* **Inspecting element attributes** [references/element-attributes.md](references/element-attributes.md)
@@ -14,7 +14,7 @@ PLAYWRIGHT_HTML_OPEN=never npm run special-test-command
To debug a failing Playwright test, run it with `--debug=cli` option. This command will pause the test at the start and print the debugging instructions.
**IMPORTANT**: run the command in the background and check the output until "Debugging Instructions" is printed.
**IMPORTANT**: run the command in the background and check the output until "Debugging Instructions" is printed. Make sure to stop the command after you have finished.
Once instructions containing a session name are printed, use `playwright-cli` to attach the session and explore the page.
@@ -11,6 +11,16 @@ playwright-cli run-code "async page => {
}"
```
You can also load the function from a file:
```bash
playwright-cli run-code --filename=./my-script.js
```
The code must be a single function expression, it is wrapped in `(...)` and evaluated.
import/export/require syntax is not supported.
## Geolocation
```bash
@@ -129,6 +129,8 @@ playwright-cli attach --cdp=msedge-dev
Supported channels: `chrome`, `chrome-beta`, `chrome-dev`, `chrome-canary`, `msedge`, `msedge-beta`, `msedge-dev`, `msedge-canary`.
When `--session` is not provided, the session is named after the channel (e.g. `--cdp=msedge` creates a session called `msedge`), so parallel attaches to Chrome and Edge don't collide on `default`. Pass `--session=<name>` to override.
### Attach via CDP endpoint
Connect to a browser that exposes a Chrome DevTools Protocol endpoint:
@@ -145,6 +147,20 @@ Connect to a browser with the Playwright extension installed:
playwright-cli attach --extension
```
### Detach
Tear down an attached session without affecting the external browser:
```bash
# Detach the default attached session
playwright-cli detach
# Detach a specific attached session
playwright-cli -s=msedge detach
```
`detach` only works on sessions created via `attach`. For sessions created via `open`, use `close`.
## Default Browser Session
When `-s` is omitted, commands use the default browser session:
@@ -38,7 +38,7 @@ The saved file contains:
"value": "abc123",
"domain": "example.com",
"path": "/",
"expires": 1735689600,
"expires": 1893456000,
"httpOnly": true,
"secure": true,
"sameSite": "Lax"
@@ -92,7 +92,7 @@ playwright-cli cookie-set session abc123
playwright-cli cookie-set session abc123 --domain=example.com --path=/ --httpOnly --secure --sameSite=Lax
# Cookie with expiration (Unix timestamp)
playwright-cli cookie-set remember_me token123 --expires=1735689600
playwright-cli cookie-set remember_me token123 --expires=1893456000
```
### Delete a Cookie
@@ -1,13 +1,19 @@
# Test Generation
# Test generation (plan → generate → heal)
Generate Playwright test code automatically as you interact with the browser.
End-to-end workflow for authoring and maintaining Playwright tests with `playwright-cli`. Every `playwright-cli` action emits the equivalent Playwright TypeScript, and that generated code is the raw material for every test. The sections below can be used independently:
## How It Works
- **How generation works** — the core mechanic everything else relies on: actions become TypeScript, plus how to add assertions.
- **Plan** — explore the app, produce a spec file describing what to test.
- **Generate** — turn a spec into Playwright test files. Update the spec if it's vague or stale.
- **Heal** — diagnose failing tests, fix the code, reconcile the spec with reality.
Every action you perform with `playwright-cli` generates corresponding Playwright TypeScript code.
This code appears in the output and can be copied directly into your test files.
Plan / generate / heal lean on the same mechanic: run `npx playwright test --debug=cli` in the background, then `playwright-cli attach tw-XXXX` to drive the paused page interactively. See [playwright-tests.md](playwright-tests.md) for the debug/attach mechanics.
## Example Workflow
---
## 0. How generation works
Every action you perform with `playwright-cli` generates corresponding Playwright TypeScript code. This code appears in the output and can be copied directly into your test files.
```bash
# Start a session
@@ -31,7 +37,7 @@ playwright-cli click e3
# await page.getByRole('button', { name: 'Sign In' }).click();
```
## Building a Test File
### Building a test file
Collect the generated code into a Playwright test:
@@ -50,9 +56,7 @@ test('login flow', async ({ page }) => {
});
```
## Best Practices
### 1. Use Semantic Locators
### Use semantic locators
The generated code uses role-based locators when possible, which are more resilient:
@@ -64,7 +68,7 @@ await page.getByRole('button', { name: 'Submit' }).click();
await page.locator('#submit-btn').click();
```
### 2. Explore Before Recording
### Explore before recording
Take snapshots to understand the page structure before recording actions:
@@ -75,14 +79,355 @@ playwright-cli snapshot
playwright-cli click e5
```
### 3. Add Assertions Manually
### Add assertions manually
Generated code captures actions but not assertions. Add expectations in your test:
Generated code captures actions but not assertions. Add expectations in your test using one of the recommended matchers:
- `toBeVisible()` — element is rendered and visible
- `toHaveText(text)` — element text content matches
- `toHaveValue(value) / toBeEmpty()` — input/select value matches
- `toBeChecked() / toBeUnchecked()` — checkbox state matches
- `toMatchAriaSnapshot(snapshot)` — page (or locator) matches a partial accessibility snapshot
Use `playwright-cli generate-locator <target>` to produce the locator expression for the assertion, and the snapshot/eval commands to capture the expected value.
When asserting text content, make sure that generated locator does not contain text from the element itself. `getByTestId()` or `getByLabel()` usually work well with asserting text. When locator is text-based, prefer `toBeVisible()` instead.
Snapshot to be matched does not have to contain all the information - only capture what's necessary for the assertion. You can use regular expressions for unstable values.
```bash
# Get a stable locator for an element ref to use in the assertion
playwright-cli --raw generate-locator e5
# getByRole('button', { name: 'Submit' })
# Capture expected text content for toHaveText
playwright-cli --raw eval "el => el.textContent" e5
# Capture expected input value for toHaveValue/toBeEmpty
playwright-cli --raw eval "el => el.value" e5
# Capture expected aria snapshot for toMatchAriaSnapshot/toBeChecked
# (whole page, or use a ref to scope to a region)
playwright-cli --raw snapshot
playwright-cli --raw snapshot e5
```
```typescript
// Generated action
await page.getByRole('button', { name: 'Submit' }).click();
// Manual assertion
await expect(page.getByText('Success')).toBeVisible();
// Manual assertions using the outputs above:
await expect(page.getByRole('alert', { name: 'Success' })).toBeVisible();
await expect(page.getByTestId('main-header')).toHaveText('Welcome, user');
await expect(page.getByRole('textbox', { name: 'Email' })).toHaveValue('user@example.com');
await expect(page.getByRole('checkbox', { name: 'Enable notifications' })).toBeChecked();
// toMatchAriaSnapshot on the whole page, finds a matching region
await expect(page).toMatchAriaSnapshot(`
- heading "Welcome, user"
- link /\\d+ new messages?/
- button "Sign out"
`);
// toMatchAriaSnapshot scoped to a region
await expect(page.getByRole('navigation')).toMatchAriaSnapshot(`
- link "Home"
- link /\\d+ new messages?/
- link "Profile"
`);
```
---
## 1. Planning
Goal: produce a spec file (e.g. `specs/<feature>.plan.md`) that enumerates the scenarios to test. **Always** write the spec to a file.
### 1.1 Prerequisite: workspace
Check the workspace has Playwright installed before anything else:
```bash
# Either of these confirms a workspace:
test -f playwright.config.ts || test -f playwright.config.js
npx --no-install playwright --version
```
If there is no Playwright install, bootstrap one and let the user pick the defaults:
```bash
npm init playwright@latest
```
### 1.2 Prerequisite: seed test
A **seed test** is a minimal test that lands the page in the state every scenario starts from: navigation to the app, any required login, feature flags, etc. Scenarios assume a fresh start *after* the seed. `--debug=cli` pauses *inside* this test, so the seed is where every planning and generation session begins.
Minimum viable seed:
```ts
// tests/seed.spec.ts
import { test } from '@playwright/test';
test('seed', async ({ page }) => {
await page.goto('https://example.com/');
});
```
Preferred — push navigation into a fixture so scenario tests reuse it:
```ts
// tests/fixtures.ts
import { test as baseTest } from '@playwright/test';
export { expect } from '@playwright/test';
export const test = baseTest.extend({
page: async ({ page }, use) => {
await page.goto('https://example.com/');
await use(page);
},
});
```
```ts
// tests/seed.spec.ts
import { test } from './fixtures';
test('seed', async ({ page }) => {
// Fixture already navigates. This empty body tells agents where to start.
});
```
If no seed exists, create one that at least navigates to the app.
### 1.3 Explore the app
Launch the app via the seed in the background and attach:
```bash
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/seed.spec.ts --debug=cli
# wait for "Debugging Instructions" and the session name tw-XXXX
playwright-cli attach tw-XXXX
```
Resume so the seed runs, then probe the app:
```bash
playwright-cli resume # resume so that seed test runs fully
playwright-cli snapshot # inventory of interactive elements
playwright-cli click e5 # follow a flow
playwright-cli eval "location.href" # read URL / state
playwright-cli show --annotate # ask the user to point at something
```
Map out:
- Interactive surfaces (forms, buttons, lists, filters, modals).
- Primary user journeys end-to-end.
- Edge cases: empty states, validation errors, very long input, boundary values.
- Persistence: reload, local/session storage, URL fragments.
- Navigation: which controls change the URL, back/forward behaviour.
**Important**: Do not just open the app url with playwright-cli, always go through the test to capture any custom setup done there.
**Important**: Stop the background test when done exploring.
### 1.4 Write the spec file
Save under `specs/<feature>.plan.md`. Use this structure:
```markdown
# <Feature> Test Plan
## Application Overview
<One paragraph describing what the feature does and why it matters.>
## Test Scenarios
### 1. <Group Name>
**Seed:** `tests/seed.spec.ts`
#### 1.1. <kebab-case-scenario-name>
**File:** `tests/<group>/<kebab-case-scenario-name>.spec.ts`
**Steps:**
1. <Concrete user step>
- expect: <observable outcome>
- expect: <another observable outcome>
2. <Next step>
- expect: <outcome>
#### 1.2. <next-scenario>
...
### 2. <Next Group>
**Seed:** `tests/seed.spec.ts`
...
```
Guidelines:
- Each scenario is independent and starts from the seed's fresh state — never chain scenarios.
- Scenario names are kebab-case and match the test file name (`should-add-single-todo``should-add-single-todo.spec.ts`).
- Cover happy path, edge cases, validation, negative flows, persistence.
- Write steps at the user level ("Type 'Buy milk' into the input"), not the API level ("call `fill`").
- Put observable outcomes in `- expect:` bullets; each becomes an assertion during generation.
---
## 2. Generate
Goal: take a spec file and produce Playwright test files. Optionally update the spec if it has drifted.
### 2.1 Inputs
- **Spec file**, e.g. `specs/basic-operations.plan.md`.
- **Target**: either a single scenario (e.g. `1.2`), a whole group (`1`), or all.
- **Seed file**, read from the `**Seed:**` line of the scenario's group.
### 2.2 Generate one scenario
For each target scenario, in sequence (never in parallel — scenarios share the seed session):
```bash
PLAYWRIGHT_HTML_OPEN=never npx playwright test <seed-file> --debug=cli # background
playwright-cli attach tw-XXXX
# resume
```
**Do not** just open the app url with playwright-cli, always go through the test to capture any custom setup done there.
Walk the scenario's `Steps:` one by one with `playwright-cli`, treating the spec as the plan and the live app as the source of truth. If a step is vague ("click the button" — which button?), references an element that no longer exists, or contradicts the app's actual behaviour, use your judgement: update the spec to match what the app really does, then keep going. Editing the spec mid-generation is expected.
Every action prints the equivalent Playwright TypeScript (see [How generation works](#0-how-generation-works)):
```bash
playwright-cli snapshot # find refs
playwright-cli fill e3 "John Doe" # -> page.getByRole('textbox', {...}).fill(...)
playwright-cli press Enter
playwright-cli click e7
```
For each `- expect:` bullet, add an explicit assertion. See [How generation works](#0-how-generation-works) for details.
Collect the generated code and write the test file at the path given in the spec:
```ts
// spec: specs/basic-operations.plan.md
// seed: tests/seed.spec.ts
import { test, expect } from './fixtures'; // or '@playwright/test' if no fixtures file
test.describe('Signing in and out', () => {
test('should sign in', async ({ page }) => {
// 1. Navigate to the application
// (handled by the seed fixture)
// 2. Type 'John Doe' into the username field
await page.getByRole('textbox', { name: 'username' }).fill('John Doe');
// 3. Type password
await page.getByRole('textbox', { name: 'password' }).fill('TestPassword');
// 4. Press Enter to submit
await page.getByRole('textbox', { name: 'password' }).press('Enter');
await expect(page.getByRole('heading')).toContainText('Welcome, John Doe!');
});
});
```
Rules:
- **One test per file.** File path, describe name, and test name come verbatim from the spec (minus the ordinal).
- Prefix each numbered step with a `// N. <step text>` comment before its actions.
- Use the describe group name verbatim from the spec (no `1.` ordinal).
- Import from `./fixtures` if the project has one; otherwise `@playwright/test`.
- **Important**: close the CLI session and stop the background test before moving to the next scenario.
### 2.3 Generate multiple scenarios
Loop 2.2 over the targeted scenarios one at a time, restarting the seed between each so every test starts from a clean page. This is safe to parallelise due to unique generated session names - just make sure each test run is stopped.
### 2.4 Run generated tests
After generation, run the new tests once:
```bash
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.ts
```
Any failure goes to Section 3.
---
## 3. Heal
Goal: fix failing tests, and update the spec if the app's intended behaviour changed.
### 3.1 Find failing tests
```bash
PLAYWRIGHT_HTML_OPEN=never npx playwright test
```
Record the list of failing `<file>:<line>` entries and process them one at a time. Do not attempt parallel fixes — shared state and the single CLI session make that fragile.
### 3.2 Debug one failure
Run the single failing test in debug mode in the background, then attach:
```bash
PLAYWRIGHT_HTML_OPEN=never npx playwright test tests/<group>/<scenario>.spec.ts:<line> --debug=cli
# wait for "Debugging Instructions" and the tw-XXXX session name
playwright-cli attach tw-XXXX
```
The test is paused at the start. Step forward or run to until just before the failing action or assertion, then diagnose:
```bash
playwright-cli snapshot # did the element change / move / rename?
playwright-cli console # app-side errors?
playwright-cli requests # failed request? wrong payload?
playwright-cli show --annotate # ask the user to point somewhere
```
Common causes: selector drift, new wrapper element, label/ARIA rename, timing (transition, async load), assertion text updated in the app, test data leaking between runs.
Rehearse the corrected interaction with `playwright-cli` — the generated code in the output is what you paste back into the test.
### 3.3 Apply the fix
Edit the test file: update the locator, assertion, step order, or inputs to match the corrected behaviour. Stop the background debug run. Rerun the single test to confirm green.
Never skip hooks or add sleeps as a fix. Never use `networkidle`.
### 3.4 Reconcile with the spec
Open the spec referenced by the `// spec:` header in the test file and locate the scenario that matches the test.
- **Fix was purely technical** (locator drift, better assertion shape) and the spec's user-level behaviour still matches the app → leave the spec alone.
- **Fix changed user-visible steps, inputs, order, or expected outcomes** that the spec describes → update the spec to match reality. Keep the scenario id and file path stable; only the step / expect lines change.
- **Unclear whether the app change is intentional** (spec is stale) **or a regression** (test was right, app is wrong) → **stop and ask the user**. Provide:
- the scenario id (e.g. `2.3`),
- the spec lines that no longer match,
- the observed app behaviour (quote a snapshot excerpt or a concrete outcome).
Only after the user answers, either update the spec (intentional change) or file/flag the test as covering a bug (regression).
### 3.5 Iteration and giving up
- Fix failures one at a time; rerun after each.
- If after thorough investigation you are confident the test is correct but the app is wrong *and* the user has confirmed it's a bug: mark the test `test.fixme(...)` with a comment pointing at the user's decision or issue link. Never silently skip.
---
## Cross-references
| For... | See |
|---|---|
| `--debug=cli` / attach mechanics | [playwright-tests.md](playwright-tests.md) |
| Mocking requests during exploration/generation | [request-mocking.md](request-mocking.md) |
| Managing the CLI browser session | [session-management.md](session-management.md) |
+1 -1
View File
@@ -19,7 +19,7 @@ playwright-cli tracing-stop
## Trace Output Files
When you start tracing, Playwright creates a `traces/` directory with several files:
When you start tracing, Playwright creates a `.playwright-cli/traces/` directory with several files:
### `trace-{timestamp}.trace`
@@ -40,7 +40,7 @@ playwright-cli video-start recordings/checkout-test-run-42.webm
### 2. Record entire hero scripts.
When recording a video for the user or as a proof of work, it is best to create a code snippet and execute it with run-code.
It allows pulling appropriate pauses between the actions and annotating the video. There are new Playwright APIs for that.
It allows inserting appropriate pauses between the actions and annotating the video. There are new Playwright APIs for that.
1) Perform scenario using CLI and take note of all locators and actions. You'll need those locators to request their bounding boxes for highlight.
2) Create a file with the intended script for video (below). Use pressSequentially w/ delay for nice typing, make reasonable pauses.
+69 -3
View File
@@ -14,6 +14,7 @@
* limitations under the License.
*/
import fs from 'fs';
import path from 'path';
import { spawn } from 'child_process';
import { test, expect } from '@playwright/test';
@@ -24,7 +25,7 @@ type CliResult = {
exitCode: number | null;
};
async function runCli(...args: string[]): Promise<CliResult> {
async function runCli(args: string[], env: Record<string, string> = {}): Promise<CliResult> {
const cliPath = path.join(__dirname, '../playwright-cli.js');
return new Promise<CliResult>((resolve, reject) => {
@@ -35,6 +36,7 @@ async function runCli(...args: string[]): Promise<CliResult> {
env: {
...process.env,
PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: test.info().outputPath(),
...env,
},
cwd: test.info().outputPath(),
});
@@ -60,13 +62,77 @@ async function runCli(...args: string[]): Promise<CliResult> {
}
test('open data URL', async ({}) => {
expect(await runCli('open', 'data:text/html,hello', '--persistent')).toEqual(expect.objectContaining({
expect(await runCli(['open', 'data:text/html,hello', '--persistent'])).toEqual(expect.objectContaining({
output: expect.stringContaining('hello'),
exitCode: 0,
}));
expect(await runCli('delete-data')).toEqual(expect.objectContaining({
expect(await runCli(['delete-data'])).toEqual(expect.objectContaining({
output: expect.stringContaining('Deleted user data for'),
exitCode: 0,
}));
});
test('warns when installed skill is out of date', async ({}) => {
expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
exitCode: 0,
}));
const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'playwright-cli', 'SKILL.md');
fs.appendFileSync(skillFile, 'x');
const env = { CI: '', NO_UPDATE_NOTIFIER: '' };
expect(await runCli(['--help'], env)).toEqual(expect.objectContaining({
error: expect.stringContaining('does not match the tool version'),
}));
expect(await runCli(['--help'], env)).toEqual(expect.objectContaining({
error: expect.not.stringContaining('does not match the tool version'),
}));
});
test('does not warn when installed skill only differs in line endings', async ({}) => {
expect(await runCli(['install', '--skills'], { NO_UPDATE_NOTIFIER: '1' })).toEqual(expect.objectContaining({
exitCode: 0,
}));
const skillFile = path.join(test.info().outputPath(), '.claude', 'skills', 'playwright-cli', 'SKILL.md');
fs.writeFileSync(skillFile, fs.readFileSync(skillFile, 'utf8').replace(/\n/g, '\r\n'));
expect(await runCli(['--help'], { CI: '', NO_UPDATE_NOTIFIER: '' })).toEqual(expect.objectContaining({
error: expect.not.stringContaining('does not match the tool version'),
}));
});
test('caches the update check in the default registry directory', async ({}) => {
// Redirect the home/cache directories so the real user cache is untouched, and
// leave PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST empty so the default path is used.
const home = test.info().outputPath('home');
fs.mkdirSync(home, { recursive: true });
const env = {
CI: '',
NO_UPDATE_NOTIFIER: '',
PLAYWRIGHT_CLI_INSTALLATION_FOR_TEST: '',
HOME: home,
USERPROFILE: home,
XDG_CACHE_HOME: path.join(home, '.cache'),
LOCALAPPDATA: path.join(home, 'AppData', 'Local'),
};
expect(await runCli(['--version'], env)).toEqual(expect.objectContaining({ exitCode: 0 }));
const found: string[] = [];
const walk = (dir: string) => {
for (const entry of fs.readdirSync(dir, { withFileTypes: true })) {
const full = path.join(dir, entry.name);
if (entry.isDirectory())
walk(full);
else if (entry.name === 'cli-update-check.json')
found.push(full);
}
};
walk(home);
expect(found).toHaveLength(1);
expect(JSON.parse(fs.readFileSync(found[0], 'utf8')).lastCheck).toEqual(expect.any(Number));
});