Files
workos__skills/scripts/smoke-test-tarball.sh
Nick Nisi 5ce9743e49 feat(workos): add CLI upgrade-path topic and tarball smoke test (#26)
* feat(workos): add CLI upgrade-path topic and tarball smoke test

Closes the loop on a customer friction case where claude diagnosed an
outdated `workos` CLI but had no skill content telling it how to
recommend an upgrade. Adds:

- `references/workos-cli-upgrade.md`: router-style reference covering
  npm/pnpm/yarn install commands plus the `npx workos@latest` no-install
  fallback. Includes explicit guardrails against fabricating "the
  latest version" — instructs the agent to run `npm view workos version`
  rather than reproduce a number from memory.

- `SKILL.md`: new row in the topic→reference map and a Rule 6 sub-case
  routing outdated-CLI symptoms (old `--version`, `unknown command`
  after following recent docs) to the new reference.

- `references/workos-management.md`: "Detecting and recommending CLI
  upgrades" subsection right after the "what the CLI can do" section,
  so agents reaching for `workos --help --json` also consider whether
  the user is just on an old version.

- `evals/evals.json` (id 11, `cli-upgrade-recommendation`): asserts the
  agent points users at `npm view workos version`, provides a concrete
  upgrade command using `@latest`, and does not pin to a fabricated
  version number.

- `scripts/smoke-test-tarball.sh` + `release.yml` step: pnpm-packs the
  package, extracts it, and asserts a hardcoded canary list of skill
  files is present. Catches the failure mode where `references/` stops
  shipping in the published tarball — a silent break for downstream
  consumers (CLI auto-install) that Read these files at runtime.

* fix(workos): align CLI upgrade topic with file conventions and drop yarn

Address Devin review feedback on PR #26:

* Restructure `workos-cli-upgrade.md` to match the convention used by
  every other topic file in `references/`: `# Title` directly into
  `## Docs`, then URL list, then the standard "If this file conflicts
  with fetched docs, follow the docs." line. The "Use this when..."
  intro paragraph moves to after the boilerplate.

* Restore the standardized boilerplate verbatim ("If this file conflicts
  with fetched docs, follow the docs.") instead of the bespoke
  "...published release notes, follow the release notes." variant. The
  exact phrasing is what consumers / the LLM at runtime use to identify
  the authoritative source.

Drop yarn from the upgrade-path content per maintainer preference:

* Remove the two yarn rows from the upgrade-commands table; npm and
  pnpm only, with `npx workos@latest` as the no-install fallback for
  anyone on a different package manager.
* Drop yarn from the management subsection's mention of supported
  package managers.
* Drop the two yarn needles from the eval entry's positive-assertion
  list (it's `content_contains_any`, so the assertion still passes
  on any of npm/pnpm/npx; yarn was just unreachable noise).

Other yarn references in the repo (authkit-sveltekit, node, etc.) are
about installing `@workos-inc/*` SDK packages, not the WorkOS CLI —
out of scope for this change.
2026-04-26 15:44:03 -05:00

50 lines
1.3 KiB
Bash
Executable File

#!/usr/bin/env bash
# Pack the package, extract it, and assert the canary reference tree is present.
# Catches the failure mode where the published tarball ships an empty `references/`
# directory — which would silently break consumers (CLI auto-install, etc.) that
# Read these files at runtime.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")/.." && pwd)"
OUT="$(mktemp -d)"
trap 'rm -rf "$OUT"' EXIT
cd "$REPO_ROOT"
pnpm pack --pack-destination "$OUT" >/dev/null
cd "$OUT"
TARBALL="$(ls workos-skills-*.tgz 2>/dev/null | head -n1)"
if [ -z "$TARBALL" ]; then
echo "ERROR: pnpm pack did not produce a workos-skills-*.tgz" >&2
exit 1
fi
tar -xzf "$TARBALL"
cd package
REQUIRED=(
"plugins/workos/skills/workos/SKILL.md"
"plugins/workos/skills/workos/references/workos-management.md"
"plugins/workos/skills/workos/references/workos-rbac.md"
"plugins/workos/skills/workos/references/workos-cli-upgrade.md"
"plugins/workos/skills/workos-widgets/SKILL.md"
"plugins/workos/skills/workos-widgets/references/detection.md"
)
MISSING=()
for f in "${REQUIRED[@]}"; do
if [ ! -f "$f" ]; then
MISSING+=("$f")
fi
done
if [ ${#MISSING[@]} -gt 0 ]; then
echo "ERROR: tarball is missing required files:" >&2
for f in "${MISSING[@]}"; do
echo " - $f" >&2
done
exit 1
fi
echo "Tarball contents OK ($TARBALL, ${#REQUIRED[@]} canary files present)"