mirror of
https://github.com/workos/skills.git
synced 2026-09-14 20:07:08 +08:00
Garen Torikian
e5b8d82641
* feat(workos): add MCP setup recovery guidance
* fix(workos): scope MCP recovery guardrail and add MCP eval coverage
Narrow the MCP-recovery guardrail so it only applies when the user wants
MCP or a configured server is broken, instead of diverting intentional
CLI workspace-management requests away from the supported CLI fallback.
Add eval cases covering direct Codex MCP recovery and a management
request with MCP intentionally absent, so regressions in the new
first-match MCP route surface as negative deltas.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(workos): defer MCP specifics to docs
* test(workos): add MCP setup and scope-selection eval case
The MCP route covered recovery (mcp-codex-startup-interrupted) and the
no-MCP management fallback (mcp-management-cli-no-mcp), but nothing
exercised first-time setup or the user-global vs per-repo scope decision
that references/workos-mcp.md guards.
Expectations are pinned to the shipped CLI rather than memory: workos/cli
src/commands/mcp.ts exposes install|remove|status, and src/lib/mcp-clients.ts
installs the Claude Code server at --scope user because the management MCP is
account-level.
Anti-patterns and hallucinations are drawn from the failure both arms
actually produced on a live run: a fabricated stdio server
({"command": "workos"}) written into a Claude Desktop config path. The case
currently scores ~77/76 (delta +1) — it detects the failure, but the skill
does not yet prevent it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(workos): restore load-bearing MCP anchors in the reference
7671643 deferred every MCP specific to the docs. When the docs are
unreachable the model has nothing to fall back on, so it invents: both eval
arms of mcp-setup-client-scope produced a local stdio server
({"command": "workos"}) written into a Claude Desktop config path, and both
still scored ~90 because the composite gives full credit for the empty
methods/imports/params/envVars dimensions.
Restore the two facts that failure hinged on — the server is remote HTTP at
https://mcp.workos.com/mcp with OAuth and no API key, and the workos CLI
configures it via mcp install|remove|status — and scope guardrail 1 so
everything else still defers to the docs.
mcp-setup-client-scope: 76% without / 91% with, delta +15%. Hallucinated
method and security issue both drop to zero in the with-skill arm.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(workos): score the MCP scope step on substance, not a flag
The step asked for `--scope user`, a flag the endorsed CLI path never
surfaces — `workos mcp install` sets it internally. Worse, the matcher's
1-of-2 keyword fallback let a bare "user" anywhere in the output satisfy it,
so the without-skill arm collected credit for guessing a config location it
had otherwise gotten wrong.
Ask instead for user scope plus the account-level reason, which needs 2 of 3
keywords co-occurring. Replayed against the previous run's transcripts:
without-skill flow drops 0.50 -> 0.20, with-skill holds at 0.53.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* ci: add lint-pr-title workflow
Enforces conventional-commit PR titles via
amannn/action-semantic-pull-request, matching the
conventional commit style this repo already follows.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>