Files
kartik-georgian 0413bc0368 Skill eval CI: run eval-experiments harness against every PR touching plugins/expo/skills (#105)
## Summary

Skill-eval CI for PRs that touch `plugins/expo/skills/**`.

Runs the [eval-experiments](https://github.com/expo/eval-experiments)
harness against all 3 reference PRDs (notes, hot_chocolate,
wiki_reader). Each is authored with Claude Code loading the skill plugin
**straight from this PR's own checked-out `plugins/expo/` directory**
(via a new `SKILL_PLUGIN_DIR` env var → `--plugin-dir`, added in
eval-experiments#10), not the published marketplace version — so this
actually tests the PR's proposed changes, not what's already released.

## Resolved: pull_request checkout uses the PR's head commit

Confirmed empirically (not documented in Expo's docs or the public
eas-cli source, so we tested it directly): opened a real test PR against
eval-experiments with a GitHub App connection in place, edited a tracked
marker file, and confirmed the automatically-triggered run's checkout
reflected the edited value — not the base branch. This confirms
`SKILL_PLUGIN_DIR` will work as designed once this repo's connection is
live: a real PR's proposed skill changes will actually get checked out
and tested, not what's already on `main`.

## Simplified: Expo MCP auth no longer needs a provisioning job at all

mcp.expo.dev now accepts `EXPO_TOKEN` directly as its bearer token
(verified live, including a real Claude Code session successfully
calling an MCP tool). This removed the whole `provision_mcp_token` job
and its `EXPO_MCP_BEARER_TOKEN`/`EXPO_MCP_AUTH_STATUS` wiring from all 3
`author_app_*` jobs — see eval-experiments#15. Submodule pin bumped to
track this. `EXPO_MCP_CLIENT_ID`/`EXPO_MCP_REFRESH_TOKEN` secrets on
this project are no longer needed (can be cleaned up, not required for
function).
2026-07-15 10:34:00 -04:00

5 lines
47 B
JSON

{
"name": "expo-skills",
"private": true
}