ci: add manifest-drift gate + correct stale generated artifacts (#103)

* ci: add manifest-drift gate and correct stale generated artifacts

The committed skill-manifest.json had drifted from its source: the
`workflow` skill carried 11 path patterns (**/chain-engine*, **/orchestrat*,
**/escalation*, etc.) that were deliberately removed from skills/workflow/
SKILL.md in 498c263 ("fixing strict workflow skill detection"). They were
silently reintroduced into the manifest in 5641bff (#94), which regenerated
the manifest on a branch predating that removal. The skill-catalog.md had
the same workflow drift plus a stale vercel-connect entry.

Root cause: CI runs `bun run build` (which regenerates the manifest into the
working tree) and then tests against that fresh copy — it never verifies the
*committed* manifest matches source, so a stale committed manifest passes.

Fix:
- Add `--check` to scripts/build-manifest.ts: rebuilds in-memory and compares
  to the committed file (normalizing the volatile generatedAt timestamp),
  exiting non-zero on drift. Exposed as `bun run build:manifest:check`.
- Wire build:manifest:check (and the existing build:from-skills:check) into
  CI *before* the build step, so they validate the committed artifacts on the
  pristine tree rather than after build clobbers them.
- Regenerate skill-manifest.json and skill-catalog.md to heal the drift.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix: sort skill directory scan for deterministic manifest across platforms

scanSkillsDir() iterated readdirSync() output directly, whose order is
filesystem-dependent — roughly alphabetical on macOS/APFS but arbitrary on
Linux/ext4. The committed manifest was therefore generated in macOS order,
and build:manifest:check failed on the Linux CI runner because a fresh build
produced the same skills in a different key order (CI run on #103).

Sort the entries so manifest/catalog generation is byte-identical on every
platform. Regenerates skill-manifest.json and skill-catalog.md into sorted
order (one-time reordering churn; no semantic change).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

.github/workflows/ci.yml

@@ -20,6 +20,14 @@ jobs:
       - name: Install dependencies
         run: bun install --frozen-lockfile
 
+      # Verify committed generated artifacts match their sources BEFORE the
+      # build step regenerates (and thus clobbers) them in the working tree.
+      - name: Check committed manifest is up-to-date
+        run: bun run build:manifest:check
+
+      - name: Check generated files are up-to-date
+        run: bun run build:from-skills:check
+
       - name: Build
         run: bun run build
 
@@ -29,8 +37,5 @@ jobs:
       - name: Validate skills
         run: bun run validate
 
-      - name: Check generated files are up-to-date
-        run: bun run build:from-skills:check
-
       - name: Test
         run: bun test

generated/skill-catalog.md

@@ -1,7 +1,7 @@
 # Skill Catalog
 
 > Auto-generated by `scripts/generate-catalog.ts` — do not edit manually.
-> Generated: 2026-06-09T19:40:07.166Z
+> Generated: 2026-06-26T18:59:09.655Z
 > Skills: 28
 
 ## Table of Contents
@@ -16,7 +16,7 @@
 | Skill | Priority | Path Patterns | Bash Patterns | Import Patterns |
 |-------|----------|---------------|---------------|-----------------|
 | `knowledge-update` | 10 | 1 | 0 | 0 |
-| `workflow` | 9 | 17 | 14 | 4 |
+| `workflow` | 9 | 6 | 14 | 4 |
 | `ai-sdk` | 8 | 27 | 11 | 2 |
 | `bootstrap` | 8 | 10 | 6 | 8 |
 | `chat-sdk` | 8 | 45 | 12 | 2 |
@@ -36,7 +36,7 @@
 | `runtime-cache` | 6 | 4 | 4 | 0 |
 | `shadcn` | 6 | 7 | 6 | 0 |
 | `nextjs` | 5 | 15 | 7 | 0 |
-| `vercel-connect` | 5 | 0 | 6 | 3 |
+| `vercel-connect` | 5 | 2 | 6 | 4 |
 | `react-best-practices` | 4 | 8 | 0 | 2 |
 | `turbopack` | 4 | 1 | 2 | 0 |
 | `vercel-agent` | 4 | 6 | 1 | 0 |
@@ -60,17 +60,6 @@
 - `src/lib/workflow.*`
 - `workflow.*`
 - `*workflow*`
-- `**/chain-engine*`
-- `**/chain_engine*`
-- `**/chainEngine*`
-- `**/pipeline-engine*`
-- `**/pipeline_engine*`
-- `**/pipelineEngine*`
-- `**/state-machine*`
-- `**/state_machine*`
-- `**/stateMachine*`
-- `**/orchestrat*`
-- `**/escalation*`
 
 **Bash patterns:**
 - `\bnpm\s+(install|i|add)\s+[^\n]*@vercel/workflow\b`
@@ -764,6 +753,10 @@
 
 #### `vercel-connect` (priority 5)
 
+**Path patterns:**
+- `agent/connections/**`
+- `agent/channels/**`
+
 **Bash patterns:**
 - `\bvercel\s+connect\b`
 - `\bvc\s+connect\b`
@@ -774,6 +767,7 @@
 
 **Import patterns:**
 - `@vercel/connect`
+- `@vercel/connect/eve`
 - `@vercel/connect/authjs`
 - `@vercel/connect/betterauth`
 

hooks/skill-map-frontmatter.mjs

@@ -344,7 +344,7 @@ function scanSkillsDir(rootDir) {
   const diagnostics = [];
   let entries;
   try {
-    entries = readdirSync(rootDir);
+    entries = readdirSync(rootDir).sort();
   } catch {
     return { skills, diagnostics };
   }

hooks/src/skill-map-frontmatter.mts

@@ -610,7 +610,11 @@ export function scanSkillsDir(rootDir: string): ScanResult {
   const diagnostics: Diagnostic[] = [];
   let entries: string[];
   try {
-    entries = readdirSync(rootDir) as string[];
+    // Sort for deterministic ordering: readdirSync returns entries in
+    // filesystem-dependent order (alphabetical-ish on macOS/APFS, arbitrary
+    // on Linux/ext4), which would make the generated manifest differ by
+    // platform and break the build:manifest:check drift gate in CI.
+    entries = (readdirSync(rootDir) as string[]).sort();
   } catch {
     return { skills, diagnostics };
   }

package.json

@@ -8,6 +8,7 @@
   "scripts": {
     "build:hooks": "tsup --config hooks/tsup.config.ts",
     "build:manifest": "bun run scripts/build-manifest.ts",
+    "build:manifest:check": "bun run scripts/build-manifest.ts --check",
     "build:from-skills": "bun run scripts/build-from-skills.ts",
     "build:skills": "bun run scripts/build-skills.ts",
     "build:skills:check": "bun run scripts/build-skills.ts --check",

scripts/build-manifest.ts

@@ -9,7 +9,7 @@
  */
 
 import { resolve, join } from "node:path";
-import { writeFileSync, mkdirSync } from "node:fs";
+import { writeFileSync, mkdirSync, readFileSync } from "node:fs";
 
 // Import the canonical skill-map builder (ESM)
 import { globToRegex, importPatternToRegex } from "../hooks/patterns.mjs";
@@ -195,16 +195,47 @@ function buildManifest(skillsDir: string): { manifest: Manifest; warnings: strin
   return { manifest, warnings: allWarnings, errors: [] };
 }
 
+/** Serialize a manifest exactly as it is written to disk. */
+function serializeManifest(manifest: Manifest): string {
+  return JSON.stringify(manifest, null, 2) + "\n";
+}
+
 /**
  * Write the manifest JSON to generated/skill-manifest.json.
  * Returns the number of skills written.
  */
 function writeManifestFile(manifest: Manifest, outDir = OUT_DIR, outFile = OUT_FILE): number {
   mkdirSync(outDir, { recursive: true });
-  writeFileSync(outFile, JSON.stringify(manifest, null, 2) + "\n");
+  writeFileSync(outFile, serializeManifest(manifest));
   return Object.keys(manifest.skills).length;
 }
 
+/**
+ * Normalize a serialized manifest for drift comparison by neutralizing the
+ * volatile `generatedAt` timestamp, which changes on every build.
+ */
+function normalizeManifestForCompare(serialized: string): string {
+  return serialized.replace(/"generatedAt":\s*"[^"]*"/, '"generatedAt": "<normalized>"');
+}
+
+/**
+ * Compare a freshly-built manifest against the committed file.
+ * Returns true when they match (ignoring the generatedAt timestamp).
+ */
+function checkManifestFile(manifest: Manifest, outFile = OUT_FILE): { ok: boolean; reason?: string } {
+  let committed: string;
+  try {
+    committed = readFileSync(outFile, "utf8");
+  } catch {
+    return { ok: false, reason: `${outFile} is missing` };
+  }
+  const fresh = serializeManifest(manifest);
+  if (normalizeManifestForCompare(committed) !== normalizeManifestForCompare(fresh)) {
+    return { ok: false, reason: `${outFile} is out of sync with skill sources` };
+  }
+  return { ok: true };
+}
+
 // ---------------------------------------------------------------------------
 // CLI entry point (only when run directly)
 // ---------------------------------------------------------------------------
@@ -218,6 +249,7 @@ function isMain() {
 }
 
 if (isMain()) {
+  const check = process.argv.slice(2).includes("--check");
   const { manifest, warnings, errors } = buildManifest(SKILLS_DIR);
 
   for (const w of warnings) console.warn(`[warn] ${w}`);
@@ -228,6 +260,16 @@ if (isMain()) {
     process.exit(1);
   }
 
-  const count = writeManifestFile(manifest);
-  console.log(`✓ Wrote ${count} skills to ${OUT_FILE}`);
+  if (check) {
+    const { ok, reason } = checkManifestFile(manifest);
+    if (!ok) {
+      console.error(`[error] ${reason}.`);
+      console.error("        Run `bun run build:manifest` and commit the result.");
+      process.exit(1);
+    }
+    console.log(`✓ skill-manifest.json is up-to-date (${Object.keys(manifest.skills).length} skills)`);
+  } else {
+    const count = writeManifestFile(manifest);
+    console.log(`✓ Wrote ${count} skills to ${OUT_FILE}`);
+  }
 }