From 31592f784d9c27d4de4e040be0dc9260b488f614 Mon Sep 17 00:00:00 2001 From: zernie Date: Wed, 17 Jun 2026 02:03:39 +0700 Subject: [PATCH] feat: select harness via project config, mirror instruction files, verify skill frontmatter (#36) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(cli): select harness via project config + mirror instruction files Add a `harness` key (string | string[]) to .vigilesrc.json, written by `vigiles init` and honoured by compile/lint, plus a `--harness=` override on compile — a deterministic, committed replacement for sniffing the cwd that kills the silent harness-mismatch footgun in the skill/agent compile path. When a repo declares >=2 harnesses and no sync tool or existing mirror fans the instruction file out, compile writes a byte-identical CLAUDE.md<->AGENTS.md copy (carrying the source's integrity hash, so a hand-edit trips the existing check). Selection is a discriminated union (a notice exists only on its variant) and the declared set is parsed once at the boundary — per the make-invalid-states- irrepresentable / parse-don't-validate guidance now recorded in CLAUDE.md. Design: research/multi-harness-compile.md. * feat(cli): warn when a declared minimal-profile harness drops skill frontmatter Slice 3 (verify half) of multi-harness compile. A skill's references are harness-agnostic; the one harness-specific surface is the SKILL.md frontmatter profile. `minimal` (Codex/OpenCode) is a strict subset of `claude-code`, so the useful, assumption-free check is the inverse: when a declared harness uses the minimal profile it DROPS the CC-only keys (disable-model-invocation, argument-hint), so a constraint the author set won't take effect there. `vigiles compile` now emits that warning per declared minimal-profile harness (src/skill-harness.ts). Per-root multi-emit stays deferred — output location is co-located today and minimal-profile key tolerance is unverified (see the design doc). Design: research/multi-harness-compile.md. * fix(cli): per-spec target disambiguation for compile dialect The harness-selection precedence documented a spec-target step that the code skipped: compile resolved one dialect per run (flag → config → detect) and ignored each instruction spec's own target. So an AGENTS.md.spec.ts could compile with the claude-code dialect — the silent mismatch the feature exists to kill. A CLAUDE.md.spec.ts is a claude-code file, an AGENTS.md.spec.ts a codex one, so the spec's target filename now selects its dialect (adapterForInstructionFile), above config/detect; the flag still overrides, and skill/agent specs (whose targets don't name a harness) keep the run-level pick. Also documents why `lint` takes no `--harness`: reference verification is harness-agnostic (it already recognizes both CLAUDE.md and AGENTS.md), unlike compile (renders one dialect) and scan (reports harness-specific structure). * docs: design for unifying scan + lint on one rule engine scan and lint are two parallel implementations with partial overlap: scan's structural findings (no-description skill, no-tool-contract agent, missing hook) are hard-coded in scan.ts/leaderboard.ts — not documented, configurable, or CI-gatable, unlike the half-shared untested-surface/orphan-docs rules. Specs Option B (the ESLint model): promote those findings to first-class rules; lint runs the configured set + gates, scan runs the structural subset with zero-config defaults + renders inventory + a rule-derived score. Rejects "scan = lint without config" (lint's reference core needs marks a bare repo lacks). Design only, separate initiative; linked from roadmap. * docs: document harness selection + save the snapshots conclusion User-facing docs for the shipped multi-harness work: a "compile — harness selection" section in docs/cli.md (the precedence, --harness=, the byte-identical mirror, the frontmatter-drop warning, why lint takes no --harness) and the `harness` config key in the docs/spec-format.md config table. Also records a session conclusion in research/testing-api-design.md: snapshots split three ways (output-text = reject, record/replay cassette = can't catch model drift, trace-structure = the viable deterministic regression tool), with the structural-snapshot vs statistical-baseline split. * docs: document scan's harness detection + --harness override * docs: deprecate the two demo scripts, add demo revamp to roadmap * docs: fix roadmap demo-revamp bullet formatting * test: close multi-harness coverage gaps - Extract mergeProjectConfig (pure) from writeProjectConfig and unit-test its branches: records harness when absent, never clobbers an existing harness, strict adds rule severities without overwriting, fully-satisfied → no write. - E2E: spec-target disambiguation — a target-less AGENTS.md.spec.ts compiles with the codex dialect (AGENTS.md heading), proving the per-spec wiring, not just the pure adapterForInstructionFile mapping. - E2E: mirror idempotence (an already-identical target isn't re-written) and compile --harness=bogus fails with an actionable "Unknown harness" error. - Assert the init default records harness "claude-code" on a greenfield repo. --------- Co-authored-by: Claude --- CLAUDE.md | 8 +- CLAUDE.md.spec.ts | 6 +- docs/cli.md | 44 ++++++ docs/harnesses.md | 17 ++- docs/spec-format.md | 15 +- examples/demo/README.md | 6 + examples/plugin-test-demo.mjs | 5 + src/adapter-registry.test.ts | 174 +++++++++++++++++++++++ src/adapter-registry.ts | 103 +++++++++++++- src/cli.test.ts | 257 ++++++++++++++++++++++++++++++++++ src/cli.ts | 184 ++++++++++++++++++++---- src/core/types.ts | 10 ++ src/setup-plan.test.ts | 73 ++++++++++ src/setup-plan.ts | 30 ++++ src/skill-harness.test.ts | 71 ++++++++++ src/skill-harness.ts | 56 ++++++++ 16 files changed, 1010 insertions(+), 49 deletions(-) create mode 100644 src/adapter-registry.test.ts create mode 100644 src/skill-harness.test.ts create mode 100644 src/skill-harness.ts diff --git a/CLAUDE.md b/CLAUDE.md index 7f4b9d9b..437364b2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,4 +1,4 @@ - + # CLAUDE.md @@ -40,7 +40,7 @@ Harness-adapter layout (hexagonal — see `research/code-adapter-architecture.md - `src/adapters/claude-code/` — the Claude Code ADAPTER: the swappable ports + bundle + harness glue (the `HarnessDialect`/`PluginLayout`/`HarnessRuntime`/`HookProtocol`/`ModelMock` port impls, the `HarnessAdapter` bundle, agent/skill runtime, run-scripts, and a thin plugin-loader wrapper that defaults the CC layout). A future `src/adapters//` mirrors it. - `src/` root — the application/composition layer AND the harness-testing LIBRARY: cli, scan, the harness-agnostic `plugin-loader` (layout-injected; the CC wrapper + every other adapter delegate to it, so no adapter imports a sibling), the deterministic runners (`harness-test`/`run-hook`/`eval` + `mock-model`/`sandbox`/`egress`/`judge`) that default to Claude Code via an injected `{ adapter }`, and the `testing`/`unit`/`integration`/`e2e` barrels that route through the runners and NEVER import an adapter directly. See `research/adapter-api-design.md`. -Two boundary rules are enforced by `eslint-plugin-boundaries` (rule `boundaries/dependencies`, error-mode, classified by directory) and dogfooded via `enforce("boundaries/dependencies")`: core ⊄ adapter (the domain stays harness-agnostic), and the `agnostic-surface` (`src/{testing,unit,integration,e2e}.ts`) ⊄ any adapter (the public agnostic entries route through the composition-root runners, so "agnostic" is enforced, not just named) — the architecture invariant is a verified reference, not a convention. Consumers select a harness by import (`vigiles/claude-code` beside the agnostic `vigiles/testing`), never a config key; the CLI auto-detects. +Two boundary rules are enforced by `eslint-plugin-boundaries` (rule `boundaries/dependencies`, error-mode, classified by directory) and dogfooded via `enforce("boundaries/dependencies")`: core ⊄ adapter (the domain stays harness-agnostic), and the `agnostic-surface` (`src/{testing,unit,integration,e2e}.ts`) ⊄ any adapter (the public agnostic entries route through the composition-root runners, so "agnostic" is enforced, not just named) — the architecture invariant is a verified reference, not a convention. Consumers select a harness by import (`vigiles/claude-code` beside the agnostic `vigiles/testing`), not a runtime config key; the CLI selects per repo — auto-detect, a `--harness=` override, or a `harness` key in `.vigilesrc.json` written by `init` (the project-level declaration, distinct from the library's import-time selection; an array for a repo targeting several harnesses also drives a byte-identical `CLAUDE.md`⇄`AGENTS.md` mirror when no sync tool fans it out). See `research/multi-harness-compile.md`. ## Key Files @@ -76,7 +76,7 @@ Two boundary rules are enforced by `eslint-plugin-boundaries` (rule `boundaries/ - `research/opencode-prototype-findings.md` — OpenCode prototype findings: the internal non-shipped src/adapters/opencode/ adapter that validates the AdapterCapabilities tier on the row that splits the matrix — mockable (harnessTesting) but in-process code-module hooks (shellHooks:false, no hookProtocol). Verdict (the tier holds: conformance accepts it without a fake hook protocol) + the gaps (Chat-Completions SSE renderer not built, no opencode binary, the shell-hook runHook tier simply never applies, wireMock op needed again). Grounds the capability matrix in docs/harnesses.md - `src/adapter.ts` — `vigiles/adapter` — the harness-adapter authoring kit (the documented small lib third parties use to build their own adapter): re-exports the five port interfaces + HarnessAdapter + the conformance kit + the registry. See docs/authoring-an-adapter.md - `src/codex.ts` — `vigiles/codex` — the OpenAI Codex adapter surface (sibling of vigiles/claude-code): re-exports the Codex ports + codexAdapter + codexDriver + the proven Responses mock (startCodexMock/renderResponsesSSE + codexMockArgs/codexMockEnv). Pillar 2 is full and usable via runHarnessTest({ adapter: codexAdapter }) against the real codex binary; pillar 1 compile is format-correct for instructions (AGENTS.md) + skills (minimal SKILL.md), subagents excluded by design (model mismatch) -- `src/adapter-registry.ts` — Adapter registry (composition root): ADAPTERS = [claudeCodeAdapter, codexAdapter] + detectAdapterResult/detectAdapter (highest detect() specificity wins, reports ambiguousWith for a repo that matches several, else Claude Code — backwards-compatible) + resolveAdapter(root, harness?) honouring a --harness override + getAdapter(name). The CLI auto-detects through this (scan prints the harness + an ambiguity warning); the library selects by import +- `src/adapter-registry.ts` — Adapter registry (composition root): ADAPTERS = [claudeCodeAdapter, codexAdapter] + detectAdapterResult/detectAdapter (highest detect() specificity wins, reports ambiguousWith for a repo that matches several, else Claude Code — backwards-compatible) + resolveAdapter(root, harness?) honouring a --harness override + getAdapter(name) (alias-aware: claude → claude-code) + resolveHarnessSelection({root,flag,configHarness}) — the pure compile/lint picker with explicit precedence (flag → single config harness → first-of-many with a loud notice → auto-detect + ambiguity warning), the deterministic replacement for cwd-sniffing. The CLI auto-detects through this (scan prints the harness + an ambiguity warning); the library selects by import - `src/adapter-conformance.ts` — Adapter conformance kit: checkAdapterConformance/assertAdapterConformance (each port populated + cross-port invariants — port names agree, instructionFile is a declared target, plugin-root tokens match — + the dialect accepts its own built-in tool through compileAgent) and assertAdapterLoadsHooks (the behavioural settings round-trip that catches the JSON-vs-TOML layout trap). Third-party adapter authors drop both in their tests - `src/core/cedar.test.ts` — Cedar policy resolution tests — filesystem-based @id() lookup with filename fallback - `src/core/generate-types.ts` — Type generator: scans linters/package.json/filesystem → emits .d.ts @@ -341,7 +341,7 @@ Two boundary rules are enforced by `eslint-plugin-boundaries` (rule `boundaries/ ### Ts Essentials -**Guidance only** — Prefer branded types over plain strings for semantic values (hashes, file paths, rule IDs). Use discriminated unions over boolean flags that gate optional fields. Add exhaustive `default: assertNever(x)` to every switch on a union type. These patterns convert runtime bugs into compile-time errors. +**Guidance only** — Prefer branded types over plain strings for semantic values (hashes, file paths, rule IDs). Use tagged/discriminated unions freely whenever they help — over boolean flags that gate optional fields, and to MAKE INVALID STATES IRREPRESENTABLE (e.g. a result that carries a `notice` only on its `kind:"notice"` variant, never a nullable string the caller can forget to check). Add exhaustive `default: assertNever(x)` to every switch on a union type. Follow PARSE, DON'T VALIDATE: parse raw/untrusted input (a config value, a `string | string[]`, a CLI flag) into a typed, normalized representation ONCE at the boundary, then pass that typed value around — don't re-validate or re-normalize the primitive at each use site. These patterns convert runtime bugs into compile-time errors. ### Prod Grade Gha Cli diff --git a/CLAUDE.md.spec.ts b/CLAUDE.md.spec.ts index 0b22cb5c..1429a843 100644 --- a/CLAUDE.md.spec.ts +++ b/CLAUDE.md.spec.ts @@ -42,7 +42,7 @@ Harness-adapter layout (hexagonal — see \`research/code-adapter-architecture.m - \`src/adapters/claude-code/\` — the Claude Code ADAPTER: the swappable ports + bundle + harness glue (the \`HarnessDialect\`/\`PluginLayout\`/\`HarnessRuntime\`/\`HookProtocol\`/\`ModelMock\` port impls, the \`HarnessAdapter\` bundle, agent/skill runtime, run-scripts, and a thin plugin-loader wrapper that defaults the CC layout). A future \`src/adapters//\` mirrors it. - \`src/\` root — the application/composition layer AND the harness-testing LIBRARY: cli, scan, the harness-agnostic \`plugin-loader\` (layout-injected; the CC wrapper + every other adapter delegate to it, so no adapter imports a sibling), the deterministic runners (\`harness-test\`/\`run-hook\`/\`eval\` + \`mock-model\`/\`sandbox\`/\`egress\`/\`judge\`) that default to Claude Code via an injected \`{ adapter }\`, and the \`testing\`/\`unit\`/\`integration\`/\`e2e\` barrels that route through the runners and NEVER import an adapter directly. See \`research/adapter-api-design.md\`. -Two boundary rules are enforced by \`eslint-plugin-boundaries\` (rule \`boundaries/dependencies\`, error-mode, classified by directory) and dogfooded via \`enforce("boundaries/dependencies")\`: core ⊄ adapter (the domain stays harness-agnostic), and the \`agnostic-surface\` (\`src/{testing,unit,integration,e2e}.ts\`) ⊄ any adapter (the public agnostic entries route through the composition-root runners, so "agnostic" is enforced, not just named) — the architecture invariant is a verified reference, not a convention. Consumers select a harness by import (\`vigiles/claude-code\` beside the agnostic \`vigiles/testing\`), never a config key; the CLI auto-detects.`, +Two boundary rules are enforced by \`eslint-plugin-boundaries\` (rule \`boundaries/dependencies\`, error-mode, classified by directory) and dogfooded via \`enforce("boundaries/dependencies")\`: core ⊄ adapter (the domain stays harness-agnostic), and the \`agnostic-surface\` (\`src/{testing,unit,integration,e2e}.ts\`) ⊄ any adapter (the public agnostic entries route through the composition-root runners, so "agnostic" is enforced, not just named) — the architecture invariant is a verified reference, not a convention. Consumers select a harness by import (\`vigiles/claude-code\` beside the agnostic \`vigiles/testing\`), not a runtime config key; the CLI selects per repo — auto-detect, a \`--harness=\` override, or a \`harness\` key in \`.vigilesrc.json\` written by \`init\` (the project-level declaration, distinct from the library's import-time selection; an array for a repo targeting several harnesses also drives a byte-identical \`CLAUDE.md\`⇄\`AGENTS.md\` mirror when no sync tool fans it out). See \`research/multi-harness-compile.md\`.`, }, keyFiles: { @@ -111,7 +111,7 @@ Two boundary rules are enforced by \`eslint-plugin-boundaries\` (rule \`boundari "src/codex.ts": "`vigiles/codex` — the OpenAI Codex adapter surface (sibling of vigiles/claude-code): re-exports the Codex ports + codexAdapter + codexDriver + the proven Responses mock (startCodexMock/renderResponsesSSE + codexMockArgs/codexMockEnv). Pillar 2 is full and usable via runHarnessTest({ adapter: codexAdapter }) against the real codex binary; pillar 1 compile is format-correct for instructions (AGENTS.md) + skills (minimal SKILL.md), subagents excluded by design (model mismatch)", "src/adapter-registry.ts": - "Adapter registry (composition root): ADAPTERS = [claudeCodeAdapter, codexAdapter] + detectAdapterResult/detectAdapter (highest detect() specificity wins, reports ambiguousWith for a repo that matches several, else Claude Code — backwards-compatible) + resolveAdapter(root, harness?) honouring a --harness override + getAdapter(name). The CLI auto-detects through this (scan prints the harness + an ambiguity warning); the library selects by import", + "Adapter registry (composition root): ADAPTERS = [claudeCodeAdapter, codexAdapter] + detectAdapterResult/detectAdapter (highest detect() specificity wins, reports ambiguousWith for a repo that matches several, else Claude Code — backwards-compatible) + resolveAdapter(root, harness?) honouring a --harness override + getAdapter(name) (alias-aware: claude → claude-code) + resolveHarnessSelection({root,flag,configHarness}) — the pure compile/lint picker with explicit precedence (flag → single config harness → first-of-many with a loud notice → auto-detect + ambiguity warning), the deterministic replacement for cwd-sniffing. The CLI auto-detects through this (scan prints the harness + an ambiguity warning); the library selects by import", "src/adapter-conformance.ts": "Adapter conformance kit: checkAdapterConformance/assertAdapterConformance (each port populated + cross-port invariants — port names agree, instructionFile is a declared target, plugin-root tokens match — + the dialect accepts its own built-in tool through compileAgent) and assertAdapterLoadsHooks (the behavioural settings round-trip that catches the JSON-vs-TOML layout trap). Third-party adapter authors drop both in their tests", "src/core/cedar.test.ts": @@ -540,7 +540,7 @@ Two boundary rules are enforced by \`eslint-plugin-boundaries\` (rule \`boundari ), "ts-essentials": guidance( - "Prefer branded types over plain strings for semantic values (hashes, file paths, rule IDs). Use discriminated unions over boolean flags that gate optional fields. Add exhaustive `default: assertNever(x)` to every switch on a union type. These patterns convert runtime bugs into compile-time errors.", + "Prefer branded types over plain strings for semantic values (hashes, file paths, rule IDs). Use tagged/discriminated unions freely whenever they help — over boolean flags that gate optional fields, and to MAKE INVALID STATES IRREPRESENTABLE (e.g. a result that carries a `notice` only on its `kind:\"notice\"` variant, never a nullable string the caller can forget to check). Add exhaustive `default: assertNever(x)` to every switch on a union type. Follow PARSE, DON'T VALIDATE: parse raw/untrusted input (a config value, a `string | string[]`, a CLI flag) into a typed, normalized representation ONCE at the boundary, then pass that typed value around — don't re-validate or re-normalize the primitive at each use site. These patterns convert runtime bugs into compile-time errors.", ), "prod-grade-gha-cli": guidance( diff --git a/docs/cli.md b/docs/cli.md index f9967d07..d6b39f05 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -59,6 +59,43 @@ scaffolded `vigiles.harness.mjs` resolves `vigiles/testing`. See the [agent setup guide](agent-setup.md) and [agent workflows](agent-workflows.md). +### `compile [files...]` — harness selection + +`compile` renders each `.spec.ts` to its instruction file / `SKILL.md` / +subagent. Which **harness dialect** it renders (the `SKILL.md` frontmatter +profile, the subagent tool catalog) is resolved deterministically — no cwd +sniffing: + +1. `--harness=` flag — wins (`claude-code`/`codex`; `claude` is an alias). +2. The **spec's own target** for an instruction file — a `CLAUDE.md.spec.ts` is + claude-code, an `AGENTS.md.spec.ts` is codex. +3. The **`harness` key** in `.vigilesrc.json` (written by `init`): + `"codex"`, or `["claude-code", "codex"]` to declare a multi-harness repo (the + first is used, with a loud notice; override per run with `--harness=`). +4. Auto-detect from the repo, warning when it's ambiguous. + +```bash +npx vigiles compile # all specs, harness from config/detect +npx vigiles compile --harness=codex # force the Codex dialect for this run +``` + +Two multi-harness behaviours: + +- **Instruction-file mirror.** When `harness` declares ≥2 harnesses and no sync + tool (Ruler/rulesync) or existing mirror fans the file out, `compile` writes a + **byte-identical** `CLAUDE.md`⇄`AGENTS.md` copy. It carries the source's + integrity hash, so a hand-edit of the mirror trips the `integrity` check. It + never clobbers a target that has its own spec. +- **Frontmatter-drop warning.** A skill that sets Claude-Code-only frontmatter + (`disable-model-invocation`, `argument-hint`) in a repo that also declares a + `minimal`-profile harness (Codex/OpenCode) gets a warning — those keys are + dropped there, so the constraint won't apply. + +`lint` takes **no** `--harness`: reference verification is harness-agnostic (it +already recognizes both `CLAUDE.md` and `AGENTS.md`), unlike `compile` (renders +one dialect) and `scan` (reports harness-specific structure). See +[research/multi-harness-compile.md](../research/multi-harness-compile.md). + ### `scan [dir]` Point vigiles at any plugin or repo (defaults to `.`) and get a read-only report @@ -70,10 +107,17 @@ footgun), hook scripts resolved across the braced/unbraced `$CLAUDE_PLUGIN_ROOT` forms (`ok` / `missing` / `unresolved`), command + MCP detection, untested-surface count, and the loader's dangling-ref / surface warnings. `--json` for CI. +`scan` reports **harness-specific structure** (plugin layout, hook resolution), +so it auto-detects the harness — printing the detected one and warning when a repo +matches several — and takes `--harness=` to override. (`compile` is +harness-aware for the same reason; `lint` isn't — reference verification is +harness-agnostic.) + ```bash npx vigiles scan ./some-plugin # human-readable report for one plugin npx vigiles scan ./some-plugin --json # structured, for pipelines npx vigiles scan ./plugins/*/ # ≥2 targets → ranked health leaderboard +npx vigiles scan ./repo --harness=codex # override harness detection ``` Pass **more than one directory** and `scan` switches to a **ranked health diff --git a/docs/harnesses.md b/docs/harnesses.md index a61c4170..e9c90698 100644 --- a/docs/harnesses.md +++ b/docs/harnesses.md @@ -45,12 +45,23 @@ Nothing in `vigiles/testing` changes, and unused adapters tree-shake out. Importing the adapter (rather than a `harness:` config key) means the bundle only carries the adapter you use, and the choice is type-checked where you write it. -## The CLI auto-detects +## The CLI auto-detects (or reads project config) The CLI can't take an import, so `vigiles compile`, `vigiles scan`, and `vigiles lint` **detect** the harness from the repo (a `.claude-plugin/`, an -`AGENTS.md`, …) and work with zero config; a `vigiles.config` override is the -planned escape hatch if detection is ever ambiguous. +`AGENTS.md`, …) and work with zero config. When detection is ambiguous, or you +want a deterministic, committed choice, set it explicitly — both override the +sniff: + +- a **`harness` key** in `.vigilesrc.json` (`"codex"`, or `["claude-code", +"codex"]` for a repo targeting several), written by `vigiles init`; +- a **`--harness=`** flag on the command, which wins over the config. + +The precedence is `--harness=` → config `harness` → auto-detect, and a +multi-harness or ambiguous pick prints a loud notice rather than silently +guessing. A repo declaring several harnesses also gets a byte-identical +`CLAUDE.md`⇄`AGENTS.md` mirror on compile when no sync tool already fans it out. +See [research/multi-harness-compile.md](../research/multi-harness-compile.md). ## What's actually harness-specific (the two axes) diff --git a/docs/spec-format.md b/docs/spec-format.md index 325f96ee..398da900 100644 --- a/docs/spec-format.md +++ b/docs/spec-format.md @@ -210,13 +210,14 @@ export default defineConfig({ }); ``` -| Option | Type | Description | -| ----------- | --------- | ------------------------------------------------------- | -| `specs` | `string` | Glob pattern to discover spec files | -| `discover` | `boolean` | Auto-discover linter rules for coverage reporting | -| `maxRules` | `number` | Compilation fails if a spec exceeds this rule count | -| `maxTokens` | `number` | Compilation fails if estimated tokens exceed this limit | -| `orphans` | `object` | Orphan-docs scan globs (see below) | +| Option | Type | Description | +| ----------- | ---------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `specs` | `string` | Glob pattern to discover spec files | +| `discover` | `boolean` | Auto-discover linter rules for coverage reporting | +| `maxRules` | `number` | Compilation fails if a spec exceeds this rule count | +| `maxTokens` | `number` | Compilation fails if estimated tokens exceed this limit | +| `orphans` | `object` | Orphan-docs scan globs (see below) | +| `harness` | `string` \| `string[]` | The harness(es) this repo targets — `"codex"`, or `["claude-code", "codex"]`. Selects the compile dialect; written by `init`. Omitted → auto-detect. See [CLI: compile](cli.md#compile-files--harness-selection) | ### Orphan-docs configuration diff --git a/examples/demo/README.md b/examples/demo/README.md index a24f51a5..863b952f 100644 --- a/examples/demo/README.md +++ b/examples/demo/README.md @@ -1,5 +1,11 @@ # vigiles — 60-second demo +> **⚠️ Deprecated — pending a demo revamp.** This curated demo still runs +> (`npm run demo`), but it's no longer the surfaced front-door demo and isn't +> actively maintained. It's slated for consolidation into a single, polished +> demo story alongside `vigiles scan` — tracked on the +> [roadmap](../../research/roadmap.md) (Demo revamp). Don't build on it. + `INSTRUCTIONS.md` reads fine. But two of its references **lie** — and `vigiles lint` catches both, while the two truthful ones pass silently. diff --git a/examples/plugin-test-demo.mjs b/examples/plugin-test-demo.mjs index 148a5fcb..e0d91f81 100644 --- a/examples/plugin-test-demo.mjs +++ b/examples/plugin-test-demo.mjs @@ -3,6 +3,11 @@ * * npm run demo:plugin (or: node examples/plugin-test-demo.mjs) * + * ⚠️ DEPRECATED — pending a demo revamp. Still runs, but no longer a surfaced + * front-door demo and not actively maintained; slated for consolidation into one + * polished demo story alongside `vigiles scan`. See research/roadmap.md + * (Demo revamp). Don't build on it. + * * It narrates, in plain words, what vigiles checks about a third-party plugin — * what it ships, what one of its hooks does, and what it phones home to — using a * real, popular plugin (oh-my-claudecode, ~36k★) vendored under examples/harness/. diff --git a/src/adapter-registry.test.ts b/src/adapter-registry.test.ts new file mode 100644 index 00000000..ff018a82 --- /dev/null +++ b/src/adapter-registry.test.ts @@ -0,0 +1,174 @@ +import { test } from "vitest"; +import assert from "node:assert/strict"; +import { writeFileSync } from "node:fs"; +import { join } from "node:path"; + +import { + normalizeHarnessName, + normalizeHarnessList, + resolveHarnessSelection, + adapterForInstructionFile, + type HarnessSelection, +} from "./adapter-registry.js"; +import { makeTmpDir, cleanupTmpDir } from "./core/test-utils.js"; + +/** Narrow a selection to its `notice` variant (asserts the discriminant). */ +function noticeOf(sel: HarnessSelection): string { + assert.equal(sel.kind, "notice"); + // The union makes `notice` reachable only on the "notice" variant. + return sel.kind === "notice" ? sel.notice : ""; +} + +test("normalizeHarnessName lowercases, trims, and aliases claude → claude-code", () => { + assert.equal(normalizeHarnessName("claude"), "claude-code"); + assert.equal(normalizeHarnessName(" Claude "), "claude-code"); + assert.equal(normalizeHarnessName("Codex"), "codex"); + assert.equal(normalizeHarnessName("claude-code"), "claude-code"); +}); + +test("normalizeHarnessList normalizes string | string[] | undefined", () => { + assert.deepEqual(normalizeHarnessList(undefined), []); + assert.deepEqual(normalizeHarnessList("codex"), ["codex"]); + assert.deepEqual(normalizeHarnessList(["claude", "codex"]), [ + "claude-code", + "codex", + ]); + assert.deepEqual(normalizeHarnessList(["", " "]), []); + // Idempotent — already-canonical input is unchanged (the parse-once guarantee). + assert.deepEqual(normalizeHarnessList(["claude-code", "codex"]), [ + "claude-code", + "codex", + ]); +}); + +test("adapterForInstructionFile maps a target filename to its harness", () => { + assert.equal(adapterForInstructionFile("CLAUDE.md")?.name, "claude-code"); + assert.equal(adapterForInstructionFile("AGENTS.md")?.name, "codex"); + assert.equal(adapterForInstructionFile("DOCS.md"), undefined); // not an instruction file +}); + +test("resolveHarnessSelection: --harness flag wins over config, kind=ok", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ + root: dir, + flag: "codex", + configHarness: "claude-code", + }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "codex"); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: flag alias (claude) resolves to claude-code", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ root: dir, flag: "claude" }); + assert.equal(sel.adapter.name, "claude-code"); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: unknown flag throws (no silent fallback)", () => { + const dir = makeTmpDir(); + try { + assert.throws( + () => resolveHarnessSelection({ root: dir, flag: "nope" }), + /Unknown harness/, + ); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: single config harness used, kind=ok (alias ok)", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ root: dir, configHarness: "claude" }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "claude-code"); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: single-element array behaves like a string", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ + root: dir, + configHarness: ["codex"], + }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "codex"); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: multiple config harnesses → first + loud notice", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ + root: dir, + configHarness: ["claude-code", "codex"], + }); + assert.equal(sel.adapter.name, "claude-code"); + const notice = noticeOf(sel); + assert.match(notice, /claude-code, codex/); + assert.match(notice, /--harness=/); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: empty array falls through to auto-detect", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ root: dir, configHarness: [] }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "claude-code"); // empty-repo default + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: no config → auto-detect, kind=ok on an empty repo", () => { + const dir = makeTmpDir(); + try { + const sel = resolveHarnessSelection({ root: dir }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "claude-code"); // backwards-compatible default + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: auto-detect a Codex-only repo (AGENTS.md)", () => { + const dir = makeTmpDir(); + try { + writeFileSync(join(dir, "AGENTS.md"), "# x\n"); + const sel = resolveHarnessSelection({ root: dir }); + assert.equal(sel.kind, "ok"); + assert.equal(sel.adapter.name, "codex"); + } finally { + cleanupTmpDir(dir); + } +}); + +test("resolveHarnessSelection: ambiguous repo (CLAUDE.md + AGENTS.md) warns", () => { + const dir = makeTmpDir(); + try { + writeFileSync(join(dir, "CLAUDE.md"), "# x\n"); + writeFileSync(join(dir, "AGENTS.md"), "# x\n"); + const sel = resolveHarnessSelection({ root: dir }); + const notice = noticeOf(sel); + assert.match(notice, /matches/); + assert.match(notice, /harness/); + } finally { + cleanupTmpDir(dir); + } +}); diff --git a/src/adapter-registry.ts b/src/adapter-registry.ts index c91fb43a..654ff3fa 100644 --- a/src/adapter-registry.ts +++ b/src/adapter-registry.ts @@ -58,9 +58,37 @@ export function detectAdapter(root: string): HarnessAdapter { return detectAdapterResult(root).adapter; } -/** Look up a registered adapter by `name` (e.g. for a `--harness` override). */ +/** + * Short-name aliases accepted anywhere a harness name is supplied (config, + * `--harness=`). `init` historically uses `"claude"`; the canonical adapter name + * is `"claude-code"`. Normalizing here keeps selection and the registry in sync. + */ +const HARNESS_ALIASES: Readonly> = { + claude: "claude-code", +}; + +/** Lower-case, trim, and map a short alias to its canonical adapter name. */ +export function normalizeHarnessName(name: string): string { + const n = name.trim().toLowerCase(); + return HARNESS_ALIASES[n] ?? n; +} + +/** Look up a registered adapter by `name` (alias-aware, e.g. `claude`). */ export function getAdapter(name: string): HarnessAdapter | undefined { - return ADAPTERS.find((a) => a.name === name); + const canonical = normalizeHarnessName(name); + return ADAPTERS.find((a) => a.name === canonical); +} + +/** + * The adapter whose instruction file is `filename` (e.g. `AGENTS.md` → codex, + * `CLAUDE.md` → claude-code), if any. The per-spec disambiguation signal: a + * `.spec.ts` compiles a `` instruction file, so the filename names + * the harness more specifically than config/detect for THAT spec. + */ +export function adapterForInstructionFile( + filename: string, +): HarnessAdapter | undefined { + return ADAPTERS.find((a) => a.layout.instructionFile === filename); } /** @@ -69,7 +97,7 @@ export function getAdapter(name: string): HarnessAdapter | undefined { * uses so detection + override live in one place. */ export function resolveAdapter(root: string, harness?: string): HarnessAdapter { - if (harness !== undefined) { + if (harness !== undefined && harness !== "") { const a = getAdapter(harness); if (!a) { const known = ADAPTERS.map((x) => x.name).join(", "); @@ -79,3 +107,72 @@ export function resolveAdapter(root: string, harness?: string): HarnessAdapter { } return detectAdapter(root); } + +/** Normalize a config `harness` value (string | string[]) to a canonical list. */ +export function normalizeHarnessList( + harness?: string | readonly string[], +): string[] { + if (harness === undefined) return []; + const arr = Array.isArray(harness) ? harness : [harness as string]; + return arr.map(normalizeHarnessName).filter(Boolean); +} + +/** + * The adapter chosen for a single-dialect operation. A discriminated union so an + * invalid state — a "notice" with no message, or a clean pick carrying a stray + * string — is unrepresentable: `kind: "ok"` has no `notice`, `kind: "notice"` + * always carries a non-empty one. Both variants carry the `adapter`. + */ +export type HarnessSelection = + | { readonly kind: "ok"; readonly adapter: HarnessAdapter } + | { + readonly kind: "notice"; + readonly adapter: HarnessAdapter; + readonly notice: string; + }; + +/** + * Resolve the single harness a compile/lint operation should use, with explicit + * precedence — the deterministic replacement for sniffing the cwd: + * + * 1. `--harness=` flag (wins; throws if unknown). + * 2. config `harness` resolving to a single entry → use it. + * 3. config `harness` with multiple entries → use the first, with a loud notice. + * 4. no config → auto-detect, with a loud notice when the repo is ambiguous. + * + * `configHarness` is parsed once (alias-normalized) at the call site and passed + * in; this function re-normalizes idempotently so it's safe either way. Pure + * (besides reading `root`'s layout for detection) so the precedence is + * unit-testable without a real compile. See research/multi-harness-compile.md. + */ +export function resolveHarnessSelection(opts: { + root: string; + flag?: string; + configHarness?: string | readonly string[]; +}): HarnessSelection { + const { root, flag, configHarness } = opts; + if (flag !== undefined && flag !== "") { + return { kind: "ok", adapter: resolveAdapter(root, flag) }; + } + const list = normalizeHarnessList(configHarness); + if (list.length === 1) { + return { kind: "ok", adapter: resolveAdapter(root, list[0]) }; + } + if (list.length > 1) { + const adapter = resolveAdapter(root, list[0]); + return { + kind: "notice", + adapter, + notice: `repo targets ${list.join(", ")} — compiling for ${adapter.name}; override with --harness=`, + }; + } + const det = detectAdapterResult(root); + if (det.ambiguousWith.length > 0) { + return { + kind: "notice", + adapter: det.adapter, + notice: `repo matches ${[det.adapter.name, ...det.ambiguousWith].join(", ")} — set "harness" in .vigilesrc.json or use --harness=`, + }; + } + return { kind: "ok", adapter: det.adapter }; +} diff --git a/src/cli.test.ts b/src/cli.test.ts index b2da6383..21b2c6f1 100644 --- a/src/cli.test.ts +++ b/src/cli.test.ts @@ -902,6 +902,247 @@ export default claude({ }); }); +// --------------------------------------------------------------------------- +// Multi-harness: config `harness` selection + the copy-mirror +// --------------------------------------------------------------------------- + +describe("CLI: multi-harness compile", () => { + let tmpDir: string; + + before(() => { + tmpDir = mkdtempSync(join(tmpdir(), "vigiles-cli-harness-")); + writeFileSync( + join(tmpDir, "CLAUDE.md.spec.ts"), + `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default claude({ + target: "CLAUDE.md", + rules: { "test-rule": guidance("Test guidance.") }, +}); +`, + ); + }); + + after(() => { + rmSync(tmpDir, { recursive: true, force: true }); + }); + + it("mirrors CLAUDE.md → AGENTS.md byte-identically when ≥2 harnesses are declared", () => { + writeFileSync( + join(tmpDir, ".vigilesrc.json"), + JSON.stringify({ harness: ["claude-code", "codex"] }, null, 2) + "\n", + ); + const { stdout, exitCode } = run("compile CLAUDE.md.spec.ts", tmpDir); + assert.equal(exitCode, 0); + assert.ok(stdout.includes("mirrored"), "should report the mirror write"); + assert.ok(existsSync(join(tmpDir, "AGENTS.md"))); + // Byte-identical — a copy, carrying the source's embedded integrity hash. + assert.equal( + readFileSync(join(tmpDir, "AGENTS.md"), "utf-8"), + readFileSync(join(tmpDir, "CLAUDE.md"), "utf-8"), + ); + }); + + it("does NOT mirror for a single declared harness", () => { + rmSync(join(tmpDir, "AGENTS.md"), { force: true }); + writeFileSync( + join(tmpDir, ".vigilesrc.json"), + JSON.stringify({ harness: "claude-code" }, null, 2) + "\n", + ); + const { exitCode } = run("compile CLAUDE.md.spec.ts", tmpDir); + assert.equal(exitCode, 0); + assert.ok( + !existsSync(join(tmpDir, "AGENTS.md")), + "no mirror for one harness", + ); + }); + + it("--harness=codex selects the minimal SKILL.md profile (CC-only keys dropped)", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-skill-")); + try { + writeFileSync( + join(dir, "SKILL.md.spec.ts"), + `import { skill, instructions } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default skill({ + name: "demo", + description: "A demo skill", + disableModelInvocation: true, + argumentHint: "", + body: instructions\`Do the thing.\`, +}); +`, + ); + // Codex → minimal frontmatter: the CC-only keys are omitted. + run("compile --harness=codex SKILL.md.spec.ts", dir); + const codex = readFileSync(join(dir, "SKILL.md"), "utf-8"); + assert.ok( + !codex.includes("disable-model-invocation"), + "codex: no CC key", + ); + assert.ok(!codex.includes("argument-hint"), "codex: no CC key"); + // Claude Code → full frontmatter: the same spec keeps the CC-only keys. + run("compile --harness=claude-code SKILL.md.spec.ts", dir); + const cc = readFileSync(join(dir, "SKILL.md"), "utf-8"); + assert.ok( + cc.includes("disable-model-invocation: true"), + "cc: CC key kept", + ); + assert.ok(cc.includes("argument-hint: "), "cc: CC key kept"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("mirror never clobbers a target that owns its own spec", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-twospecs-")); + try { + const specImport = `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}";`; + writeFileSync( + join(dir, "CLAUDE.md.spec.ts"), + `${specImport}\nexport default claude({ target: "CLAUDE.md", rules: { "r": guidance("c") } });\n`, + ); + writeFileSync( + join(dir, "AGENTS.md.spec.ts"), + `${specImport}\nexport default claude({ target: "AGENTS.md", rules: { "r": guidance("a") } });\n`, + ); + writeFileSync( + join(dir, ".vigilesrc.json"), + JSON.stringify({ harness: ["claude-code", "codex"] }, null, 2) + "\n", + ); + const { stdout } = run("compile", dir); + // Each file is its OWN compiled output — the mirror skipped the spec-owned + // target rather than overwriting AGENTS.md with a copy of CLAUDE.md. + assert.ok( + readFileSync(join(dir, "AGENTS.md"), "utf-8").includes("# AGENTS.md"), + "AGENTS.md kept its own compiled content", + ); + assert.ok( + readFileSync(join(dir, "CLAUDE.md"), "utf-8").includes("# CLAUDE.md"), + ); + assert.ok(!stdout.includes("mirrored"), "no mirror when both own specs"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("mirror defers to a sync tool (.ruler present → no copy written)", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-ruler-")); + try { + writeFileSync( + join(dir, "CLAUDE.md.spec.ts"), + `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default claude({ target: "CLAUDE.md", rules: { "r": guidance("c") } }); +`, + ); + writeFileSync( + join(dir, ".vigilesrc.json"), + JSON.stringify({ harness: ["claude-code", "codex"] }, null, 2) + "\n", + ); + mkdirSync(join(dir, ".ruler")); // Ruler owns fan-out + const { stdout } = run("compile CLAUDE.md.spec.ts", dir); + assert.ok( + !existsSync(join(dir, "AGENTS.md")), + "a sync tool owns fan-out — vigiles must not also write the mirror", + ); + assert.ok(!stdout.includes("mirrored")); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("warns when a declared minimal-profile harness drops a skill's CC-only keys", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-skillwarn-")); + try { + writeFileSync( + join(dir, "SKILL.md.spec.ts"), + `import { skill, instructions } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default skill({ + name: "demo", + description: "A demo skill", + disableModelInvocation: true, + body: instructions\`Do the thing.\`, +}); +`, + ); + writeFileSync( + join(dir, ".vigilesrc.json"), + JSON.stringify({ harness: ["claude-code", "codex"] }, null, 2) + "\n", + ); + const { stdout, exitCode } = run("compile SKILL.md.spec.ts", dir); + assert.equal(exitCode, 0); + assert.match(stdout, /disable-model-invocation/); + assert.match(stdout, /codex/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("spec-target disambiguation: a target-less AGENTS.md.spec.ts compiles as codex", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-spectarget-")); + try { + // No `target` field and no config/flag → the spec's filename (AGENTS.md) + // selects the codex dialect, whose instructionTargets[0] becomes the + // heading. Before the fix this used the hard-coded claude-code dialect. + writeFileSync( + join(dir, "AGENTS.md.spec.ts"), + `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default claude({ rules: { r: guidance("a") } }); +`, + ); + const { exitCode } = run("compile AGENTS.md.spec.ts", dir); + assert.equal(exitCode, 0); + const md = readFileSync(join(dir, "AGENTS.md"), "utf-8"); + assert.match(md, /^# AGENTS\.md/m, "codex dialect → AGENTS.md heading"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("mirror is idempotent — an already-identical target isn't rewritten", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-idem-")); + try { + writeFileSync( + join(dir, "CLAUDE.md.spec.ts"), + `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default claude({ target: "CLAUDE.md", rules: { r: guidance("c") } }); +`, + ); + writeFileSync( + join(dir, ".vigilesrc.json"), + JSON.stringify({ harness: ["claude-code", "codex"] }, null, 2) + "\n", + ); + run("compile CLAUDE.md.spec.ts", dir); // first run writes the mirror + const second = run("compile CLAUDE.md.spec.ts", dir); // already identical + assert.equal(second.exitCode, 0); + assert.ok( + !second.stdout.includes("mirrored"), + "no re-mirror when the target is already byte-identical", + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("compile --harness=bogus fails with an actionable error", () => { + const dir = mkdtempSync(join(tmpdir(), "vigiles-harness-bogus-")); + try { + writeFileSync( + join(dir, "CLAUDE.md.spec.ts"), + `import { claude, guidance } from "${resolve(process.cwd(), "src/core/spec.js")}"; +export default claude({ target: "CLAUDE.md", rules: { r: guidance("c") } }); +`, + ); + const { stdout, stderr, exitCode } = run( + "compile --harness=bogus CLAUDE.md.spec.ts", + dir, + ); + assert.notEqual(exitCode, 0); + assert.match(stdout + stderr, /Unknown harness/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + // --------------------------------------------------------------------------- // vigiles strengthen // --------------------------------------------------------------------------- @@ -1011,6 +1252,11 @@ describe("CLI: vigiles init — both pillars + workflow", () => { const yaml = readFileSync(wf, "utf-8"); assert.match(yaml, /uses: zernie\/vigiles@v1/); assert.match(yaml, /npx vigiles test/); // the harness job + // A greenfield repo (no AGENTS.md) records the default harness. + const cfg = JSON.parse( + readFileSync(join(dir, ".vigilesrc.json"), "utf-8"), + ) as { harness?: unknown }; + assert.equal(cfg.harness, "claude-code"); } finally { rmSync(dir, { recursive: true, force: true }); } @@ -1274,6 +1520,12 @@ describe("CLI: installation smoke test (deterministic)", () => { ); const md = readFileSync(join(dir, "AGENTS.md"), "utf-8"); assert.ok(md.includes("Keep this prose."), "AGENTS.md preserved"); + // The harness is recorded in project config so compile/lint select it + // deterministically instead of sniffing the cwd. + const cfg = JSON.parse( + readFileSync(join(dir, ".vigilesrc.json"), "utf-8"), + ) as { harness?: unknown }; + assert.equal(cfg.harness, "codex"); assertNoVendoring(dir); } finally { rmSync(dir, { recursive: true, force: true }); @@ -1294,6 +1546,11 @@ describe("CLI: installation smoke test (deterministic)", () => { assert.equal(exitCode, 0, stdout); assert.ok(existsSync(join(dir, "CLAUDE.md.spec.ts")), "CLAUDE spec"); assert.ok(existsSync(join(dir, "AGENTS.md.spec.ts")), "AGENTS spec"); + // Both declared harnesses recorded as the supported set (canonical names). + const cfg = JSON.parse( + readFileSync(join(dir, ".vigilesrc.json"), "utf-8"), + ) as { harness?: unknown }; + assert.deepEqual(cfg.harness, ["claude-code", "codex"]); assertNoVendoring(dir); } finally { rmSync(dir, { recursive: true, force: true }); diff --git a/src/cli.ts b/src/cli.ts index 6eb1edd9..ffdb72b5 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -27,6 +27,7 @@ import { shouldPrompt, resolvePlan, planPluginInstall, + mergeProjectConfig, type SetupPlan, type SetupAnswers, type ParsedSetupArgs, @@ -40,10 +41,16 @@ import { ruleSeverity, ruleOptions } from "./core/types.js"; import { findUntestedSurfaces, formatUntestedReport } from "./test-coverage.js"; import { scanPlugin, formatScanReport } from "./scan.js"; import { - detectAdapter, detectAdapterResult, resolveAdapter, + resolveHarnessSelection, + normalizeHarnessName, + normalizeHarnessList, + getAdapter, + adapterForInstructionFile, } from "./adapter-registry.js"; +import type { HarnessDialect } from "./core/dialect.js"; +import { skillFrontmatterDropWarnings } from "./skill-harness.js"; import { rankPlugins, formatLeaderboard } from "./leaderboard.js"; import { @@ -58,12 +65,15 @@ import { } from "./core/compile.js"; import type { CompileError } from "./core/compile.js"; import type { ClaudeSpec, SkillSpec, AgentSpec, Railway } from "./core/spec.js"; -import { claudeCodeDialect } from "./adapters/claude-code/dialect.js"; import { findSimilarRules } from "./core/proofs.js"; import { parseInlineRules } from "./core/inline.js"; import { parseFrontmatterRules } from "./core/frontmatter.js"; import { generateSchema } from "./core/generate-schema.js"; -import { detectInstructionMirror, composeCollisions } from "./core/compose.js"; +import { + detectInstructionMirror, + composeCollisions, + detectSyncTools, +} from "./core/compose.js"; import type { InstructionMirror } from "./core/compose.js"; import { compileGeneratorSkill } from "./core/compile-generator.js"; import { evaluateAction, loadActionGates } from "./action-gate.js"; @@ -226,12 +236,13 @@ function compileClaudeToFile( spec: ClaudeSpec, specPath: string, config: VigilesConfig, + dialect: HarnessDialect, ): boolean { const basePath = process.cwd(); const { markdown, errors, linterResults, targets } = compileClaude(spec, { basePath, specFile: specPath, - dialect: claudeCodeDialect, + dialect, maxRules: config.maxRules, maxTokens: config.maxTokens, maxSectionLines: config.maxSectionLines, @@ -264,15 +275,56 @@ function compileClaudeToFile( return true; } +/** + * Branch 3 of the mirror story (research/multi-harness-compile.md): when a repo + * declares ≥2 harnesses and nothing else fans out the instruction file, write a + * byte-identical copy to each other harness's instruction file (e.g. CLAUDE.md → + * AGENTS.md). A copy — not a symlink — because it works everywhere and carries + * the source's embedded integrity hash by construction, so a hand-edit of the + * mirror trips the existing `integrity` check. Never fights a sync tool or + * clobbers a target that owns its own spec. + */ +function writeInstructionMirrors( + primaryOutput: string, + harnesses: string[], +): void { + if (harnesses.length < 2) return; + const cwd = process.cwd(); + // A sync tool (Ruler/rulesync) owns fan-out — don't fight it. + if (detectSyncTools(cwd).length > 0) return; + const primaryName = basename(primaryOutput); + const primaryAbs = resolve(cwd, primaryOutput); + if (!existsSync(primaryAbs)) return; + const content = readFileSync(primaryAbs, "utf-8"); + for (const name of harnesses) { + const adapter = getAdapter(name); + if (!adapter) continue; + const target = adapter.layout.instructionFile; + if (target === primaryName) continue; // the file we just compiled + // Never clobber a target that has its own spec (a genuinely separate file). + if (existsSync(resolve(cwd, `${target}.spec.ts`))) continue; + const targetAbs = resolve(cwd, target); + if (existsSync(targetAbs) && readFileSync(targetAbs, "utf-8") === content) { + continue; // already byte-identical + } + writeFileSync(targetAbs, content); + console.log(` ↳ mirrored ${primaryName} → ${target} (byte-identical)`); + } +} + /** Compile a declarative SkillSpec → SKILL.md. */ -function compileSkillToFile(spec: SkillSpec, specPath: string): boolean { +function compileSkillToFile( + spec: SkillSpec, + specPath: string, + dialect: HarnessDialect, +): boolean { const outputPath = specPath.replace(/\.spec\.ts$/, ""); const { markdown, errors } = compileSkill(spec, { basePath: process.cwd(), specFile: specPath, - // Pick the SKILL.md frontmatter profile from the detected harness — a Codex + // The SKILL.md frontmatter profile comes from the resolved harness — a Codex // repo gets a minimal (name + description) SKILL.md; CC gets the full set. - dialect: detectAdapter(process.cwd()).dialect, + dialect, }); writeFileSync(resolve(process.cwd(), outputPath), markdown); if (errors.length === 0) { @@ -285,12 +337,16 @@ function compileSkillToFile(spec: SkillSpec, specPath: string): boolean { } /** Compile a subagent spec → agents/.md (with its result-contract section). */ -function compileAgentToFile(spec: AgentSpec, specPath: string): boolean { +function compileAgentToFile( + spec: AgentSpec, + specPath: string, + dialect: HarnessDialect, +): boolean { const outputPath = specPath.replace(/\.spec\.ts$/, ""); const { markdown, errors } = compileAgent(spec, { basePath: process.cwd(), specFile: specPath, - dialect: detectAdapter(process.cwd()).dialect, + dialect, }); writeFileSync(resolve(process.cwd(), outputPath), markdown); if (errors.length === 0) { @@ -340,8 +396,21 @@ async function collectAgentNames(): Promise { async function compile( specPaths: string[], config: VigilesConfig, + opts: { harnessFlag?: string } = {}, ): Promise { let allValid = true; + // Parse the declared harness set ONCE (alias-normalized) and feed both the + // dialect pick and the mirror from it — no re-parsing, no cwd-sniffing in the + // helpers. A loud notice (never a silent guess) on a multi-harness or + // ambiguous-detection pick. + const declaredHarnesses = normalizeHarnessList(config.harness); + const selection = resolveHarnessSelection({ + root: process.cwd(), + flag: opts.harnessFlag, + configHarness: declaredHarnesses, + }); + if (selection.kind === "notice") console.log(`⚠ ${selection.notice}`); + const dialect = selection.adapter.dialect; // Resolved lazily on the first railway spec — every delegate() target is // checked against the agents defined anywhere in the project. let knownAgents: string[] | null = null; @@ -362,11 +431,37 @@ async function compile( continue; } if (spec._specType === "claude") { - if (!compileClaudeToFile(spec, specPath, config)) allValid = false; + // Spec-target disambiguation: a CLAUDE.md.spec.ts is a claude-code file, an + // AGENTS.md.spec.ts a codex one — the strongest dialect signal for THIS + // spec. The flag still overrides; absent one, the spec's own target wins + // over config/detect. (Skill/agent targets don't name a harness, so they + // keep the run-level dialect.) + const targetFile = basename(specPath).replace(/\.spec\.ts$/, ""); + const specDialect = + opts.harnessFlag === undefined + ? (adapterForInstructionFile(targetFile)?.dialect ?? dialect) + : dialect; + if (compileClaudeToFile(spec, specPath, config, specDialect)) { + writeInstructionMirrors( + specPath.replace(/\.spec\.ts$/, ""), + declaredHarnesses, + ); + } else { + allValid = false; + } } else if (spec._specType === "skill") { - if (!compileSkillToFile(spec, specPath)) allValid = false; + // Cross-harness verify: flag CC-only frontmatter a declared minimal-profile + // harness (Codex/OpenCode) would silently drop. + const forHarnesses = + declaredHarnesses.length > 0 + ? declaredHarnesses + : [selection.adapter.name]; + for (const w of skillFrontmatterDropWarnings(spec, forHarnesses)) { + console.log(`⚠ ${w}`); + } + if (!compileSkillToFile(spec, specPath, dialect)) allValid = false; } else if (spec._specType === "agent") { - if (!compileAgentToFile(spec, specPath)) allValid = false; + if (!compileAgentToFile(spec, specPath, dialect)) allValid = false; } else if (spec._specType === "railway") { knownAgents ??= await collectAgentNames(); if (!compileRailwayToFile(spec, specPath, knownAgents)) allValid = false; @@ -2199,26 +2294,54 @@ async function setup(args: string[]): Promise { console.log(" npm install -D rule-porter"); } - // Strict config. - if (strict) { - const configPath = resolve(process.cwd(), ".vigilesrc.json"); - if (!existsSync(configPath)) { - writeFileSync( - configPath, - JSON.stringify( - { rules: { "require-spec": "error", "require-skill-spec": "error" } }, - null, - 2, - ) + "\n", - ); - console.log("✓ Created .vigilesrc.json with strict rules"); - written.push(".vigilesrc.json"); - } - } + // Project config — record the harness(es) so compile/lint select the dialect + // deterministically (no cwd sniffing), plus strict rule severities on --strict. + writeProjectConfig({ harnesses, strict, written }); printSetupSummary({ plan, strict, targets, needsMigration, written }); } +/** Canonical, de-duplicated harness list → a config value (string when one). */ +function harnessConfigValue(harnesses: string[]): string | string[] { + const canon = [...new Set(harnesses.map(normalizeHarnessName))]; + return canon.length === 1 ? canon[0] : canon; +} + +/** + * Merge the resolved harness(es) (and strict rule severities) into + * `.vigilesrc.json` without clobbering existing keys — an existing `harness` + * stays, a missing one is added, a malformed file is left untouched. + */ +function writeProjectConfig(opts: { + harnesses: string[]; + strict: boolean; + written: string[]; +}): void { + const configPath = resolve(process.cwd(), ".vigilesrc.json"); + const existed = existsSync(configPath); + let existing: Record = {}; + if (existed) { + try { + existing = JSON.parse(readFileSync(configPath, "utf-8")) as Record< + string, + unknown + >; + } catch { + return; // user-owned malformed config — never clobber it + } + } + const merged = mergeProjectConfig(existing, { + harness: harnessConfigValue(opts.harnesses), + strict: opts.strict, + }); + if (!merged) return; + writeFileSync(configPath, JSON.stringify(merged, null, 2) + "\n"); + console.log(`✓ ${existed ? "Updated" : "Created"} .vigilesrc.json`); + if (!opts.written.includes(".vigilesrc.json")) { + opts.written.push(".vigilesrc.json"); + } +} + // --------------------------------------------------------------------------- // Strengthen: guidance() → enforce() suggestions // --------------------------------------------------------------------------- @@ -2963,7 +3086,10 @@ async function main(): Promise { console.log("Run `vigiles init` to create one."); process.exit(0); } - const valid = await compile(specs, config); + const harnessFlag = args + .find((a) => a.startsWith("--harness=")) + ?.slice("--harness=".length); + const valid = await compile(specs, config, { harnessFlag }); console.log(""); if (valid) { console.log("Compilation complete."); diff --git a/src/core/types.ts b/src/core/types.ts index 62fea39d..dc8e0881 100644 --- a/src/core/types.ts +++ b/src/core/types.ts @@ -160,6 +160,16 @@ export interface VigilesConfig { linters?: Record; /** Orphan-docs check configuration. Include/exclude globs, tsconfig-style. */ orphans?: OrphansConfig; + /** + * The harness(es) this repo targets — selects the compile dialect / skill + * frontmatter profile / instruction-file shape, instead of sniffing the cwd. + * A single name (`"codex"`) for the common single-harness repo, or an array + * (`["claude-code", "codex"]`) declaring the supported set. Written by + * `vigiles init`. Omitted → the CLI auto-detects (backwards-compatible). + * Canonical adapter names; `"claude"` is accepted as an alias for + * `"claude-code"`. See research/multi-harness-compile.md. + */ + harness?: string | string[]; } /** Valid marker types for rule detection. */ diff --git a/src/setup-plan.test.ts b/src/setup-plan.test.ts index aac3f115..4f48c4e5 100644 --- a/src/setup-plan.test.ts +++ b/src/setup-plan.test.ts @@ -8,6 +8,7 @@ import { shouldPrompt, resolvePlan, planPluginInstall, + mergeProjectConfig, } from "./setup-plan.js"; test("defaults: both pillars, CI, plugin, non-strict", () => { @@ -157,3 +158,75 @@ test("shouldPrompt: only a TTY human with unpinned choices", () => { false, ); }); + +// --- mergeProjectConfig: what `vigiles init` writes to .vigilesrc.json --- + +test("mergeProjectConfig: empty config gets the harness", () => { + assert.deepEqual( + mergeProjectConfig({}, { harness: "claude-code", strict: false }), + { + harness: "claude-code", + }, + ); +}); + +test("mergeProjectConfig: array harness is recorded as-is", () => { + assert.deepEqual( + mergeProjectConfig( + {}, + { harness: ["claude-code", "codex"], strict: false }, + ), + { harness: ["claude-code", "codex"] }, + ); +}); + +test("mergeProjectConfig: never clobbers an existing harness (returns null)", () => { + assert.equal( + mergeProjectConfig( + { harness: "codex" }, + { harness: "claude-code", strict: false }, + ), + null, + ); +}); + +test("mergeProjectConfig: preserves other existing keys while adding harness", () => { + assert.deepEqual( + mergeProjectConfig({ maxRules: 50 }, { harness: "codex", strict: false }), + { maxRules: 50, harness: "codex" }, + ); +}); + +test("mergeProjectConfig: strict adds rule severities alongside harness", () => { + assert.deepEqual( + mergeProjectConfig({}, { harness: "claude-code", strict: true }), + { + harness: "claude-code", + rules: { "require-spec": "error", "require-skill-spec": "error" }, + }, + ); +}); + +test("mergeProjectConfig: strict doesn't overwrite an existing rule severity", () => { + const out = mergeProjectConfig( + { harness: "codex", rules: { "require-spec": "warn" } }, + { harness: "codex", strict: true }, + ); + assert.deepEqual(out, { + harness: "codex", + rules: { "require-spec": "warn", "require-skill-spec": "error" }, + }); +}); + +test("mergeProjectConfig: fully-satisfied config returns null (no write)", () => { + assert.equal( + mergeProjectConfig( + { + harness: "codex", + rules: { "require-spec": "error", "require-skill-spec": "error" }, + }, + { harness: "codex", strict: true }, + ), + null, + ); +}); diff --git a/src/setup-plan.ts b/src/setup-plan.ts index fbb4b76f..574949db 100644 --- a/src/setup-plan.ts +++ b/src/setup-plan.ts @@ -84,6 +84,36 @@ export function defaultPlan(strict = false): SetupPlan { }; } +/** + * Pure config-merge for what `vigiles init` writes to `.vigilesrc.json`: record + * the `harness` if absent, add strict rule severities if `--strict`, NEVER + * clobber an existing key. Returns the merged config, or `null` when nothing + * changed (so the IO layer skips the write). The IO (read/parse/write + the + * malformed-file guard) stays in cli.ts. + */ +export function mergeProjectConfig( + existing: Record, + opts: { harness: string | string[]; strict: boolean }, +): Record | null { + const config = { ...existing }; + let changed = false; + if (config.harness === undefined) { + config.harness = opts.harness; + changed = true; + } + if (opts.strict) { + const rules = { ...(config.rules as Record | undefined) }; + for (const r of ["require-spec", "require-skill-spec"]) { + if (rules[r] === undefined) { + rules[r] = "error"; + changed = true; + } + } + config.rules = rules; + } + return changed ? config : null; +} + /** * Whether to drop into interactive prompts: a human at a TTY who passed neither * `--yes` nor an explicit `--target`, and who hasn't already pinned every choice diff --git a/src/skill-harness.test.ts b/src/skill-harness.test.ts new file mode 100644 index 00000000..5b4141fa --- /dev/null +++ b/src/skill-harness.test.ts @@ -0,0 +1,71 @@ +import { test } from "vitest"; +import assert from "node:assert/strict"; + +import { skill, instructions } from "./core/spec.js"; +import { + claudeOnlyFrontmatterKeys, + skillFrontmatterDropWarnings, +} from "./skill-harness.js"; + +const base = { name: "demo", description: "d", body: instructions`x` }; + +test("claudeOnlyFrontmatterKeys picks up disable-model-invocation + argument-hint", () => { + assert.deepEqual(claudeOnlyFrontmatterKeys(skill({ ...base })), []); + assert.deepEqual( + claudeOnlyFrontmatterKeys(skill({ ...base, disableModelInvocation: true })), + ["disable-model-invocation"], + ); + assert.deepEqual( + claudeOnlyFrontmatterKeys(skill({ ...base, argumentHint: "" })), + ["argument-hint"], + ); + // `inputs` also drive the argument-hint key. + assert.deepEqual( + claudeOnlyFrontmatterKeys( + skill({ ...base, inputs: [{ name: "x", hint: "an x" }] }), + ), + ["argument-hint"], + ); +}); + +test("warns for a declared minimal-profile harness (codex) that drops CC-only keys", () => { + const spec = skill({ ...base, disableModelInvocation: true }); + const warnings = skillFrontmatterDropWarnings(spec, ["claude-code", "codex"]); + assert.equal(warnings.length, 1); + assert.match(warnings[0], /disable-model-invocation/); + assert.match(warnings[0], /codex/); + assert.match(warnings[0], /drops it/); +}); + +test("no warning when the only declared harness keeps the keys (claude-code)", () => { + const spec = skill({ + ...base, + disableModelInvocation: true, + argumentHint: "", + }); + assert.deepEqual(skillFrontmatterDropWarnings(spec, ["claude-code"]), []); +}); + +test("no warning when the skill uses no CC-only keys", () => { + assert.deepEqual( + skillFrontmatterDropWarnings(skill({ ...base }), ["codex"]), + [], + ); +}); + +test("plural phrasing + dedupe across repeated harness names", () => { + const spec = skill({ + ...base, + disableModelInvocation: true, + argumentHint: "", + }); + const warnings = skillFrontmatterDropWarnings(spec, ["codex", "codex"]); + assert.equal(warnings.length, 1, "deduped per harness"); + assert.match(warnings[0], /disable-model-invocation, argument-hint are/); + assert.match(warnings[0], /drops them/); +}); + +test("unknown harness names are ignored, not thrown", () => { + const spec = skill({ ...base, disableModelInvocation: true }); + assert.deepEqual(skillFrontmatterDropWarnings(spec, ["bogus"]), []); +}); diff --git a/src/skill-harness.ts b/src/skill-harness.ts new file mode 100644 index 00000000..68f71836 --- /dev/null +++ b/src/skill-harness.ts @@ -0,0 +1,56 @@ +/** + * Cross-harness skill-frontmatter verification (slice 3 of + * research/multi-harness-compile.md, the *verify* half). + * + * A skill's `SKILL.md` references are harness-agnostic; the one harness-specific + * surface is the frontmatter PROFILE. The `claude-code` profile emits CC-only + * keys (`disable-model-invocation`, `argument-hint`); the `minimal` profile + * (Codex, OpenCode) omits them. So a skill that sets those keys, in a repo that + * also targets a minimal-profile harness, has a silent semantic gap: the + * constraint the author expressed won't take effect there. + * + * This reports that gap. It is ASSUMPTION-FREE — the minimal profile *drops* the + * keys, so the warning states a fact about vigiles's own output, not a guess + * about another tool's parser tolerance. + */ +import type { SkillSpec } from "./core/spec.js"; +import { getAdapter } from "./adapter-registry.js"; + +/** The Claude-Code-only frontmatter keys a skill spec would emit. */ +export function claudeOnlyFrontmatterKeys(spec: SkillSpec): string[] { + const keys: string[] = []; + if (spec.disableModelInvocation !== undefined) { + keys.push("disable-model-invocation"); + } + if (spec.argumentHint || (spec.inputs && spec.inputs.length > 0)) { + keys.push("argument-hint"); + } + return keys; +} + +/** + * Warn for each declared harness whose `minimal` SKILL.md profile would DROP a + * skill's Claude-Code-only frontmatter. Empty when the skill uses no such keys or + * no declared harness is minimal-profile. + */ +export function skillFrontmatterDropWarnings( + spec: SkillSpec, + harnessNames: readonly string[], +): string[] { + const ccKeys = claudeOnlyFrontmatterKeys(spec); + if (ccKeys.length === 0) return []; + const warnings: string[] = []; + const seen = new Set(); + for (const name of harnessNames) { + const adapter = getAdapter(name); + if (!adapter || seen.has(adapter.name)) continue; + seen.add(adapter.name); + if (adapter.dialect.skillFrontmatter === "minimal") { + const one = ccKeys.length === 1; + warnings.push( + `skill "${spec.name}": ${ccKeys.join(", ")} ${one ? "is" : "are"} Claude-Code-only — declared harness "${adapter.name}" drops ${one ? "it" : "them"}.`, + ); + } + } + return warnings; +}