mirror of
https://github.com/zernie/vigiles.git
synced 2026-09-14 20:53:57 +08:00
zernie
ea9a0a693d
* Retire buggy legacy evals; document the faithful reproduce command The bench/evals caveman-claim + ecosystem-pilot evals used the old bare-SKILL.md delivery (dropped in the run cwd, which Claude Code never registers), so the skill never loaded — and their docstrings still broadcast the resulting discredited numbers (output up, costs more, ~1% bill) that contradict the corrected canonical run in bench/ecosystem/FINDINGS.md. - Delete bench/evals/caveman-claim.eval.mjs and ecosystem-pilot.eval.mjs. They are fully superseded by bench/ecosystem/benchmark.mjs, which installs skills the faithful way (--plugin-dir + activation hook) over the same corpus. - Repoint every reference (corpus + CLAUDE.md.spec.ts + benchmark header + headroom-pilot header + research/roadmap + research/benchmark-methodology) to the faithful benchmark / FINDINGS.md, and refresh roadmap's stale numbers to the corrected finding (~6% output cut, pooled bill flat, output ~20% of the dollar bill). - Regenerate CLAUDE.md from the spec (fresh integrity hash). - Add bench/ecosystem/README.md documenting the reproduce commands (cheap pilot + the published 7x5 Sonnet run), named explicitly so there is no accidental full-manifest fan-out. Verified: npm run build, vigiles self-lint (integrity + refs, exit 0), prettier --check. * Remove remaining buggy-era pilots: headroom-pilot eval + launch-article draft - Delete bench/evals/headroom-pilot.eval.mjs — same June-2026 bare-SKILL.md delivery bug as the retired caveman/ecosystem pilots (skill never registers). Not superseded content-wise (it measured correctness-lift, a distinct claim), but the delivery is broken; rebuild faithfully if headroom becomes a priority. - Delete research/launch-article-draft.md — a publicly-visible early draft of the now-published, corrected article, still carrying the discredited pre-correction numbers. Remove its index entry (research/README.md) and Key-File entry (research/CLAUDE.md.spec.ts) + recompile research/CLAUDE.md. Verified: vigiles self-lint (integrity + refs) exit 0, prettier --check. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012zGXEtUA57J6FwW9VzMWyz --------- Co-authored-by: Claude <noreply@anthropic.com>