2026-07-14 22:01:50 -04:00
|
|
|
<!-- generated from skills/*/SKILL.md metadata -->
|
|
|
|
|
|
|
|
|
|
# AgentOps skill graph
|
|
|
|
|
|
|
|
|
|
```mermaid
|
|
|
|
|
graph LR
|
|
|
|
|
account_rotation["account-rotation"]
|
|
|
|
|
agent_mail["agent-mail"]
|
|
|
|
|
agent_native["agent-native"]
|
|
|
|
|
agy_native["agy-native"]
|
feat(skills): add anti-ceremony guard to RPI (#1064)
## Summary\n\n- add a clean-room, artifact-free anti-ceremony skill\n-
make RPI invoke its STOP/CONTINUE guard exactly once before Plan\n- STOP
before dispatching any core phase; CONTINUE preserves Plan → Implement →
fresh Validate\n- retain durable intent by reference and snapshot only
non-durable intent\n- update the canonical traversal, workflow,
standards, dependency contracts, tests, and generated projections\n\n##
Verification\n\n- RPI unit tests: 13/13\n- dependency/product-boundary
Bats: 15/15\n- RPI and Anti-Ceremony validators: PASS\n- Cathedral Cut
conformance: PASS\n- mesh, Codex/Gemini projections, parity, hashes,
manifests, and runtime formats: PASS\n- changed-scope
generation/contract checks: PASS\n- full exact-range gate: 68 pass, 0
warn, 0 fail, 0 unknown, 0 skip\n- fresh final semantic review on head
8b4a6a5ef: PASS\n\nThe branch was rebased onto the merged skill-contract
work and regenerated from canonical owners. Unrelated working-copy
changes remain excluded. Hosted Validate is running on the exact
reviewed head.
2026-08-16 16:02:53 -04:00
|
|
|
anti_ceremony["anti-ceremony"]
|
2026-07-14 22:01:50 -04:00
|
|
|
automation_shape_routing["automation-shape-routing"]
|
|
|
|
|
bootstrap["bootstrap"]
|
|
|
|
|
cass["cass"]
|
|
|
|
|
cc_hooks["cc-hooks"]
|
|
|
|
|
codebase_recon["codebase-recon"]
|
|
|
|
|
codex_exec["codex-exec"]
|
|
|
|
|
converter["converter"]
|
|
|
|
|
council["council"]
|
feat: add craft-goal and humanize RPI reports (#988)
## What changed
- add `craft-goal`, a bounded Mayor-style goal compiler/linter for
persistent campaigns over bead-shaped RPI experiments
- add its Codex and Gemini runtime projections plus generated catalogs,
manifests, router, graph, maps, tiers, and registry entries
- split RPI reporting into two explicit surfaces: concise human-readable
interactive output by default, with the full `rpi-report.v1` object
retained as the machine/audit artifact
- add feature and validation assertions that prevent raw schema dumps
from becoming the default interactive response again
## Why
Recent AgentOps history repeatedly showed long-running goal campaigns
confusing the outer goal with one RPI experiment. It also exposed a
presentation bug where the RPI skill instructed agents to paste the
transport-shaped YAML report into chat. This change captures the
corrected goal model and keeps machine evidence separate from the
operator-facing response.
## Validation
- `bash scripts/regen-all.sh --check`
- `bash skills/craft-goal/scripts/validate.sh`
- `bash skills/rpi/scripts/validate.sh`
- `python3 -m unittest skills/rpi/tests/test_run_once.py`
- Codex parity, generated-artifact, and override-coverage checks
- `git diff --check origin/main...HEAD`
- `bash scripts/ci-local-release.sh --fast` ran all branch-relevant
checks successfully; its aggregate exit remained nonzero only for the
pre-existing `skills/using-gc` description-budget violation, which is
unchanged from `origin/main`
2026-07-23 19:24:56 -04:00
|
|
|
craft_goal["craft-goal"]
|
Loop restore: converge and crank as control flow under the verdict contract (ADR-0017) (#1099)
## Loop restore: converge and crank as control flow under the verdict
contract (ADR-0017)
Intent source: `docs/plans/2026-09-03-loop-restore.md` (in this PR).
Decision record:
`docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md`.
**Why.** The 2026-07-14 single-pass cut (`482307762`) removed the
iterate loop (discovery, crank, converge, evolve, the learn write-half)
together with the unproven compounding claim, although ADR-0011 demoted
only the latter. The control flow was never demoted, and its absence
showed on 2026-09-02, when a three-lane fix needed eight validators and
two stops because the contract had no repair phase. This restores the
loop as control flow and nothing else: no knowledge store, no `ao
converge`/`ao crank`, no evolve, no canary. ADR-0004 and ADR-0011 stay
in force.
**What changes.**
- **RPI gains a bounded repair phase.** On `FAIL` or `NOT_PROVEN` with
findings, repair and re-validate freshly under the convergence law:
caller-declared `repair_rounds` (default 2); open finding set keyed by
stable `findings[].id`, union across validator families, non-growing; no
closed id reopens; the subject digest changed or, for `NOT_PROVEN`, new
digest-bound evidence resolved a named gap. Converged = fresh PASS plus
cross-family PASS on risky surfaces. Plan and Implement keep their
single dispatch. `skills/rpi/scripts/run_once.py` models the law as pure
data (33 tests): rounds are validated for shape (digest required, no
duplicate ids, no PASS with findings, no FAIL without findings),
condition 4's evidence branch needs a NOT_PROVEN previous round, a
non-FAIL current round, new evidence, and a resolved finding, and a PASS
over unchanged bytes after a FAIL is a flip that reports NOT_PROVEN.
`workflows/rpi.js` runs validation as legs (spawned or external primary,
plus a caller-supplied `crossFamily.command` on risky surfaces) merged
worst-of with a union of stable ids; a risky surface without a
cross-family leg is `diversity_unsatisfied` and never converges or
enters repair; a failed repair or re-validation returns NOT_PROVEN with
no stale verdict. Validators return `subjectDigest`, stable finding ids,
and `evidenceRefs`.
- **crank returns as a thin wave executor** (113 lines): the caller
selects the wave and the repair bound, crank invokes RPI per lane
(parallel only on disjoint write and regen scopes), runs the wave
acceptance once, returns evidence, and stops. No retry, budget, queue,
claim, lease, Git, closure, or next-work ownership. Routing golden
`rq-07-wave-execution` ranks it first.
- **validate is cross-family by default on risky surfaces**
(`cli/internal/gates/**`, `scripts/check-*.sh`, `tests/**`,
`skills/*/scripts/**`, hook policies, `lib/**`, security-scanned paths)
with the LAW-0 dispatch table: Claude orchestrating uses read-only
`codex exec`; Codex orchestrating uses an interactive Claude session in
an NTM pane, never `claude -p`. No live adapter means
`diversity_unsatisfied`, which on a risky surface is `NOT_PROVEN`. The
full literal CI command set runs once on the final integrated subject;
routine rounds keep the receipt-driven freshness contract.
- **Conformance assertions flipped under ADR-0017 only:**
`scripts/check-cathedral-cut-conformance.py` (crank live; "Stop
regardless" replaced by positive canaries for the law's four conditions;
a bounded `for` loop that compares against `repair_rounds` is required
in `run_repair_phase`, and the gate executes the law's canaries against
the reference behavior), `workflows/rpi.js`,
`skills/rpi/scripts/validate.sh`,
`evals/agentops-core/rpi-behavior.json`,
`skills/rpi/references/rpi.feature`. Every single-pass public surface
(README, AGENTS.md, PRODUCT.md, CI-CD, agent-workflow-reference,
rpi-traversal, cli/README, quickstart and demo commands, the
operating-contract and product-boundary bats, the Codex-description
oracle) now states repair to convergence.
**Known approximation, disclosed.** The Claude conveyor has no
deterministic shell primitive, so changed paths are derived by the fresh
validator (git status and diff against the clean pre-run tree) and
unioned with the implementer's report; risk is classified over that
union and unreported paths are coverage findings. A validator is still a
model; runtime derivation outside every agent is a follow-up. Family
distinctness of the cross-family leg is asserted by the caller's choice
of command and not verified by the script.
**Not in scope.** Premortem stays a single advisory judge and Plan still
only names the first check (phase boundaries unchanged). No `verdict.v2`
or `rpi-report.v1` change. The loop's own effect on outcomes is
unmeasured and owed a seeded-defect probe, like the rest of the corpus.
**Evidence on the tip.** Regen check clean; full gate green with a
HEAD-built binary; CI's bats command green; Go build/vet/test green;
golangci-lint clean; security gate quick PASS; one fresh validator over
the whole diff; one cross-family read of the design before
implementation (13 findings folded) and two of the integrated diff (9
findings in round one, 11 by round two, 15 by round three, each round
repaired and re-reviewed; the fresh validator passed the tip after round
two and the final tip 1e8adb72d passed a fresh validator (14-scenario
independent harness of the law, full gate 71/71 with a HEAD-built
binary, CI bats 1164/0) and a cross-family read by Gemini 3.8 via AGY,
which closed all six remaining residues with no new findings; Codex was
unreachable at push time).
**Follow-ups filed from the final reviews, not blockers:** the JS
violation check tests growth before reopen while Python tests reopen
first (same stop, different label when both occur in one round);
`cli/testdata/compatibility-baseline/families/{demo,quickstart}/case.json`
assert help-text substrings Cobra never prints (pre-existing, no
consumer); runtime derivation of changed paths outside every agent in
the Claude conveyor.
2026-09-03 11:16:08 -04:00
|
|
|
crank["crank"]
|
2026-07-14 22:01:50 -04:00
|
|
|
dcg["dcg"]
|
|
|
|
|
doc["doc"]
|
|
|
|
|
domain["domain"]
|
Structural: goals→fitness rename with compatibility alias; shared retired to a declared contract owner (#1007)
## Summary
The two structural beads deferred from the skill-overhaul reboot waves.
**goals → fitness** (`age-skill-overhaul-reboot-sjv7v.10`, 07-24 plan
D9)
- The semantic skill is now `fitness`; the `ao goals` CLI command family
deliberately keeps its name.
- `skills/goals` remains a thin, non-advertised compatibility alias:
user-invocable, zero capabilities/effects/dependencies, an `alias-of`
context_rel edge to fitness, physical deletion only under the
observed-zero policy.
- `alias-of` added to the frontmatter schema enums (the catalog schema's
own description already listed it as valid relationship vocabulary; the
enum lagged).
- bootstrap's `goals` consumer edge migrated; the stale `goals.feature`
row in the scenario-linkage contract corrected (the feature file was
already absent).
**shared retirement** (`age-skill-overhaul-reboot-sjv7v.11`, 07-24 plan
D9)
- The runtime-neutrality contract moved to its declared owner:
`docs/contracts/runtime-neutrality.md`, scope unchanged (shared
references a consuming skill loads).
- The last consumer edge (`bootstrap consumes: shared`) removed;
`skills/shared` reduced to a non-routable tombstone pending the
observed-zero deletion window; the ports-and-adapters contract updated
to executed state.
## Validation
- Strict frontmatter 50/50 (fitness + alias); regen byte-current;
scenario-linkage PASS; liveness/anti-spiral/desc-budget bats green;
`cli/internal/skills` Go tests pass
- Cross-family review, two rounds: round 1 — one finding fixed
(unintended contract-scope broadening reverted), one rejected with
evidence (the reviewer inferred a pre-existing feature file from a stale
doc row; the pre-change tree had none); round 2 delta **VERDICT: PASS**
Tracker: `age-skill-overhaul-reboot-sjv7v.10`, `.11`
2026-07-29 01:03:12 -04:00
|
|
|
fitness["fitness"]
|
2026-07-14 22:01:50 -04:00
|
|
|
handoff["handoff"]
|
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`):
the round-2 findings were the same failure shape — round-1 repairs
patched cited lines instead of sweeping the class — so this commit
sweeps each file whole: every remaining SATURATED-row-append site in
skill-eval now routes to RUNBOOK retirement, the human-only-skills
*description* is runtime-conditional, premortem's "(MEASURED)" label is
gone, SKILL-API's context table carries all 25 rows and the enforcement
table gains `disable-model-invocation`, and the fixture-identity claim
is stated precisely (probe id, honesty note, and control arm are the
only differing fields — as the acceptance permits). Post-sweep:
validators, full Go suite, 68/68 gates, goldens + headroom bats green,
projections current, gemini in sync. Merging per Bo's instruction.
## What
Train 1 of the accepted [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent
landed at `986a4feaf`): the measurement substrate, the skill-context
diet, and the retrieval-eval contract. Three worktree-isolated lanes,
each independently validated by a fresh context, plus one integration
commit. 103 files, +5,510/−76.
**L1 — measurement substrate** (`instrument/measurement-substrate`)
- Gate `skill.probe-headroom` (advisory, Fast|Full): answers the
question `skill.probe-coverage` cannot — not "does a probe result exist"
but "could one have existed at all". The rule, ported to Go
(`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin
check script — the witness-crosscheck pattern, **no new `ao` root
command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels
⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it;
treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed
fixture pairs read `INERT` to everything else in the repo; the failing
separation test predates the implementation, and a bats negative-control
swaps fixture bytes and asserts the gate flips.
- **First reading on real data: 7 of 11 historical probe groups are
SATURATED** — including both `validate-not-proven` runs. Those INERT
rows were never honest nulls; they were void. The 0/12 ledger number now
argues itself.
- Declared denominator for probe-coverage:
`scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry
without an argument, stale slug, or duplicate ⇒ exit 2). One entry
(`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12
→ 0/12: alias left, `one-way-door` entered) — the gain is a declared
number, not a better-looking one.
- Re-landed from the recovered clean-room commit (`9872483bd`),
re-validated against *current* main: `skill-eval` (defers saturation to
the gate id; its shell scripts dropped, not shipped — ratchet intent),
`route`, `one-way-door`, premortem reversibility check, council
`caller_challenge` (schema + validator, per the agent-core boundary that
the panel may challenge, never overrule).
**L4 — context diet** (`instrument/context-diet`)
- `disable-model-invocation: true` on 4 human-only skills (key verified
verbatim against Anthropic's docs). The plan guessed 35 candidates; the
graph said otherwise — 23 carry `user-invocable: true`, and 19 of those
are excluded on cited evidence (rpi consumes
anti-ceremony/implement/plan/validate; workflow scripts reach others;
`goals` is a live migration tombstone). The exclusion evidence is
retained in the lane report.
- One router skill (`human-only-skills`) — the single always-loaded
description that replaces four; it hints, never fires.
- `.out-of-scope/` formalized with this week's three refusals
(checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B
as the measurement unit), each citing its evidence.
- Deterministic proof, no model eval: before/after bytes of
always-loaded description load reported in the lane summary.
**L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane
verdict PASS 10/10)
- `AGENTS.md` federated row now names **ee (eidetic-engine)** as a
concrete caller-selected memory system — consume, never build; symlink
intact.
- `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens
+ `scripts/check-routing-probe-goldens.sh` graded against `ao skills
find`, wired as an **advisory** nightly job. Zero goldens is a failing
state — no new zero-denominator green.
- **The instrument caught a real miss on day one — and its own
prescription fixed it.** Golden `rq-04` expects `validate` for "judge
whether this finished change is actually proven before I merge it"; at
authoring, `ao skills find` ranked the *forbidden* `premortem` first and
`validate` nowhere in six natural phrasings. The pointer-wording-first
repair (validate's description gained the caller's own words: finished,
proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the
golden pins the repair — a description regression reopens it.
## Integration
`regen-all.sh` once over the merged lanes (catalog 52 → 56, four new
codex twins, mesh, router, manifests); `skills/route/SKILL.md`
catalog/router links became prose repo-root references (the projected
twin cannot resolve `../catalog.json` — this was both the
portable-conformance failure and the sole broken doc link);
`codex-portable-conformance.bats` pin 52 → 56.
## Evidence
- `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 ·
`ao gate check --full` **68/68** · four skill validators PASS ·
probe-headroom / routing-goldens / probe-coverage bats PASS ·
`regen-all.sh --check` all current.
- Per-lane fresh validators re-ran every suite on detached content; L5
PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved
for integration (their remaining acceptance observed green), settled
above. Cross-family (Codex) review of the integrated diff recorded in
the session report.
- Two disclosed scope stretches accepted at integration: a one-line
`.gitignore` entry mirroring the witness-crosscheck precedent, and the
probe LEDGER.md fact-correction L1's own change made necessary (noted
for Train 2's L2, which owns that file next).
## Cross-family review (Codex, fresh context)
Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the
control arm; the goldens grader was red where the plan's acceptance says
green) and eight majors (contract contradictions in the re-landed
skills, a converter-substitution false claim in the codex router twin,
two overreaching `.out-of-scope` entries, stale SKILL-API counts). All
repaired in one bounded round (`db68935a3`): fixtures now byte-identical
outside the control arm, the routing miss actually fixed rather than
tolerated, every cited contradiction reconciled at the source and
re-projected. Post-repair: full Go suite exit 0, `gate check --full`
68/68, all validators and probe/goldens bats green, projections current,
gemini byte-identity restored. Focused re-check verdict recorded in the
session report.
## Follow-ups (Train 2, already planned)
Seeded-defect probes for the judgment spine (every ledger row citing a
passing headroom pre-screen) and the gate-hardening pair
(`Gate-Loosen-Reason` tightening ratchet; mechanical
grounding-validation over evidence docs). Plus, surfaced by this train:
a latent `valid_keys`/schema divergence in `validate-skill-schema.sh`
(two keys the schema defines are absent from the script's allowlist —
pre-existing).
2026-08-26 19:16:51 -04:00
|
|
|
human_only_skills["human-only-skills"]
|
2026-07-14 22:01:50 -04:00
|
|
|
idea_genie["idea-genie"]
|
|
|
|
|
implement["implement"]
|
|
|
|
|
learn["learn"]
|
2026-09-09 10:10:29 -04:00
|
|
|
memory["memory"]
|
2026-07-14 22:01:50 -04:00
|
|
|
ms["ms"]
|
|
|
|
|
ntm["ntm"]
|
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`):
the round-2 findings were the same failure shape — round-1 repairs
patched cited lines instead of sweeping the class — so this commit
sweeps each file whole: every remaining SATURATED-row-append site in
skill-eval now routes to RUNBOOK retirement, the human-only-skills
*description* is runtime-conditional, premortem's "(MEASURED)" label is
gone, SKILL-API's context table carries all 25 rows and the enforcement
table gains `disable-model-invocation`, and the fixture-identity claim
is stated precisely (probe id, honesty note, and control arm are the
only differing fields — as the acceptance permits). Post-sweep:
validators, full Go suite, 68/68 gates, goldens + headroom bats green,
projections current, gemini in sync. Merging per Bo's instruction.
## What
Train 1 of the accepted [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent
landed at `986a4feaf`): the measurement substrate, the skill-context
diet, and the retrieval-eval contract. Three worktree-isolated lanes,
each independently validated by a fresh context, plus one integration
commit. 103 files, +5,510/−76.
**L1 — measurement substrate** (`instrument/measurement-substrate`)
- Gate `skill.probe-headroom` (advisory, Fast|Full): answers the
question `skill.probe-coverage` cannot — not "does a probe result exist"
but "could one have existed at all". The rule, ported to Go
(`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin
check script — the witness-crosscheck pattern, **no new `ao` root
command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels
⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it;
treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed
fixture pairs read `INERT` to everything else in the repo; the failing
separation test predates the implementation, and a bats negative-control
swaps fixture bytes and asserts the gate flips.
- **First reading on real data: 7 of 11 historical probe groups are
SATURATED** — including both `validate-not-proven` runs. Those INERT
rows were never honest nulls; they were void. The 0/12 ledger number now
argues itself.
- Declared denominator for probe-coverage:
`scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry
without an argument, stale slug, or duplicate ⇒ exit 2). One entry
(`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12
→ 0/12: alias left, `one-way-door` entered) — the gain is a declared
number, not a better-looking one.
- Re-landed from the recovered clean-room commit (`9872483bd`),
re-validated against *current* main: `skill-eval` (defers saturation to
the gate id; its shell scripts dropped, not shipped — ratchet intent),
`route`, `one-way-door`, premortem reversibility check, council
`caller_challenge` (schema + validator, per the agent-core boundary that
the panel may challenge, never overrule).
**L4 — context diet** (`instrument/context-diet`)
- `disable-model-invocation: true` on 4 human-only skills (key verified
verbatim against Anthropic's docs). The plan guessed 35 candidates; the
graph said otherwise — 23 carry `user-invocable: true`, and 19 of those
are excluded on cited evidence (rpi consumes
anti-ceremony/implement/plan/validate; workflow scripts reach others;
`goals` is a live migration tombstone). The exclusion evidence is
retained in the lane report.
- One router skill (`human-only-skills`) — the single always-loaded
description that replaces four; it hints, never fires.
- `.out-of-scope/` formalized with this week's three refusals
(checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B
as the measurement unit), each citing its evidence.
- Deterministic proof, no model eval: before/after bytes of
always-loaded description load reported in the lane summary.
**L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane
verdict PASS 10/10)
- `AGENTS.md` federated row now names **ee (eidetic-engine)** as a
concrete caller-selected memory system — consume, never build; symlink
intact.
- `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens
+ `scripts/check-routing-probe-goldens.sh` graded against `ao skills
find`, wired as an **advisory** nightly job. Zero goldens is a failing
state — no new zero-denominator green.
- **The instrument caught a real miss on day one — and its own
prescription fixed it.** Golden `rq-04` expects `validate` for "judge
whether this finished change is actually proven before I merge it"; at
authoring, `ao skills find` ranked the *forbidden* `premortem` first and
`validate` nowhere in six natural phrasings. The pointer-wording-first
repair (validate's description gained the caller's own words: finished,
proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the
golden pins the repair — a description regression reopens it.
## Integration
`regen-all.sh` once over the merged lanes (catalog 52 → 56, four new
codex twins, mesh, router, manifests); `skills/route/SKILL.md`
catalog/router links became prose repo-root references (the projected
twin cannot resolve `../catalog.json` — this was both the
portable-conformance failure and the sole broken doc link);
`codex-portable-conformance.bats` pin 52 → 56.
## Evidence
- `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 ·
`ao gate check --full` **68/68** · four skill validators PASS ·
probe-headroom / routing-goldens / probe-coverage bats PASS ·
`regen-all.sh --check` all current.
- Per-lane fresh validators re-ran every suite on detached content; L5
PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved
for integration (their remaining acceptance observed green), settled
above. Cross-family (Codex) review of the integrated diff recorded in
the session report.
- Two disclosed scope stretches accepted at integration: a one-line
`.gitignore` entry mirroring the witness-crosscheck precedent, and the
probe LEDGER.md fact-correction L1's own change made necessary (noted
for Train 2's L2, which owns that file next).
## Cross-family review (Codex, fresh context)
Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the
control arm; the goldens grader was red where the plan's acceptance says
green) and eight majors (contract contradictions in the re-landed
skills, a converter-substitution false claim in the codex router twin,
two overreaching `.out-of-scope` entries, stale SKILL-API counts). All
repaired in one bounded round (`db68935a3`): fixtures now byte-identical
outside the control arm, the routing miss actually fixed rather than
tolerated, every cited contradiction reconciled at the source and
re-projected. Post-repair: full Go suite exit 0, `gate check --full`
68/68, all validators and probe/goldens bats green, projections current,
gemini byte-identity restored. Focused re-check verdict recorded in the
session report.
## Follow-ups (Train 2, already planned)
Seeded-defect probes for the judgment spine (every ledger row citing a
passing headroom pre-screen) and the gate-hardening pair
(`Gate-Loosen-Reason` tightening ratchet; mechanical
grounding-validation over evidence docs). Plus, surfaced by this train:
a latent `valid_keys`/schema divergence in `validate-skill-schema.sh`
(two keys the schema defines are absent from the script's allowlist —
pre-existing).
2026-08-26 19:16:51 -04:00
|
|
|
one_way_door["one-way-door"]
|
2026-07-14 22:01:50 -04:00
|
|
|
operationalize["operationalize"]
|
|
|
|
|
pattern_mining["pattern-mining"]
|
|
|
|
|
plan["plan"]
|
|
|
|
|
postmortem["postmortem"]
|
|
|
|
|
premortem["premortem"]
|
|
|
|
|
product["product"]
|
|
|
|
|
rch["rch"]
|
|
|
|
|
reality_check["reality-check"]
|
|
|
|
|
refactor["refactor"]
|
|
|
|
|
research["research"]
|
|
|
|
|
reverse_engineer["reverse-engineer"]
|
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`):
the round-2 findings were the same failure shape — round-1 repairs
patched cited lines instead of sweeping the class — so this commit
sweeps each file whole: every remaining SATURATED-row-append site in
skill-eval now routes to RUNBOOK retirement, the human-only-skills
*description* is runtime-conditional, premortem's "(MEASURED)" label is
gone, SKILL-API's context table carries all 25 rows and the enforcement
table gains `disable-model-invocation`, and the fixture-identity claim
is stated precisely (probe id, honesty note, and control arm are the
only differing fields — as the acceptance permits). Post-sweep:
validators, full Go suite, 68/68 gates, goldens + headroom bats green,
projections current, gemini in sync. Merging per Bo's instruction.
## What
Train 1 of the accepted [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent
landed at `986a4feaf`): the measurement substrate, the skill-context
diet, and the retrieval-eval contract. Three worktree-isolated lanes,
each independently validated by a fresh context, plus one integration
commit. 103 files, +5,510/−76.
**L1 — measurement substrate** (`instrument/measurement-substrate`)
- Gate `skill.probe-headroom` (advisory, Fast|Full): answers the
question `skill.probe-coverage` cannot — not "does a probe result exist"
but "could one have existed at all". The rule, ported to Go
(`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin
check script — the witness-crosscheck pattern, **no new `ao` root
command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels
⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it;
treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed
fixture pairs read `INERT` to everything else in the repo; the failing
separation test predates the implementation, and a bats negative-control
swaps fixture bytes and asserts the gate flips.
- **First reading on real data: 7 of 11 historical probe groups are
SATURATED** — including both `validate-not-proven` runs. Those INERT
rows were never honest nulls; they were void. The 0/12 ledger number now
argues itself.
- Declared denominator for probe-coverage:
`scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry
without an argument, stale slug, or duplicate ⇒ exit 2). One entry
(`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12
→ 0/12: alias left, `one-way-door` entered) — the gain is a declared
number, not a better-looking one.
- Re-landed from the recovered clean-room commit (`9872483bd`),
re-validated against *current* main: `skill-eval` (defers saturation to
the gate id; its shell scripts dropped, not shipped — ratchet intent),
`route`, `one-way-door`, premortem reversibility check, council
`caller_challenge` (schema + validator, per the agent-core boundary that
the panel may challenge, never overrule).
**L4 — context diet** (`instrument/context-diet`)
- `disable-model-invocation: true` on 4 human-only skills (key verified
verbatim against Anthropic's docs). The plan guessed 35 candidates; the
graph said otherwise — 23 carry `user-invocable: true`, and 19 of those
are excluded on cited evidence (rpi consumes
anti-ceremony/implement/plan/validate; workflow scripts reach others;
`goals` is a live migration tombstone). The exclusion evidence is
retained in the lane report.
- One router skill (`human-only-skills`) — the single always-loaded
description that replaces four; it hints, never fires.
- `.out-of-scope/` formalized with this week's three refusals
(checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B
as the measurement unit), each citing its evidence.
- Deterministic proof, no model eval: before/after bytes of
always-loaded description load reported in the lane summary.
**L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane
verdict PASS 10/10)
- `AGENTS.md` federated row now names **ee (eidetic-engine)** as a
concrete caller-selected memory system — consume, never build; symlink
intact.
- `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens
+ `scripts/check-routing-probe-goldens.sh` graded against `ao skills
find`, wired as an **advisory** nightly job. Zero goldens is a failing
state — no new zero-denominator green.
- **The instrument caught a real miss on day one — and its own
prescription fixed it.** Golden `rq-04` expects `validate` for "judge
whether this finished change is actually proven before I merge it"; at
authoring, `ao skills find` ranked the *forbidden* `premortem` first and
`validate` nowhere in six natural phrasings. The pointer-wording-first
repair (validate's description gained the caller's own words: finished,
proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the
golden pins the repair — a description regression reopens it.
## Integration
`regen-all.sh` once over the merged lanes (catalog 52 → 56, four new
codex twins, mesh, router, manifests); `skills/route/SKILL.md`
catalog/router links became prose repo-root references (the projected
twin cannot resolve `../catalog.json` — this was both the
portable-conformance failure and the sole broken doc link);
`codex-portable-conformance.bats` pin 52 → 56.
## Evidence
- `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 ·
`ao gate check --full` **68/68** · four skill validators PASS ·
probe-headroom / routing-goldens / probe-coverage bats PASS ·
`regen-all.sh --check` all current.
- Per-lane fresh validators re-ran every suite on detached content; L5
PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved
for integration (their remaining acceptance observed green), settled
above. Cross-family (Codex) review of the integrated diff recorded in
the session report.
- Two disclosed scope stretches accepted at integration: a one-line
`.gitignore` entry mirroring the witness-crosscheck precedent, and the
probe LEDGER.md fact-correction L1's own change made necessary (noted
for Train 2's L2, which owns that file next).
## Cross-family review (Codex, fresh context)
Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the
control arm; the goldens grader was red where the plan's acceptance says
green) and eight majors (contract contradictions in the re-landed
skills, a converter-substitution false claim in the codex router twin,
two overreaching `.out-of-scope` entries, stale SKILL-API counts). All
repaired in one bounded round (`db68935a3`): fixtures now byte-identical
outside the control arm, the routing miss actually fixed rather than
tolerated, every cited contradiction reconciled at the source and
re-projected. Post-repair: full Go suite exit 0, `gate check --full`
68/68, all validators and probe/goldens bats green, projections current,
gemini byte-identity restored. Focused re-check verdict recorded in the
session report.
## Follow-ups (Train 2, already planned)
Seeded-defect probes for the judgment spine (every ledger row citing a
passing headroom pre-screen) and the gate-hardening pair
(`Gate-Loosen-Reason` tightening ratchet; mechanical
grounding-validation over evidence docs). Plus, surfaced by this train:
a latent `valid_keys`/schema divergence in `validate-skill-schema.sh`
(two keys the schema defines are absent from the script's allowlist —
pre-existing).
2026-08-26 19:16:51 -04:00
|
|
|
route["route"]
|
2026-07-14 22:01:50 -04:00
|
|
|
rpi["rpi"]
|
|
|
|
|
sbh["sbh"]
|
|
|
|
|
scaffold["scaffold"]
|
|
|
|
|
security["security"]
|
|
|
|
|
skill_builder["skill-builder"]
|
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`):
the round-2 findings were the same failure shape — round-1 repairs
patched cited lines instead of sweeping the class — so this commit
sweeps each file whole: every remaining SATURATED-row-append site in
skill-eval now routes to RUNBOOK retirement, the human-only-skills
*description* is runtime-conditional, premortem's "(MEASURED)" label is
gone, SKILL-API's context table carries all 25 rows and the enforcement
table gains `disable-model-invocation`, and the fixture-identity claim
is stated precisely (probe id, honesty note, and control arm are the
only differing fields — as the acceptance permits). Post-sweep:
validators, full Go suite, 68/68 gates, goldens + headroom bats green,
projections current, gemini in sync. Merging per Bo's instruction.
## What
Train 1 of the accepted [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent
landed at `986a4feaf`): the measurement substrate, the skill-context
diet, and the retrieval-eval contract. Three worktree-isolated lanes,
each independently validated by a fresh context, plus one integration
commit. 103 files, +5,510/−76.
**L1 — measurement substrate** (`instrument/measurement-substrate`)
- Gate `skill.probe-headroom` (advisory, Fast|Full): answers the
question `skill.probe-coverage` cannot — not "does a probe result exist"
but "could one have existed at all". The rule, ported to Go
(`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin
check script — the witness-crosscheck pattern, **no new `ao` root
command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels
⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it;
treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed
fixture pairs read `INERT` to everything else in the repo; the failing
separation test predates the implementation, and a bats negative-control
swaps fixture bytes and asserts the gate flips.
- **First reading on real data: 7 of 11 historical probe groups are
SATURATED** — including both `validate-not-proven` runs. Those INERT
rows were never honest nulls; they were void. The 0/12 ledger number now
argues itself.
- Declared denominator for probe-coverage:
`scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry
without an argument, stale slug, or duplicate ⇒ exit 2). One entry
(`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12
→ 0/12: alias left, `one-way-door` entered) — the gain is a declared
number, not a better-looking one.
- Re-landed from the recovered clean-room commit (`9872483bd`),
re-validated against *current* main: `skill-eval` (defers saturation to
the gate id; its shell scripts dropped, not shipped — ratchet intent),
`route`, `one-way-door`, premortem reversibility check, council
`caller_challenge` (schema + validator, per the agent-core boundary that
the panel may challenge, never overrule).
**L4 — context diet** (`instrument/context-diet`)
- `disable-model-invocation: true` on 4 human-only skills (key verified
verbatim against Anthropic's docs). The plan guessed 35 candidates; the
graph said otherwise — 23 carry `user-invocable: true`, and 19 of those
are excluded on cited evidence (rpi consumes
anti-ceremony/implement/plan/validate; workflow scripts reach others;
`goals` is a live migration tombstone). The exclusion evidence is
retained in the lane report.
- One router skill (`human-only-skills`) — the single always-loaded
description that replaces four; it hints, never fires.
- `.out-of-scope/` formalized with this week's three refusals
(checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B
as the measurement unit), each citing its evidence.
- Deterministic proof, no model eval: before/after bytes of
always-loaded description load reported in the lane summary.
**L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane
verdict PASS 10/10)
- `AGENTS.md` federated row now names **ee (eidetic-engine)** as a
concrete caller-selected memory system — consume, never build; symlink
intact.
- `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens
+ `scripts/check-routing-probe-goldens.sh` graded against `ao skills
find`, wired as an **advisory** nightly job. Zero goldens is a failing
state — no new zero-denominator green.
- **The instrument caught a real miss on day one — and its own
prescription fixed it.** Golden `rq-04` expects `validate` for "judge
whether this finished change is actually proven before I merge it"; at
authoring, `ao skills find` ranked the *forbidden* `premortem` first and
`validate` nowhere in six natural phrasings. The pointer-wording-first
repair (validate's description gained the caller's own words: finished,
proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the
golden pins the repair — a description regression reopens it.
## Integration
`regen-all.sh` once over the merged lanes (catalog 52 → 56, four new
codex twins, mesh, router, manifests); `skills/route/SKILL.md`
catalog/router links became prose repo-root references (the projected
twin cannot resolve `../catalog.json` — this was both the
portable-conformance failure and the sole broken doc link);
`codex-portable-conformance.bats` pin 52 → 56.
## Evidence
- `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 ·
`ao gate check --full` **68/68** · four skill validators PASS ·
probe-headroom / routing-goldens / probe-coverage bats PASS ·
`regen-all.sh --check` all current.
- Per-lane fresh validators re-ran every suite on detached content; L5
PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved
for integration (their remaining acceptance observed green), settled
above. Cross-family (Codex) review of the integrated diff recorded in
the session report.
- Two disclosed scope stretches accepted at integration: a one-line
`.gitignore` entry mirroring the witness-crosscheck precedent, and the
probe LEDGER.md fact-correction L1's own change made necessary (noted
for Train 2's L2, which owns that file next).
## Cross-family review (Codex, fresh context)
Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the
control arm; the goldens grader was red where the plan's acceptance says
green) and eight majors (contract contradictions in the re-landed
skills, a converter-substitution false claim in the codex router twin,
two overreaching `.out-of-scope` entries, stale SKILL-API counts). All
repaired in one bounded round (`db68935a3`): fixtures now byte-identical
outside the control arm, the routing miss actually fixed rather than
tolerated, every cited contradiction reconciled at the source and
re-projected. Post-repair: full Go suite exit 0, `gate check --full`
68/68, all validators and probe/goldens bats green, projections current,
gemini byte-identity restored. Focused re-check verdict recorded in the
session report.
## Follow-ups (Train 2, already planned)
Seeded-defect probes for the judgment spine (every ledger row citing a
passing headroom pre-screen) and the gate-hardening pair
(`Gate-Loosen-Reason` tightening ratchet; mechanical
grounding-validation over evidence docs). Plus, surfaced by this train:
a latent `valid_keys`/schema divergence in `validate-skill-schema.sh`
(two keys the schema defines are absent from the script's allowlist —
pre-existing).
2026-08-26 19:16:51 -04:00
|
|
|
skill_eval["skill-eval"]
|
2026-07-14 22:01:50 -04:00
|
|
|
standards["standards"]
|
|
|
|
|
status["status"]
|
|
|
|
|
swarm["swarm"]
|
|
|
|
|
test["test"]
|
|
|
|
|
toil_mining["toil-mining"]
|
2026-07-30 10:44:54 -04:00
|
|
|
using_flywheel["using-flywheel"]
|
2026-07-14 22:01:50 -04:00
|
|
|
using_gc["using-gc"]
|
|
|
|
|
validate["validate"]
|
|
|
|
|
workflow_builder["workflow-builder"]
|
Loop restore: converge and crank as control flow under the verdict contract (ADR-0017) (#1099)
## Loop restore: converge and crank as control flow under the verdict
contract (ADR-0017)
Intent source: `docs/plans/2026-09-03-loop-restore.md` (in this PR).
Decision record:
`docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md`.
**Why.** The 2026-07-14 single-pass cut (`482307762`) removed the
iterate loop (discovery, crank, converge, evolve, the learn write-half)
together with the unproven compounding claim, although ADR-0011 demoted
only the latter. The control flow was never demoted, and its absence
showed on 2026-09-02, when a three-lane fix needed eight validators and
two stops because the contract had no repair phase. This restores the
loop as control flow and nothing else: no knowledge store, no `ao
converge`/`ao crank`, no evolve, no canary. ADR-0004 and ADR-0011 stay
in force.
**What changes.**
- **RPI gains a bounded repair phase.** On `FAIL` or `NOT_PROVEN` with
findings, repair and re-validate freshly under the convergence law:
caller-declared `repair_rounds` (default 2); open finding set keyed by
stable `findings[].id`, union across validator families, non-growing; no
closed id reopens; the subject digest changed or, for `NOT_PROVEN`, new
digest-bound evidence resolved a named gap. Converged = fresh PASS plus
cross-family PASS on risky surfaces. Plan and Implement keep their
single dispatch. `skills/rpi/scripts/run_once.py` models the law as pure
data (33 tests): rounds are validated for shape (digest required, no
duplicate ids, no PASS with findings, no FAIL without findings),
condition 4's evidence branch needs a NOT_PROVEN previous round, a
non-FAIL current round, new evidence, and a resolved finding, and a PASS
over unchanged bytes after a FAIL is a flip that reports NOT_PROVEN.
`workflows/rpi.js` runs validation as legs (spawned or external primary,
plus a caller-supplied `crossFamily.command` on risky surfaces) merged
worst-of with a union of stable ids; a risky surface without a
cross-family leg is `diversity_unsatisfied` and never converges or
enters repair; a failed repair or re-validation returns NOT_PROVEN with
no stale verdict. Validators return `subjectDigest`, stable finding ids,
and `evidenceRefs`.
- **crank returns as a thin wave executor** (113 lines): the caller
selects the wave and the repair bound, crank invokes RPI per lane
(parallel only on disjoint write and regen scopes), runs the wave
acceptance once, returns evidence, and stops. No retry, budget, queue,
claim, lease, Git, closure, or next-work ownership. Routing golden
`rq-07-wave-execution` ranks it first.
- **validate is cross-family by default on risky surfaces**
(`cli/internal/gates/**`, `scripts/check-*.sh`, `tests/**`,
`skills/*/scripts/**`, hook policies, `lib/**`, security-scanned paths)
with the LAW-0 dispatch table: Claude orchestrating uses read-only
`codex exec`; Codex orchestrating uses an interactive Claude session in
an NTM pane, never `claude -p`. No live adapter means
`diversity_unsatisfied`, which on a risky surface is `NOT_PROVEN`. The
full literal CI command set runs once on the final integrated subject;
routine rounds keep the receipt-driven freshness contract.
- **Conformance assertions flipped under ADR-0017 only:**
`scripts/check-cathedral-cut-conformance.py` (crank live; "Stop
regardless" replaced by positive canaries for the law's four conditions;
a bounded `for` loop that compares against `repair_rounds` is required
in `run_repair_phase`, and the gate executes the law's canaries against
the reference behavior), `workflows/rpi.js`,
`skills/rpi/scripts/validate.sh`,
`evals/agentops-core/rpi-behavior.json`,
`skills/rpi/references/rpi.feature`. Every single-pass public surface
(README, AGENTS.md, PRODUCT.md, CI-CD, agent-workflow-reference,
rpi-traversal, cli/README, quickstart and demo commands, the
operating-contract and product-boundary bats, the Codex-description
oracle) now states repair to convergence.
**Known approximation, disclosed.** The Claude conveyor has no
deterministic shell primitive, so changed paths are derived by the fresh
validator (git status and diff against the clean pre-run tree) and
unioned with the implementer's report; risk is classified over that
union and unreported paths are coverage findings. A validator is still a
model; runtime derivation outside every agent is a follow-up. Family
distinctness of the cross-family leg is asserted by the caller's choice
of command and not verified by the script.
**Not in scope.** Premortem stays a single advisory judge and Plan still
only names the first check (phase boundaries unchanged). No `verdict.v2`
or `rpi-report.v1` change. The loop's own effect on outcomes is
unmeasured and owed a seeded-defect probe, like the rest of the corpus.
**Evidence on the tip.** Regen check clean; full gate green with a
HEAD-built binary; CI's bats command green; Go build/vet/test green;
golangci-lint clean; security gate quick PASS; one fresh validator over
the whole diff; one cross-family read of the design before
implementation (13 findings folded) and two of the integrated diff (9
findings in round one, 11 by round two, 15 by round three, each round
repaired and re-reviewed; the fresh validator passed the tip after round
two and the final tip 1e8adb72d passed a fresh validator (14-scenario
independent harness of the law, full gate 71/71 with a HEAD-built
binary, CI bats 1164/0) and a cross-family read by Gemini 3.8 via AGY,
which closed all six remaining residues with no new findings; Codex was
unreachable at push time).
**Follow-ups filed from the final reviews, not blockers:** the JS
violation check tests growth before reopen while Python tests reopen
first (same stop, different label when both occur in one round);
`cli/testdata/compatibility-baseline/families/{demo,quickstart}/case.json`
assert help-text substrings Cobra never prints (pre-existing, no
consumer); runtime derivation of changed paths outside every agent in
the Claude conveyor.
2026-09-03 11:16:08 -04:00
|
|
|
crank --> rpi
|
2026-07-14 22:01:50 -04:00
|
|
|
rpi --> implement
|
|
|
|
|
rpi --> plan
|
|
|
|
|
rpi --> validate
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Hard dependencies only. Optional context relationships are listed in the context map.
|