mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
Bo
2ef932b617
## RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated Provenance: running the loop for real across three trains on 2026-09-03 (#1099, #1100, #1101). The spine held; the flow around it did not. A risky-surface design shipped with no premortem and needed six repair passes. The open finding set grew three rounds running inside one class while the id-counting law watched. A same-family PASS and a cross-family FAIL had no tie-break until the caller named one afterwards. Every harness edit orphaned bound evidence with no warning until verify failed. A doc called an egress log published while the repository's `*.log` rule kept it out of the tree, and the same overclaim class came back four rounds. **What changes.** - **Premortem at plan exit.** When the write scope hits a risky surface, one fresh judge reads the frozen plan before Implement; a blocking finding returns NOT_PLANNED with the findings. The caller may declare `premortem: skip`; the report says so. - **A split is recorded, never elected and never closed inside the loop.** The law stands: a risky surface converges only when both legs PASS, and no finding leaves the open set because someone chose a judge. Plan declares `binding_judge` as the caller's disposition for a split that survives repair; it is bound in the plan identity, carried in the report, and never mutates the verdict. On a risky split that survives the repair budget (never after a law stop), one council leg receives a bounded packet marked untrusted and returns per-finding rulings, exactly one per id, which the traversal validates and records under `council.rulings` for the caller's next intent; the verdict and the open finding set stay exactly as repair left them. A council closure mechanism was built, drew a new finding class in three consecutive judge rounds (path containment, digest exclusion, presence-as-evidence, partial closure downgrading a FAIL), and was cut under the train's own class rule rather than hardened again. A non-risky split stays worst-of. - **Finding class in the convergence law.** Findings carry an optional stable `class`. A class carried by a resolved id that reappears on a new id with no surviving prior id carrying it is `class_reopened`, whether or not a classless round sits between; a blank class is an invalid round; a round can carry both a reopened id and a reopened class and stops on either. Repair stops, the traversal returns NOT_PROVEN, and the caller goes back to Plan, because the design is wrong, not the patch. Same rule in `run_once.py` and `workflows/rpi.js`, driven by one shared fixture file, with executed canaries in the cathedral gate. - **Plan identity, and the premortem's risky test.** On a risky scope the Plan carries `binding_judge` and the evidence it expects to orphan; `plan_digest`, when the plan declares one, must equal the script's digest over the acceptance, the write scope, the binding judge, and the intent-source digest, or the traversal stops before Implement. The risky test over a declared scope is glob intersection: witness paths, the risky regexes over the glob with wildcards instantiated, and any literal prefix that reaches a risky root; a bare `**` or `*` is risky. - **Intent is verified before anything is built.** An `intent-snapshot` receipt re-derives the digest and compares the snapshot bytes to the caller intent before premortem or Implement; absent, mis-hashed, or wrong stops the traversal with `plan_identity_mismatch`. The Python reference refuses on the same rule. Declared scopes are canonicalized (repeated `./` collapsed; absolute paths and `..` refused at plan exit), and the four spellings of `tests/**` share one plan digest while `tests/` does not. - **Orphaned evidence is a receipt, not a surprise.** After Implement and after every repair round the workflow runs `scripts/evidence-orphans.sh` over the runtime-derived path union, keyed on the round so a same-set repair still reruns it and appends every scorecard and fixture set whose bound evaluator or skill digests cover them to the check receipts the validator reads, each with a cause (`changed_path`, `digest_drift`, `both`, `skill_changed`). Plan budgets that recapture up front. On today's tree the receipt already lists the 08-16 and 08-26 sets whose bound harness digests moved long ago. - **Doc claims are gated.** The stronger scan already caught one stale claim in the tree (a 2026-06-22 harvest note calling evidence committed after Train 2 had removed that tree); the sentence now says where the evidence went. `docs.claims-tracked` (blocking) requires backticked repo paths in `evals/` and `docs/evals/` markdown to be tracked, and sentences that say published, tracked, or committed to name files that exist. - **Risky scope is a glob intersection, not a heuristic.** One segment-wise `globsIntersect` over one list of risky segment globs answers both the declared-scope question at Plan exit and the changed-path question for the cross-family rule; two heuristic designs missed in opposite directions and were replaced rather than patched. - **Contracts.** rpi gains a judgment dispatch table: risky scope routes to premortem, a risky split to council, an irreversible landing to one-way-door as a caller choice. plan, implement, validate, premortem, council, and learn carry their halves; learn proposes a deterministic check for any finding class that repeats. `docs/architecture/rpi-traversal.md` is the source of record and was updated with the flow. **Evidence on the tip.** Regen check clean; Go bar; full gate green with a HEAD-built binary (the new gate fires and passes); CI's literal bats command green; rpi unit tests; cathedral gate with executed canaries for the class law; routing goldens; token budgets; gate-tightening ratchet; coverage 1 of 12 unchanged. **Judgment, stated plainly.** Six judging rounds, each on a frozen tip. The caller elected Fable 5.1 as the binding validator; its round-6 verdict is PASS with no verdict-driving findings (lows: a council payload outside the schema throws a raw error instead of recording `invalid-rulings`; two stale comments; a dead parameter; a dead fixture element; the contracts say exactly one ruling per finding while the code enforces at most one). The cross-family Codex read of the same tip failed on three input-validation items, which the final commit closes: control characters are now refused on the original scope value with the full Unicode control class and the caller scope is canonicalized in place so one spelling reaches every prompt; the intent receipt no longer forgives a trailing-newline difference; and the Python reference accepts a digest string only from a verifier callback and matches digests with `fullmatch`. One half of one Codex item stays open and is disclosed: the workflow does not itself hash the caller intent's bytes to compare with the plan's declared digest (it relies on the receipt's exact-bytes match plus digest equality); binding that locally means every probe fixture carries a real digest of its intent text, which is the next pass. Also open, pre-existing and outside this train: the risky-surface list names `lib/**` while the preamble every gate sources lives under `scripts/lib/`. Across the rounds the loop behaved as designed once: the council closure mechanism drew a fresh finding class three rounds running and was cut under the train's own class rule instead of hardened a fourth time.
ao — AgentOps CLI
ao supplies deterministic repository utilities and evidence inspection. It
is the optional checks/linking CLI of the AgentOps operations layer; the
semantic protocol lives in the skills:
RPI → Plan → Implement → fresh Validate → repair to convergence → report
The CLI does not own retries, queues, work claims, Git delivery, release, closure, or semantic validation. Consumer repositories choose their own Git and CI policy.
Install
go install github.com/boshu2/agentops/cli/cmd/ao@latest
Current executable truth
ao capabilities
ao robot-docs
ao --help
The generated command reference follows the published Cobra tree. Removed lifecycle commands are not registered at all: invoking one fails as an unknown command with a pointer to its replacement, and no build tag or compatibility profile restores their implementation.
Development
make build
make test
Add deterministic utilities only when they do not become lifecycle or delivery authorities. Keep semantic judgment in the Validate skill, optional verdict persistence with declared consumers, and external delivery in the consumer repository.