mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
e1fae0dae63d73be508bb8348a12106df043cab0
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bo
|
e1fae0dae6 |
Make the engineering harness lean and add optional topic memory (#1116)
RPI now owns the authorized outcome through finish, with Plan and Memory
loaded only when useful. Known defects get direct repair, and evidence
can change the approach under unchanged acceptance. Fresh exact-content
validation remains required. Memory provides optional recall, mining and
curation of reviewed topic pages; specialists and the fixed-dispatch
adapter remain optional.
The change reconciles current documentation and generated skill
projections. It preserves native budget and permission authority, BD
work ownership, protected external evidence storage and the distinction
between a supported lesson and demonstrated later benefit. It adds no
scheduler, work store, Go command or evidence schema.
Validation: required local Go/build/vet/race checks, aggregate suite,
generated-output check and 72 gates pass. The complete 44-test executor
suite passes; its shared-deadline fixture now tolerates CI scheduling
jitter while still requiring deadline exhaustion and preventing a third
launch. Fresh author-distinct review passed all 112 changed paths with
no findings; all seven exact-head CI checks passed at
|
||
|
Bo
|
8061085c89 |
Ship native evidence helpers and fresh-family review defaults (#1110)
AO now performs intent snapshots, subject manifests, strict evidence verification, atomic verdict storage, and orphan inspection through the Go binary. The command handler keeps verification separate from presentation so it meets the existing complexity limit. These operations preserve the existing evidence formats, require explicit protected storage where applicable, and run outside a checkout without Python. The unchanged Python implementation remains a developer oracle; agents still provide semantic judgment. Codex and Claude skills now default to a fresh reviewer from the author’s model family. Callers can explicitly request cross-model review or pin its model. Reviewer adapters use a finite caller timeout or remaining deadline instead of a fixed ten-minute default, while retaining output limits and abnormal-termination cleanup. Validation: Go build, vet, tests and race/shuffle tests; 1,334 shell tests; aggregate runner; regeneration check; 72 full-mode gates. Independent checks exercised 84 storage-boundary rejections and 21 evidence operations with an empty PATH. Both canonical and generated RPI reference suites pass all 48 tests after updating the migrated oracle import without weakening assertions. Change-sensitive checks explicitly compare the final committed candidate with the original PR base. Linux, Windows, installer, security, and required summary checks are green. |
||
|
Bo
|
af1e530159 |
Adopt CDLC contracts and reconcile BD tooling (#1108)
AgentOps now describes the selected Context Delivery Lifecycle alongside standalone RPI: external reviewed memory, native work and runtime ownership, independent validation, and explicit limits on what the pilots proved. The active ADRs name which historical prescriptions are superseded and which invariants remain. This integrates the BD foundation, including doctor diagnostics that distinguish BD from BR and protection against private tracker data entering outgoing Git history. It also fixes skills audit resolution of explicit relative reference paths. Source-owned projections were regenerated, and the landed stopping behavior is preserved. Validation: Go build/vet/test and race/shuffle, 1,325 Bats cases, the local aggregate, and full AO gates. The gate registry reports 71 passes and one nonblocking preamble-adoption failure on the existing report script; that advisory remains disclosed. Historical probe receipts whose source hashes changed remain historical evidence. This is foundation adoption. It does not claim that the later memory, goal, background, privacy-release, or benefit-measurement work is implemented. Independent Astra and Fable judgments both PASS over the exact integrated content. Earlier negative results and their factual resolution remain preserved in private work evidence. |
||
|
Bo
|
baa24e1b04 |
Bound headless execution and require acceptance-linked repair progress (#1107)
Headless invocations could run without a finite bound, and repair loops could count changed digests or finding counts as progress. The existing adapter now enforces positive finite deadlines, an optional inherited absolute deadline, bounded output capture and cleanup of its owned process group, refusing execution when protection is unavailable. The existing goal, RPI and validation skills now require acceptance-linked progress, distinguish newly discovered failures from regressions, and bound Andon recovery. Review effort follows the effect on acceptance and enforcement. These contracts do not claim native aggregate goal-budget or pause enforcement. Validation on the clean main baseline: - 1,317 Bats cases, zero failures, two skips; 46 RPI reference cases. - Go build, vet, tests and race/shuffle tests; local aggregate runner. - Generated projections current; all 72 AO gates pass. - Fresh independent Astra and Fable 5.1 xhigh judgments passed the exact clean source. Earlier uncommitted CDLC adoption, private work/evidence, and unrelated Go changes are excluded from this PR. |
||
|
Bo
|
021b41c951 |
Skills only: cut the RPI workflow script and the machinery around the loop (#1105)
## Skills only: cut the RPI workflow script and the machinery around the loop #1104 landed the right ideas in the wrong form. This repository is skills only: a skill is an instruction to a system, models route and judge, and deterministic checks exist where reality is deterministic (digests, tests, gates, "is this file tracked"). #1104 built a workflow script and a set of schemas, enums, and engines where instruction was wanted, and then spent four judge rounds hardening them. This PR deletes that and keeps the ideas as instruction. **Deleted.** `workflows/rpi.js` and everything that existed only to exercise it: its probe harness in the product-boundary suite, the convergence-law and glob-intersection fixture corpora, the cathedral canaries that pinned its mechanisms, and the pointers to it. **Reverted to the #1099 form.** `skills/rpi/scripts/run_once.py` and its tests: the repair law as accepted in the loop restore, without the class-law machinery, the snapshot verification, or the cross-leg checks. **Kept.** `docs.claims-tracked` (a blocking gate that caught a real stale claim on its first run), `scripts/evidence-orphans.sh` (a receipt a shell script computes), the optional `findings[].class` field on `verdict.v2` (a word a validator writes), and the corrected harvest-note sentence. **Kept as instruction.** The rpi, plan, implement, validate, council, learn, and boundaries contracts and `docs/architecture/rpi-traversal.md` now say, in plain sentences a model follows: a risky change gets one fresh judge on the frozen plan before Implement; validators name a class per finding and a returning class means go back to Plan; two judges disagreeing is the orchestrator's decision made in the open, with both reads in the report and a risky surface still needing both to pass; the orphan receipt runs after Implement and each repair and rides in the check receipts; a doc sentence that says published, pinned, or proven is a claim to check; Learn proposes a check for any repeating finding class. **Evidence on the tip.** Regen check clean; rpi unit tests; cathedral gate; routing goldens; token budgets; coverage 1 of 12; Go bar; full gate green with a HEAD-built binary; CI's bats command green; gate-tightening ratchet. One fresh read by Fable 5.1, asked in plain language for what it found, recorded in the PR thread. |
||
|
Bo
|
2ef932b617 |
RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated (#1104)
## RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated Provenance: running the loop for real across three trains on 2026-09-03 (#1099, #1100, #1101). The spine held; the flow around it did not. A risky-surface design shipped with no premortem and needed six repair passes. The open finding set grew three rounds running inside one class while the id-counting law watched. A same-family PASS and a cross-family FAIL had no tie-break until the caller named one afterwards. Every harness edit orphaned bound evidence with no warning until verify failed. A doc called an egress log published while the repository's `*.log` rule kept it out of the tree, and the same overclaim class came back four rounds. **What changes.** - **Premortem at plan exit.** When the write scope hits a risky surface, one fresh judge reads the frozen plan before Implement; a blocking finding returns NOT_PLANNED with the findings. The caller may declare `premortem: skip`; the report says so. - **A split is recorded, never elected and never closed inside the loop.** The law stands: a risky surface converges only when both legs PASS, and no finding leaves the open set because someone chose a judge. Plan declares `binding_judge` as the caller's disposition for a split that survives repair; it is bound in the plan identity, carried in the report, and never mutates the verdict. On a risky split that survives the repair budget (never after a law stop), one council leg receives a bounded packet marked untrusted and returns per-finding rulings, exactly one per id, which the traversal validates and records under `council.rulings` for the caller's next intent; the verdict and the open finding set stay exactly as repair left them. A council closure mechanism was built, drew a new finding class in three consecutive judge rounds (path containment, digest exclusion, presence-as-evidence, partial closure downgrading a FAIL), and was cut under the train's own class rule rather than hardened again. A non-risky split stays worst-of. - **Finding class in the convergence law.** Findings carry an optional stable `class`. A class carried by a resolved id that reappears on a new id with no surviving prior id carrying it is `class_reopened`, whether or not a classless round sits between; a blank class is an invalid round; a round can carry both a reopened id and a reopened class and stops on either. Repair stops, the traversal returns NOT_PROVEN, and the caller goes back to Plan, because the design is wrong, not the patch. Same rule in `run_once.py` and `workflows/rpi.js`, driven by one shared fixture file, with executed canaries in the cathedral gate. - **Plan identity, and the premortem's risky test.** On a risky scope the Plan carries `binding_judge` and the evidence it expects to orphan; `plan_digest`, when the plan declares one, must equal the script's digest over the acceptance, the write scope, the binding judge, and the intent-source digest, or the traversal stops before Implement. The risky test over a declared scope is glob intersection: witness paths, the risky regexes over the glob with wildcards instantiated, and any literal prefix that reaches a risky root; a bare `**` or `*` is risky. - **Intent is verified before anything is built.** An `intent-snapshot` receipt re-derives the digest and compares the snapshot bytes to the caller intent before premortem or Implement; absent, mis-hashed, or wrong stops the traversal with `plan_identity_mismatch`. The Python reference refuses on the same rule. Declared scopes are canonicalized (repeated `./` collapsed; absolute paths and `..` refused at plan exit), and the four spellings of `tests/**` share one plan digest while `tests/` does not. - **Orphaned evidence is a receipt, not a surprise.** After Implement and after every repair round the workflow runs `scripts/evidence-orphans.sh` over the runtime-derived path union, keyed on the round so a same-set repair still reruns it and appends every scorecard and fixture set whose bound evaluator or skill digests cover them to the check receipts the validator reads, each with a cause (`changed_path`, `digest_drift`, `both`, `skill_changed`). Plan budgets that recapture up front. On today's tree the receipt already lists the 08-16 and 08-26 sets whose bound harness digests moved long ago. - **Doc claims are gated.** The stronger scan already caught one stale claim in the tree (a 2026-06-22 harvest note calling evidence committed after Train 2 had removed that tree); the sentence now says where the evidence went. `docs.claims-tracked` (blocking) requires backticked repo paths in `evals/` and `docs/evals/` markdown to be tracked, and sentences that say published, tracked, or committed to name files that exist. - **Risky scope is a glob intersection, not a heuristic.** One segment-wise `globsIntersect` over one list of risky segment globs answers both the declared-scope question at Plan exit and the changed-path question for the cross-family rule; two heuristic designs missed in opposite directions and were replaced rather than patched. - **Contracts.** rpi gains a judgment dispatch table: risky scope routes to premortem, a risky split to council, an irreversible landing to one-way-door as a caller choice. plan, implement, validate, premortem, council, and learn carry their halves; learn proposes a deterministic check for any finding class that repeats. `docs/architecture/rpi-traversal.md` is the source of record and was updated with the flow. **Evidence on the tip.** Regen check clean; Go bar; full gate green with a HEAD-built binary (the new gate fires and passes); CI's literal bats command green; rpi unit tests; cathedral gate with executed canaries for the class law; routing goldens; token budgets; gate-tightening ratchet; coverage 1 of 12 unchanged. **Judgment, stated plainly.** Six judging rounds, each on a frozen tip. The caller elected Fable 5.1 as the binding validator; its round-6 verdict is PASS with no verdict-driving findings (lows: a council payload outside the schema throws a raw error instead of recording `invalid-rulings`; two stale comments; a dead parameter; a dead fixture element; the contracts say exactly one ruling per finding while the code enforces at most one). The cross-family Codex read of the same tip failed on three input-validation items, which the final commit closes: control characters are now refused on the original scope value with the full Unicode control class and the caller scope is canonicalized in place so one spelling reaches every prompt; the intent receipt no longer forgives a trailing-newline difference; and the Python reference accepts a digest string only from a verifier callback and matches digests with `fullmatch`. One half of one Codex item stays open and is disclosed: the workflow does not itself hash the caller intent's bytes to compare with the plan's declared digest (it relies on the receipt's exact-bytes match plus digest equality); binding that locally means every probe fixture carries a real digest of its intent text, which is the next pass. Also open, pre-existing and outside this train: the risky-surface list names `lib/**` while the preamble every gate sources lives under `scripts/lib/`. Across the rounds the loop behaved as designed once: the council closure mechanism drew a fresh finding class three rounds running and was cut under the train's own class rule instead of hardened a fourth time. |
||
|
Bo
|
568e99d436 |
Loop restore: converge and crank as control flow under the verdict contract (ADR-0017) (#1099)
## Loop restore: converge and crank as control flow under the verdict
contract (ADR-0017)
Intent source: `docs/plans/2026-09-03-loop-restore.md` (in this PR).
Decision record:
`docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md`.
**Why.** The 2026-07-14 single-pass cut (`482307762`) removed the
iterate loop (discovery, crank, converge, evolve, the learn write-half)
together with the unproven compounding claim, although ADR-0011 demoted
only the latter. The control flow was never demoted, and its absence
showed on 2026-09-02, when a three-lane fix needed eight validators and
two stops because the contract had no repair phase. This restores the
loop as control flow and nothing else: no knowledge store, no `ao
converge`/`ao crank`, no evolve, no canary. ADR-0004 and ADR-0011 stay
in force.
**What changes.**
- **RPI gains a bounded repair phase.** On `FAIL` or `NOT_PROVEN` with
findings, repair and re-validate freshly under the convergence law:
caller-declared `repair_rounds` (default 2); open finding set keyed by
stable `findings[].id`, union across validator families, non-growing; no
closed id reopens; the subject digest changed or, for `NOT_PROVEN`, new
digest-bound evidence resolved a named gap. Converged = fresh PASS plus
cross-family PASS on risky surfaces. Plan and Implement keep their
single dispatch. `skills/rpi/scripts/run_once.py` models the law as pure
data (33 tests): rounds are validated for shape (digest required, no
duplicate ids, no PASS with findings, no FAIL without findings),
condition 4's evidence branch needs a NOT_PROVEN previous round, a
non-FAIL current round, new evidence, and a resolved finding, and a PASS
over unchanged bytes after a FAIL is a flip that reports NOT_PROVEN.
`workflows/rpi.js` runs validation as legs (spawned or external primary,
plus a caller-supplied `crossFamily.command` on risky surfaces) merged
worst-of with a union of stable ids; a risky surface without a
cross-family leg is `diversity_unsatisfied` and never converges or
enters repair; a failed repair or re-validation returns NOT_PROVEN with
no stale verdict. Validators return `subjectDigest`, stable finding ids,
and `evidenceRefs`.
- **crank returns as a thin wave executor** (113 lines): the caller
selects the wave and the repair bound, crank invokes RPI per lane
(parallel only on disjoint write and regen scopes), runs the wave
acceptance once, returns evidence, and stops. No retry, budget, queue,
claim, lease, Git, closure, or next-work ownership. Routing golden
`rq-07-wave-execution` ranks it first.
- **validate is cross-family by default on risky surfaces**
(`cli/internal/gates/**`, `scripts/check-*.sh`, `tests/**`,
`skills/*/scripts/**`, hook policies, `lib/**`, security-scanned paths)
with the LAW-0 dispatch table: Claude orchestrating uses read-only
`codex exec`; Codex orchestrating uses an interactive Claude session in
an NTM pane, never `claude -p`. No live adapter means
`diversity_unsatisfied`, which on a risky surface is `NOT_PROVEN`. The
full literal CI command set runs once on the final integrated subject;
routine rounds keep the receipt-driven freshness contract.
- **Conformance assertions flipped under ADR-0017 only:**
`scripts/check-cathedral-cut-conformance.py` (crank live; "Stop
regardless" replaced by positive canaries for the law's four conditions;
a bounded `for` loop that compares against `repair_rounds` is required
in `run_repair_phase`, and the gate executes the law's canaries against
the reference behavior), `workflows/rpi.js`,
`skills/rpi/scripts/validate.sh`,
`evals/agentops-core/rpi-behavior.json`,
`skills/rpi/references/rpi.feature`. Every single-pass public surface
(README, AGENTS.md, PRODUCT.md, CI-CD, agent-workflow-reference,
rpi-traversal, cli/README, quickstart and demo commands, the
operating-contract and product-boundary bats, the Codex-description
oracle) now states repair to convergence.
**Known approximation, disclosed.** The Claude conveyor has no
deterministic shell primitive, so changed paths are derived by the fresh
validator (git status and diff against the clean pre-run tree) and
unioned with the implementer's report; risk is classified over that
union and unreported paths are coverage findings. A validator is still a
model; runtime derivation outside every agent is a follow-up. Family
distinctness of the cross-family leg is asserted by the caller's choice
of command and not verified by the script.
**Not in scope.** Premortem stays a single advisory judge and Plan still
only names the first check (phase boundaries unchanged). No `verdict.v2`
or `rpi-report.v1` change. The loop's own effect on outcomes is
unmeasured and owed a seeded-defect probe, like the rest of the corpus.
**Evidence on the tip.** Regen check clean; full gate green with a
HEAD-built binary; CI's bats command green; Go build/vet/test green;
golangci-lint clean; security gate quick PASS; one fresh validator over
the whole diff; one cross-family read of the design before
implementation (13 findings folded) and two of the integrated diff (9
findings in round one, 11 by round two, 15 by round three, each round
repaired and re-reviewed; the fresh validator passed the tip after round
two and the final tip
|
||
|
Bo
|
8694b07ebf |
feat(skills): add anti-ceremony guard to RPI (#1064)
## Summary\n\n- add a clean-room, artifact-free anti-ceremony skill\n-
make RPI invoke its STOP/CONTINUE guard exactly once before Plan\n- STOP
before dispatching any core phase; CONTINUE preserves Plan → Implement →
fresh Validate\n- retain durable intent by reference and snapshot only
non-durable intent\n- update the canonical traversal, workflow,
standards, dependency contracts, tests, and generated projections\n\n##
Verification\n\n- RPI unit tests: 13/13\n- dependency/product-boundary
Bats: 15/15\n- RPI and Anti-Ceremony validators: PASS\n- Cathedral Cut
conformance: PASS\n- mesh, Codex/Gemini projections, parity, hashes,
manifests, and runtime formats: PASS\n- changed-scope
generation/contract checks: PASS\n- full exact-range gate: 68 pass, 0
warn, 0 fail, 0 unknown, 0 skip\n- fresh final semantic review on head
|
||
|
Bo
|
8d890ac1ca |
Align skill contracts with operating contract and regenerate projections (#1062)
### Motivation - Resolve inconsistencies between skill contracts and the repository operating contract and README, where some skills implied starting downstream execution, multi-behavior planning, or implicit durable state creation. - Make routing decisions advisory-only and ensure orchestration and factory adapters remain owner-operated rather than implicitly started by routers. - Ensure RPI, Plan, and Implement respect a single caller-owned intent source, fresh-validation boundaries, and explicit caller authorization for scope or multi-lane changes. - Keep Bootstrap explicitly opt-in for creating durable verdict storage and project docs to avoid unexpected repo changes. ### Description - Updated skill contracts to enforce the operating contract: `skills/plan/SKILL.md` removed the manifest-mode trigger and now consistently shapes one active behavior in the existing intent source. - Clarified RPI semantics in `skills/rpi/SKILL.md` to preserve durable caller-owned intent by reference and only snapshot when no durable source exists, and to stop/report when a fresh-context test fails rather than auto-starting new planning lanes. - Made routing advisory-only in `skills/automation-shape-routing/SKILL.md` so the router returns the chosen owner without copying or starting the delegated workflow. - Constrained `bootstrap` in `skills/bootstrap/SKILL.md` to create only explicitly requested project docs and optional `.agents/ao/verdicts` storage, and adjusted `implement` and `swarm` wording to stop/return factual evidence rather than implicitly revise or declare semantic verdicts. - Regenerated derived projections and parity twins (Codex and Gemini artifacts), updated `skills-codex/*` generated metadata, and updated the `skills/catalog.json` to reflect the revised descriptions and hashes. ### Testing - Ran regenerated-projections and parity checks with `PYTHONPATH=/tmp bash scripts/regen-all.sh --check --skills automation-shape-routing,bootstrap,implement,plan,rpi,swarm,using-gc` which completed and reported projections current. ✅ - Validated Codex artifacts and coverage with `bash scripts/validate-codex-generated-artifacts.sh` and `bash scripts/validate-codex-override-coverage.sh` which passed. ✅ - Ran repository checks including `scripts/check-skill-size.sh`, `scripts/check-doc-skill-refs.sh`, `scripts/check-orchestration-skill-boundaries.sh`, `scripts/check-skill-python-ratchet.sh`, and per-skill validators `skills/plan/scripts/validate.sh` and `skills/implement/scripts/validate.sh`, all of which passed. ✅ - Frontmatter/schema checks initially failed due to missing Python packages (`PyYAML`, `jsonschema`) in the environment and network restrictions prevented installation, but parity, mesh generation, lint, and the metadata-driven regeneration were completed using a local shim so the main automated validation suite passed with the noted environment limitation. ⚠️ ------ [Codex Task](https://chatgpt.com/codex/cloud/tasks/task_e_6a7e8a00a680832b808af3ef40055848) |
||
|
Bo
|
7a765cde19 |
Align AgentOps around its operations-layer identity (#1051)
Executes docs/plans/2026-08-07-agentops-operations-layer-alignment.md: AgentOps is the operations layer for agentic engineering; the federated integration graph is the topology, the semantic work-and-proof protocol is the contract, and RPI is the standard one-experiment traversal. Retires the ao flywheel command family and all knowledge-flywheel product state, tombstones the seven-move operating-loop workflow, narrows ao init and the .agents state writers to declared destinations, renames the core architecture page to rpi-traversal.md with a compatibility redirect, aligns AGENTS.md, 25 skills, public and package copy, regenerates every owned projection, and strengthens the conformance gates with planted-negative proofs. Both the alignment subject and the follow-up gate-bookkeeping commit carry fresh author-distinct validation PASS verdicts with empty not_checked scope. Test-Removal-Reason: dead knowledge-flywheel and session-store surfaces were deleted with their tests (operations-layer alignment) |