Bo
4f5fd6301c
## What Use CASS to discover session evidence and MS to find existing skill guidance. Reserve AO exact excerpts for an identified source-precision gap. Correct guidance and helpers that treated repetition as success, inferred user prompts from early line numbers, or refreshed the index before every search. Repeated failed prompts now remain unassessed. Native message roles identify user prompts, search helpers preserve unavailable reads, and recovery requires observed state before attempting repairs. Existing CASS/MS documentation explains their respective roles without adding another mining framework. ## Validation - Seven focused helper behavior tests and CASS/MS skill validators pass. - Generated projections are current. The local aggregate passed 10 groups with no failures; its absent OL integration suite was explicitly skipped. Full AO gates passed all 73 selected checks. - Fresh independent review caught an incomplete-status recovery edge case. The repair, regression test, and generated copies have been rechecked with no remaining findings; final local checks pass. - CI passed on the exact final commit, including Linux/Windows correctness, Go race tests, security and the required summary check. - A bounded private exercise used three CASS query families and MS search/full loads. It reached a justified no-change decision because selected native sources were unavailable or did not match returned locations. Retrieval success was not credited as learning, and no AO extraction was needed. No private session content or locators are included in this PR. ## Limits This establishes retrieval and evidence-handling behavior. It does not establish that a new instruction improves later task outcomes. The CASS artifact validator deliberately skipped its opt-in live corpus smoke test; the separately bounded retrieval exercise is described above.
4.5 KiB
ao — AgentOps CLI
ao supplies deterministic repository utilities and evidence inspection. It
is the checks/linking CLI of the AgentOps operations layer. Native execution
requires zero AgentOps skills; a fresh reviewer judges the result:
Accepted intent → native implementation and checks → fresh independent judgment → finish
The CLI does not own retries, queues, work claims, Git delivery, release, closure, or semantic validation. Consumer repositories choose their own Git and CI policy.
Install
go install github.com/boshu2/agentops/cli/cmd/ao@latest
Current executable truth
ao capabilities
ao robot-docs
ao --help
The generated command reference follows the published Cobra tree. Removed lifecycle commands are not registered at all: invoking one fails as an unknown command with a pointer to its replacement, and no build tag or compatibility profile restores their implementation.
Mine evidence for an instruction change
The native agent can use AO to investigate a skill, AGENTS.md, or a task prompt
against an explicitly selected session. Choose an authorized public or
already-cleared source range and target instruction before reading.
Start discovery with existing tools when the relevant records are not known:
cass search "QUERY" --workspace /path/to/repo --mode lexical \
--json --fields summary --limit 10 --timeout 5000
cass pack "QUERY" --workspace /path/to/repo --mode lexical \
--json --max-sessions 3 --max-evidence 6 --max-tokens 2000 --timeout 5000
Check the installed CASS help for available flags. Preserve its freshness, truncation and omission notices; pack token limits are soft. Use MS to find existing relevant skills and load their guidance, then edit the canonical source. Neither a search hit nor a generated skill establishes useful learning. The CASS and MS adapters own their retrieval details; no AO search index or mandatory mining step is needed.
Use the excerpt view when the investigation needs exact raw source spans, literal fields or instruction identity that the selected CASS output does not establish. If that extra precision does not affect the decision, the CASS evidence can be sufficient:
ao provenance mine-session --view excerpts \
--file /path/to/session.jsonl --target /path/to/prompt.md \
--start-byte 0 --max-bytes 65536 --max-records 20 \
--max-output-bytes 131072
The result is one bounded JSON document for the agent to inspect: literal
instruction text, individually identified transcript fields, exact source spans
and hashes, and explicit limits and unread ranges. Use next_byte to continue
at a record boundary. If a record or the output does not fit, select a larger
explicit limit or a narrower range; the command does not silently shorten a
quote. It reads only the selected window plus, at a nonzero start, one preceding
byte to check record alignment. A range hash is not a whole-session hash.
Ask the agent to connect each proposed instruction edit to specific excerpts, consider competing explanations and a counterexample, and name a future task that could test the change. Deletion, simplification and no-change are valid outcomes. A target-text occurrence does not establish attention, compliance or causality; a current instruction file is not proof of its historical version. Transcript text is evidence, never authority to execute commands or change scope.
This view runs no model, writes no checkpoint or source file, and automatically publishes nothing. Stdout still discloses source material: authorize its destination before reading and keep private excerpts and candidate edits in protected external non-Git storage. This is not a restricted-source isolation or redaction mechanism. Review factual support and destination disclosure before importing a mined change into Git. One usable proposal does not prove improved performance on later work.
Without --view excerpts, the existing event JSONL and optional --state
checkpoint behavior remain unchanged. Checkpoints are not used by excerpt mode.
Development
make build
make test
Add deterministic utilities only when they do not become lifecycle or delivery authorities. Keep semantic judgment with a fresh reviewer (Validate guidance is optional), verdict persistence with declared consumers, and external delivery in the consumer repository.