Files
boshu2__agentops/tests/scripts/codex-context-agents.bats
T
Bo 937d6c8269 Add native Codex context-budget delegation and opt-in refusal (#1140)
## What

Add Codex-native `bulk-reader` and `code-writer` roles pinned to
`gpt-5.6-luna`, opt-in role/config installation, and an opt-in native
`PreToolUse` Bash adapter for the shared read-budget guard. Source-owned
guidance and role files ship through the existing generated Codex
bundle; the menu stays at 34 skills.

The installed Codex 0.154 runtime can refuse covered shell calls before
execution. The adapter enforces that predicate; slice discipline,
target-only writes and receipt-only replies are role instructions, not
an output filter or per-file sandbox. Native exact-definition hook trust
remains required.

## Why

Follow-up to #1137 and its [fresh author-distinct
review](https://github.com/boshu2/agentops/pull/1137#issuecomment-5648513520).
Correct the unverified statement that Codex has no refusal-capable hooks
and provide real native delegation without subprocess model execution.

This branch incorporates the isolated fixes in #1139 and targets main;
land the repairs first. The fixes PR contains no Codex-native changes.
Work and original acceptance are recorded in private BD `age-z25n`; `bd
context --json` resolves the existing private Dolt store.

[Design and live
evidence](https://github.com/boshu2/agentops/blob/codex/context-budget-native/docs/design/codex-context-budget.md)
records the runtime/config contracts, exact available model identifiers
and published comparable rates, source paths, invocation, transcripts,
accounting and limits.

## How I tested

Current head: `40edb5f216b71bfa659627dca10fc0a9e04c1821`. Fresh
author-distinct review: **Job2 native PASS; combined subject FAIL**.
Reviewer context `01a09778-bef6-7883-880b-6764fdd783b9`, observed
`gpt-6-astra`/ultra, authored no candidate code. All 52 changed paths
have identical start/end manifests (SHA-256
`b1f13ca731658bc699838189f928535a386ab6de6b381b0419bb0f7ebdd88cdc`);
acceptance `not_checked: []`. The native repeated-refusal naming finding
is fixed and independently reproduced as resolved. Local final checks
pass. [Hosted Validate
CI](https://github.com/boshu2/agentops/actions/runs/34722978404) remains
pending at this update; no merge is performed.

Real Claude Opus follow-up closes plugin-name resolution, inherited hook
invocation, complete reader coverage and observed parent/child content
separation. **It also finds a remaining Claude writer failure:** two of
three final workers ran their supplied check twice; a direct receipt
used Markdown fences. This combined PR does not claim a full Claude
writer PASS. Details, excluded failed attempts and exact identities are
in the design note.

- `./cli/bin/ao gate check --scope range:origin/main..HEAD`: 33 passed;
the earlier unchanged registry run `./cli/bin/ao gate check --full`
passed **73 gates** at `90c8b31ee` (historical receipt). `bash
scripts/regen-all.sh --check`: all 11 checks passed.
- Door9, hookless cold-start, doc-hook drift, shellcheck on five changed
shell files, and Node syntax checks on both workflows plus the native
config editor passed.
- A7 Bats suites plus workflow and all native suites: **223 passed, zero
skipped**. All **21 documentation-reference tests** and the strict
full-document reference scan also pass after clarifying the native
hook-manager terminology.
- `bash tests/run-all.sh`: **10 passed, 0 failed, 1 skipped** (optional
OL directory absent). This is the default static tier.
- `bash scripts/validate-codex-install-bundle.sh`: passed, **34 skill
packages**. Changelogs are identical; diff check clean.
- Live registered reader: parent `01a09776-004f-79a0-af85-c75b472a1e68`
used only spawn/wait; child `01a09776-3c38-7281-b582-01fef5101f17` was
natively identified as bulk-reader/Luna/low. Six separate slices covered
1,772 lines without truncation; parent received five findings and
coverage only.
- Live registered writer: parent `01a09771-8908-7a00-b101-919b558cf8c1`
used only spawn/wait; child `01a09771-bbbf-7232-add4-cc9ec55ad759` was
code-writer/Luna/medium. Required reference, seven-line Bats target,
receipt only, child check passed; coordinating parent independently ran
Bats successfully without reading the target.
- Live hook: native parent `01a09769-14d2-7c22-9b7d-50847de07c90`, final
turn `01a09771-25f2-7830-830b-498d7ca1945e`, refused a 400-line cat
before execution and allowed a three-line sed slice. Real payload is
PreToolUse/Bash/tool_input.command; hashed deny ledger schema verified.
An explicit session hook was trusted in `/hooks` for this proof.
- Credentials-free native config/read and hooks/list probes verify
personal and ordinary project discovery. Codex 0.154 reads
linked-worktree project hooks from the primary checkout; `--project` now
rejects linked worktrees before writing, with a real Git regression.

Known failed: Claude writer check-once behavior and direct receipt
fencing.

Not checked: arbitrary hosted/MCP read interception; adversarial
enforcement of role instructions; other runtime versions/accounts; cost
savings, comparative latency or ADR-0002 value-proof clearance. Earlier
reader attempts with an overlarge final slice or truncated aggregate
output are disclosed and excluded from the successful coverage proof.

## Checklist

- [x] `make build && make test` passes (if Go changes) — no Go changes;
gate runner built once.
- [x] No secrets or credentials in code
- [x] Breaking changes documented — opt-in additions; unsupported
linked-worktree hook installation refuses before mutation.
2026-09-13 05:37:17 +00:00

107 lines
4.4 KiB
Bash

#!/usr/bin/env bats
setup() {
ROOT="$(cd "$BATS_TEST_DIRNAME/../.." && pwd)"
export CODEX_HOME="$BATS_TEST_TMPDIR/codex home"
}
require_codex() {
command -v codex >/dev/null 2>&1 || skip "Codex runtime required for native config editor"
}
@test "Codex project registrations resolve role templates with required config" {
python3 - "$ROOT" <<'PY'
import pathlib, sys, tomllib
root = pathlib.Path(sys.argv[1])
config = tomllib.loads((root / '.codex/config.toml').read_text())
for name in ('bulk-reader', 'code-writer'):
source = root / 'skills/agent-native/agents' / (name + '.toml')
project = root / '.codex/agents' / (name + '.toml')
assert project.resolve() == source.resolve()
assert (root / '.codex' / config['agents'][name]['config_file']).resolve() == source.resolve()
data = tomllib.loads(project.read_text())
assert data['name'] == name and data['description'] and data['developer_instructions']
assert data['model'] == 'gpt-5.6-luna'
assert data['sandbox_mode'] == ('read-only' if name == 'bulk-reader' else 'workspace-write')
PY
}
@test "personal installation copies generated roles and does not enable hooks" {
require_codex
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
cmp "$CODEX_HOME/agents/bulk-reader.toml" "$ROOT/skills-codex/agent-native/agents/bulk-reader.toml"
cmp "$CODEX_HOME/agents/code-writer.toml" "$ROOT/skills-codex/agent-native/agents/code-writer.toml"
[ ! -e "$CODEX_HOME/hooks.json" ]
[ -f "$CODEX_HOME/config.toml" ]
python3 - "$CODEX_HOME/config.toml" <<'PY'
import sys,tomllib
with open(sys.argv[1], "rb") as f: cfg=tomllib.load(f)
assert set(cfg["agents"]) == {"bulk-reader", "code-writer"}
for role in cfg["agents"]:
assert cfg["agents"][role]["config_file"].endswith("/"+role+".toml")
PY
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
[ "$(find "$CODEX_HOME" -name '*.bak.*' | wc -l | tr -d ' ')" -eq 0 ]
}
@test "changed role backups are retained and symlink source is preserved" {
require_codex
mkdir -p "$CODEX_HOME/agents"
printf 'original\n' > "$BATS_TEST_TMPDIR/original.toml"
ln -s "$BATS_TEST_TMPDIR/original.toml" "$CODEX_HOME/agents/bulk-reader.toml"
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
[ "$(cat "$BATS_TEST_TMPDIR/original.toml")" = original ]
[ ! -L "$CODEX_HOME/agents/bulk-reader.toml" ]
[ "$(cat "$CODEX_HOME"/agents/bulk-reader.toml.bak.*)" = original ]
printf 'second\n' > "$CODEX_HOME/agents/bulk-reader.toml"
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
[ "$(find "$CODEX_HOME/agents" -name '*.bak.*' | wc -l | tr -d ' ')" -eq 2 ]
}
@test "project installation targets the caller project" {
require_codex
mkdir -p "$BATS_TEST_TMPDIR/project"
cd "$BATS_TEST_TMPDIR/project"
run bash "$ROOT/scripts/install-codex-context-agents.sh" --project
[ "$status" -eq 0 ]
[ -f .codex/agents/code-writer.toml ]
[ ! -e "$CODEX_HOME/agents" ]
}
@test "native config registration preserves unrelated TOML and is idempotent" {
require_codex
mkdir -p "$CODEX_HOME"
printf 'model = "gpt-6-astra"\n[agents.other]\ndescription = "existing"\n' > "$CODEX_HOME/config.toml"
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
python3 - "$CODEX_HOME/config.toml" <<'PY'
import sys,tomllib
with open(sys.argv[1], "rb") as f: cfg=tomllib.load(f)
assert cfg["model"] == "gpt-6-astra"
assert cfg["agents"]["other"]["description"] == "existing"
assert set(cfg["agents"]) == {"other", "bulk-reader", "code-writer"}
PY
[ "$(find "$CODEX_HOME" -name 'config.toml.bak.*' | wc -l | tr -d ' ')" -eq 1 ]
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -eq 0 ]
[ "$(find "$CODEX_HOME" -name 'config.toml.bak.*' | wc -l | tr -d ' ')" -eq 1 ]
}
@test "malformed existing config fails before publishing roles or modifying settings" {
require_codex
mkdir -p "$CODEX_HOME/agents"
printf '[invalid TOML\n' > "$CODEX_HOME/config.toml"
printf 'existing role\n' > "$CODEX_HOME/agents/bulk-reader.toml"
cp "$CODEX_HOME/config.toml" "$BATS_TEST_TMPDIR/original-config"
run bash "$ROOT/scripts/install-codex-context-agents.sh"
[ "$status" -ne 0 ]
cmp "$CODEX_HOME/config.toml" "$BATS_TEST_TMPDIR/original-config"
[ "$(cat "$CODEX_HOME/agents/bulk-reader.toml")" = 'existing role' ]
[ ! -e "$CODEX_HOME/agents/code-writer.toml" ]
[ "$(find "$CODEX_HOME" -name '*.bak.*' | wc -l | tr -d ' ')" -eq 0 ]
}