> **Residues closed on the caller's merge instruction** (`499d916a6`): the round-2 findings were the same failure shape — round-1 repairs patched cited lines instead of sweeping the class — so this commit sweeps each file whole: every remaining SATURATED-row-append site in skill-eval now routes to RUNBOOK retirement, the human-only-skills *description* is runtime-conditional, premortem's "(MEASURED)" label is gone, SKILL-API's context table carries all 25 rows and the enforcement table gains `disable-model-invocation`, and the fixture-identity claim is stated precisely (probe id, honesty note, and control arm are the only differing fields — as the acceptance permits). Post-sweep: validators, full Go suite, 68/68 gates, goldens + headroom bats green, projections current, gemini in sync. Merging per Bo's instruction. ## What Train 1 of the accepted [instrument-panel roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent landed at `986a4feaf`): the measurement substrate, the skill-context diet, and the retrieval-eval contract. Three worktree-isolated lanes, each independently validated by a fresh context, plus one integration commit. 103 files, +5,510/−76. **L1 — measurement substrate** (`instrument/measurement-substrate`) - Gate `skill.probe-headroom` (advisory, Fast|Full): answers the question `skill.probe-coverage` cannot — not "does a probe result exist" but "could one have existed at all". The rule, ported to Go (`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin check script — the witness-crosscheck pattern, **no new `ao` root command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels ⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it; treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed fixture pairs read `INERT` to everything else in the repo; the failing separation test predates the implementation, and a bats negative-control swaps fixture bytes and asserts the gate flips. - **First reading on real data: 7 of 11 historical probe groups are SATURATED** — including both `validate-not-proven` runs. Those INERT rows were never honest nulls; they were void. The 0/12 ledger number now argues itself. - Declared denominator for probe-coverage: `scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry without an argument, stale slug, or duplicate ⇒ exit 2). One entry (`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12 → 0/12: alias left, `one-way-door` entered) — the gain is a declared number, not a better-looking one. - Re-landed from the recovered clean-room commit (`9872483bd`), re-validated against *current* main: `skill-eval` (defers saturation to the gate id; its shell scripts dropped, not shipped — ratchet intent), `route`, `one-way-door`, premortem reversibility check, council `caller_challenge` (schema + validator, per the agent-core boundary that the panel may challenge, never overrule). **L4 — context diet** (`instrument/context-diet`) - `disable-model-invocation: true` on 4 human-only skills (key verified verbatim against Anthropic's docs). The plan guessed 35 candidates; the graph said otherwise — 23 carry `user-invocable: true`, and 19 of those are excluded on cited evidence (rpi consumes anti-ceremony/implement/plan/validate; workflow scripts reach others; `goals` is a live migration tombstone). The exclusion evidence is retained in the lane report. - One router skill (`human-only-skills`) — the single always-loaded description that replaces four; it hints, never fires. - `.out-of-scope/` formalized with this week's three refusals (checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B as the measurement unit), each citing its evidence. - Deterministic proof, no model eval: before/after bytes of always-loaded description load reported in the lane summary. **L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane verdict PASS 10/10) - `AGENTS.md` federated row now names **ee (eidetic-engine)** as a concrete caller-selected memory system — consume, never build; symlink intact. - `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens + `scripts/check-routing-probe-goldens.sh` graded against `ao skills find`, wired as an **advisory** nightly job. Zero goldens is a failing state — no new zero-denominator green. - **The instrument caught a real miss on day one — and its own prescription fixed it.** Golden `rq-04` expects `validate` for "judge whether this finished change is actually proven before I merge it"; at authoring, `ao skills find` ranked the *forbidden* `premortem` first and `validate` nowhere in six natural phrasings. The pointer-wording-first repair (validate's description gained the caller's own words: finished, proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the golden pins the repair — a description regression reopens it. ## Integration `regen-all.sh` once over the merged lanes (catalog 52 → 56, four new codex twins, mesh, router, manifests); `skills/route/SKILL.md` catalog/router links became prose repo-root references (the projected twin cannot resolve `../catalog.json` — this was both the portable-conformance failure and the sole broken doc link); `codex-portable-conformance.bats` pin 52 → 56. ## Evidence - `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 · `ao gate check --full` **68/68** · four skill validators PASS · probe-headroom / routing-goldens / probe-coverage bats PASS · `regen-all.sh --check` all current. - Per-lane fresh validators re-ran every suite on detached content; L5 PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved for integration (their remaining acceptance observed green), settled above. Cross-family (Codex) review of the integrated diff recorded in the session report. - Two disclosed scope stretches accepted at integration: a one-line `.gitignore` entry mirroring the witness-crosscheck precedent, and the probe LEDGER.md fact-correction L1's own change made necessary (noted for Train 2's L2, which owns that file next). ## Cross-family review (Codex, fresh context) Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the control arm; the goldens grader was red where the plan's acceptance says green) and eight majors (contract contradictions in the re-landed skills, a converter-substitution false claim in the codex router twin, two overreaching `.out-of-scope` entries, stale SKILL-API counts). All repaired in one bounded round (`db68935a3`): fixtures now byte-identical outside the control arm, the routing miss actually fixed rather than tolerated, every cited contradiction reconciled at the source and re-projected. Post-repair: full Go suite exit 0, `gate check --full` 68/68, all validators and probe/goldens bats green, projections current, gemini byte-identity restored. Focused re-check verdict recorded in the session report. ## Follow-ups (Train 2, already planned) Seeded-defect probes for the judgment spine (every ledger row citing a passing headroom pre-screen) and the gate-hardening pair (`Gate-Loosen-Reason` tightening ratchet; mechanical grounding-validation over evidence docs). Plus, surfaced by this train: a latent `valid_keys`/schema divergence in `validate-skill-schema.sh` (two keys the schema defines are absent from the script's allowlist — pre-existing).
13 KiB
Codex Skill API Contract
Source of truth for what the Codex runtime actually supports. All converter output and validation must conform to this contract. Orientation contract:
AGENTS.md.
Official docs:
SKILL.md Frontmatter
Every generated AgentOps Codex projection emits only the two fields Codex uses for discovery:
---
name: skill-name
description: 'Explain when this skill triggers and when it does not.'
---
The portable Agent Skills specification requires name and description
and also permits license, compatibility, string-to-string metadata,
and the experimental space-separated allowed-tools field. The AgentOps
generator does not currently emit those optional fields. AgentOps-only fields
such as skill_api_version, context, model, user-invocable, and
output_contract must be stripped from Codex output.
Optional: agents/openai.yaml
Codex skills may include agents/openai.yaml for display metadata and policy:
interface:
display_name: "User-facing name"
short_description: "Brief description"
icon_small: "./assets/small-logo.svg"
icon_large: "./assets/large-logo.png"
brand_color: "#3B82F6"
default_prompt: "Optional surrounding prompt"
policy:
allow_implicit_invocation: false
dependencies:
tools:
- type: "mcp"
value: "toolName"
description: "Tool description"
transport: "streamable_http"
url: "https://example.com"
| Field | Purpose |
|---|---|
interface.display_name |
User-visible name in Codex UI |
interface.short_description |
Brief description for skill browser |
policy.allow_implicit_invocation |
false prevents auto-activation (explicit $skill only) |
dependencies.tools |
MCP server dependencies |
Skill Discovery And Installation
The portable Codex skill root is .agents/skills/ at repository or parent
scope and $HOME/.agents/skills/ at user scope. Current Codex desktop installs
may also index $HOME/.codex/skills/; on Bo's development fleet both user roots
are symlinks to this repository's canonical skills/ tree.
| Scope | Path | Use Case |
|---|---|---|
| Repo (nearest) | .agents/skills/ from CWD |
Folder-specific workflows |
| Repo (parent) | ../.agents/skills/ |
Nested repo organization |
| Repo (root) | $REPO_ROOT/.agents/skills/ |
Organization-wide skills |
| User | $HOME/.agents/skills/ |
Portable personal skill collection |
| Host compatibility | $HOME/.codex/skills/ |
Codex-host symlink root when configured |
| Admin | /etc/codex/skills/ |
System-wide defaults |
| System | Bundled with Codex | Built-in skills |
Development symlinks may point directly at skills/<name>/; checked-in
skills-codex/ packages are the portable Agent Skills release projection.
Canonical skills/ frontmatter is intentionally host-extended and is evaluated
against the AgentOps profile, not mislabeled as portable. The release gate
scripts/validate-codex-api-conformance.sh validates every projected package
against the portable field, type, identity, body, and relative-resource-link
contract before applying Codex-specific checks. Plugin caches are neither
source nor an installation target and may be deleted without affecting the
canonical repository skills.
Skill Invocation
| Method | Syntax | Description |
|---|---|---|
| Explicit | $skill-name or /skills menu |
User directly requests the skill |
| Implicit | Automatic | Codex matches task to skill description |
Skills are loaded via progressive disclosure: metadata first (name,
description), full SKILL.md only when activated. Because the description is
the implicit-routing surface, a projection must preserve the source's complete
Triggers: clause while keeping its introductory prose within the catalog
budget; a generic summary or dropped trigger is a behavioral defect, even when
the body remains byte-complete.
Multi-Agent (Sub-Agents)
Current Codex releases enable subagent workflows by default. Codex only spawns subagents when the operator or parent agent explicitly asks it to do so. Subagents inherit the parent sandbox policy and live runtime overrides.
Agent Roles
Codex ships with these built-in roles:
| Role | Purpose |
|---|---|
default |
General-purpose fallback |
worker |
Execution-focused implementation |
explorer |
Read-heavy codebase exploration |
Custom agents live as standalone TOML files under $HOME/.codex/agents/ for
personal agents or .codex/agents/ for project-scoped agents. Each custom
agent file must define name, description, and developer_instructions.
Optional fields such as nickname_candidates, model,
model_reasoning_effort, sandbox_mode, mcp_servers, and skills.config
inherit from the parent session when omitted.
Global subagent limits stay in the [agents] section of config.toml:
[agents]
max_threads = 6
max_depth = 1
job_max_runtime_seconds = 1800
[agents.reviewer]
description = "Code review specialist"
config_file = "codex-reviewer.toml"
agents.max_threads defaults to 6; agents.max_depth defaults to 1,
which allows direct child agents but prevents deeper recursive fan-out.
Batch Processing
spawn_agents_on_csv processes batches of similar tasks:
| Parameter | Description |
|---|---|
csv_path |
Source CSV file |
instruction |
Worker prompt template with {column_name} placeholders |
id_column |
Stable identifiers |
output_schema |
Fixed JSON structure for worker results |
output_csv_path |
Destination CSV containing row metadata and results |
max_concurrency |
Parallel worker limit |
max_runtime_seconds |
Worker timeout |
Workers call report_agent_job_result exactly once.
sqlite_home controls where Codex stores the SQLite-backed state used for
agent jobs and exported results.
Codex Built-in Tools
Tools available inside a Codex agent session:
| Tool | Purpose |
|---|---|
read_file |
Read file contents |
list_dir |
List directory contents |
glob_file_search |
Find files by pattern |
apply_patch |
Apply file edits (diff-based) |
rg |
Ripgrep search |
git |
Git operations |
| Shell/terminal tool | Shell command execution |
spawn_agent |
Create a focused sub-agent |
send_input |
Send follow-up input to a sub-agent |
wait_agent |
Wait for one or more sub-agents |
close_agent |
Stop a stuck or no-longer-needed sub-agent |
Claude → Codex Primitive Mapping
| Claude Code | Codex Equivalent | Converter Action |
|---|---|---|
Read tool |
read_file |
Map |
Edit tool |
apply_patch |
Map |
Grep tool |
rg |
Map |
Glob tool |
glob_file_search |
Map |
Agent(subagent_type="Explore") |
Explorer agent role | Map |
Skill(skill="name") |
$name invocation |
Map |
TaskCreate / TaskList / TaskUpdate |
No equivalent (todo_write/update_plan not available — empirically verified) |
Strip |
TeamCreate / TeamDelete |
No equivalent | Strip |
SendMessage |
send_input for brief follow-up only |
Rewrite or strip |
EnterPlanMode / ExitPlanMode |
No equivalent | Strip |
EnterWorktree |
No equivalent | Strip |
context.window |
No equivalent | Strip from frontmatter |
context.sections.exclude |
No equivalent | Strip from frontmatter |
context.intel_scope |
Deprecated — ignored, no reader | Does not exist |
disable-model-invocation |
No equivalent — Codex has no model-invocation switch and always advertises a $name skill |
Strip from frontmatter |
Skills referencing these primitives produce broken instructions in Codex.
disable-model-invocation is a Claude Code context-budget control, not a
behavioral instruction, so stripping it loses nothing a Codex reader could act
on. The consequence is intended and asymmetric: a human-only skill is invisible
to Claude's implicit routing but stays $name-invocable under Codex. Requirement
1 below already strips it — the row exists so the divergence is declared rather
than discovered.
Converter Requirements
When generating Codex skills from source skills:
- Strip all non-Codex frontmatter — emit only
name+description, compacting introductory prose while preserving the complete sourceTriggers:clause as the activation catalog signal - Map Claude tools to Codex tools — Read→read_file, Edit→apply_patch, Grep→rg, Glob→glob_file_search
- Rewrite
Skill(skill="X")to$X— Codex uses dollar-prefix invocation - Strip ALL task/team primitives — TaskCreate, TaskList, TeamCreate, SendMessage (none have working Codex equivalents as direct tool calls —
todo_write/update_planempirically unavailable, andsend_inputis follow-up-only) - Fix paths — avoid host-specific installed-skill paths; use repository-relative links in shipped instructions
- Rewrite reference files —
.mdfiles in references/ pass throughcodex_rewrite_text()during copy - Preserve skill body — the SKILL.md body (instructions) is the skill's value; keep it functional
Validation Criteria
A Codex-conformant skill must:
- Have frontmatter with only
nameanddescription - Contain no Claude-only primitive names (TaskCreate, TeamCreate, SendMessage, etc.)
- Contain no Claude-specific paths and no dependency on one host's installed-skill root
- Have valid
agents/openai.yamlif present - Not reference non-existent Codex features (context controls, plan mode, etc.)
Generated inventories
skills/<name>/SKILL.md metadata owns the skill inventory and dependency graph.
Run scripts/regen-all.sh after changing it. The CLI command surface is a
separate executable projection generated from Cobra; no handwritten CLI-to-skill
map is maintained.
Codex Skill Maintenance
Codex is a first-class runtime in this repo.
skills/<name>/SKILL.mdis the canonical behavior contract.skills-codex-overrides/<name>/is the Codex-specific tailoring layer.skills-codex-overrides/catalog.jsonis the machine-readable treatment map for the full catalog.skills-codex/<name>/is the generated checked-in Codex runtime artifact.
Editing an EXISTING parity skill regenerates its Codex twin — not just hashes.
make regen-all / scripts/codex-sync.sh refresh parity-only twins from
skills/<name>/ whenever their generated body, prompt, or mirrored references
drift. scripts/regen-codex-hashes.sh remains the bookkeeping step after
content is current. Manual edits under skills-codex/<name>/ are reserved for
bespoke skills or deliberate Codex-only divergence recorded in
skills-codex-overrides/catalog.json; otherwise fix the source skill or the
codex-sync transform/template and regenerate.
The current catalog has no bespoke or pointer twins. Every live Codex package is
a generated parity projection of skills/<name>/SKILL.md plus its linked local
files. scripts/codex-sync.sh owns those packages; manual edits under
skills-codex/<name>/ are drift and will be overwritten.
skills-codex-overrides/catalog.json remains the explicit treatment registry.
If a future runtime-specific implementation is genuinely necessary, declare it
there before editing a twin and add focused parity tests in the same change.
Do not create an undocumented exception or a second skill inventory.
When a skill change affects Codex behavior, phrasing, orchestration, or UX:
-
Update the source skill under
skills/when the shared contract changes. -
For parity-only skills, update source or the codex-sync transform/template and regenerate. Update
skills-codex/<name>/SKILL.mddirectly only when the Codex runtime copy is bespoke, or updateskills-codex-overrides/<name>/when the Codex experience should differ from Claude.- Prompt/operator-layer changes belong in
skills-codex-overrides/<name>/prompt.md. - Durable Codex-only body rewrites belong in
skills-codex-overrides/<name>/SKILL.md.
- Prompt/operator-layer changes belong in
-
Run the semantic audit if the checked-in Codex body looks suspicious:
bash scripts/audit-codex-parity.sh # or target one skill bash scripts/audit-codex-parity.sh --skill <name> -
Validate the checked-in Codex artifacts:
bash scripts/audit-codex-parity.sh bash scripts/validate-codex-override-coverage.sh bash scripts/validate-codex-generated-artifacts.sh --scope worktree bash scripts/validate-headless-runtime-skills.sh python3 scripts/check-cathedral-cut-conformance.py
Think of skills/ as the shared contract, skills-codex-overrides/ as the durable Codex-only tailoring layer, and skills-codex/ as the checked-in Codex artifact shipped to users.