Files
boshu2__agentops/docs/contracts/codex-skill-api.md
T
Bo 8cdcb5a903 Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`):
the round-2 findings were the same failure shape — round-1 repairs
patched cited lines instead of sweeping the class — so this commit
sweeps each file whole: every remaining SATURATED-row-append site in
skill-eval now routes to RUNBOOK retirement, the human-only-skills
*description* is runtime-conditional, premortem's "(MEASURED)" label is
gone, SKILL-API's context table carries all 25 rows and the enforcement
table gains `disable-model-invocation`, and the fixture-identity claim
is stated precisely (probe id, honesty note, and control arm are the
only differing fields — as the acceptance permits). Post-sweep:
validators, full Go suite, 68/68 gates, goldens + headroom bats green,
projections current, gemini in sync. Merging per Bo's instruction.

## What

Train 1 of the accepted [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent
landed at `986a4feaf`): the measurement substrate, the skill-context
diet, and the retrieval-eval contract. Three worktree-isolated lanes,
each independently validated by a fresh context, plus one integration
commit. 103 files, +5,510/−76.

**L1 — measurement substrate** (`instrument/measurement-substrate`)
- Gate `skill.probe-headroom` (advisory, Fast|Full): answers the
question `skill.probe-coverage` cannot — not "does a probe result exist"
but "could one have existed at all". The rule, ported to Go
(`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin
check script — the witness-crosscheck pattern, **no new `ao` root
command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels
⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it;
treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed
fixture pairs read `INERT` to everything else in the repo; the failing
separation test predates the implementation, and a bats negative-control
swaps fixture bytes and asserts the gate flips.
- **First reading on real data: 7 of 11 historical probe groups are
SATURATED** — including both `validate-not-proven` runs. Those INERT
rows were never honest nulls; they were void. The 0/12 ledger number now
argues itself.
- Declared denominator for probe-coverage:
`scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry
without an argument, stale slug, or duplicate ⇒ exit 2). One entry
(`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12
→ 0/12: alias left, `one-way-door` entered) — the gain is a declared
number, not a better-looking one.
- Re-landed from the recovered clean-room commit (`9872483bd`),
re-validated against *current* main: `skill-eval` (defers saturation to
the gate id; its shell scripts dropped, not shipped — ratchet intent),
`route`, `one-way-door`, premortem reversibility check, council
`caller_challenge` (schema + validator, per the agent-core boundary that
the panel may challenge, never overrule).

**L4 — context diet** (`instrument/context-diet`)
- `disable-model-invocation: true` on 4 human-only skills (key verified
verbatim against Anthropic's docs). The plan guessed 35 candidates; the
graph said otherwise — 23 carry `user-invocable: true`, and 19 of those
are excluded on cited evidence (rpi consumes
anti-ceremony/implement/plan/validate; workflow scripts reach others;
`goals` is a live migration tombstone). The exclusion evidence is
retained in the lane report.
- One router skill (`human-only-skills`) — the single always-loaded
description that replaces four; it hints, never fires.
- `.out-of-scope/` formalized with this week's three refusals
(checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B
as the measurement unit), each citing its evidence.
- Deterministic proof, no model eval: before/after bytes of
always-loaded description load reported in the lane summary.

**L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane
verdict PASS 10/10)
- `AGENTS.md` federated row now names **ee (eidetic-engine)** as a
concrete caller-selected memory system — consume, never build; symlink
intact.
- `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens
+ `scripts/check-routing-probe-goldens.sh` graded against `ao skills
find`, wired as an **advisory** nightly job. Zero goldens is a failing
state — no new zero-denominator green.
- **The instrument caught a real miss on day one — and its own
prescription fixed it.** Golden `rq-04` expects `validate` for "judge
whether this finished change is actually proven before I merge it"; at
authoring, `ao skills find` ranked the *forbidden* `premortem` first and
`validate` nowhere in six natural phrasings. The pointer-wording-first
repair (validate's description gained the caller's own words: finished,
proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the
golden pins the repair — a description regression reopens it.

## Integration

`regen-all.sh` once over the merged lanes (catalog 52 → 56, four new
codex twins, mesh, router, manifests); `skills/route/SKILL.md`
catalog/router links became prose repo-root references (the projected
twin cannot resolve `../catalog.json` — this was both the
portable-conformance failure and the sole broken doc link);
`codex-portable-conformance.bats` pin 52 → 56.

## Evidence

- `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 ·
`ao gate check --full` **68/68** · four skill validators PASS ·
probe-headroom / routing-goldens / probe-coverage bats PASS ·
`regen-all.sh --check` all current.
- Per-lane fresh validators re-ran every suite on detached content; L5
PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved
for integration (their remaining acceptance observed green), settled
above. Cross-family (Codex) review of the integrated diff recorded in
the session report.
- Two disclosed scope stretches accepted at integration: a one-line
`.gitignore` entry mirroring the witness-crosscheck precedent, and the
probe LEDGER.md fact-correction L1's own change made necessary (noted
for Train 2's L2, which owns that file next).

## Cross-family review (Codex, fresh context)

Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the
control arm; the goldens grader was red where the plan's acceptance says
green) and eight majors (contract contradictions in the re-landed
skills, a converter-substitution false claim in the codex router twin,
two overreaching `.out-of-scope` entries, stale SKILL-API counts). All
repaired in one bounded round (`db68935a3`): fixtures now byte-identical
outside the control arm, the routing miss actually fixed rather than
tolerated, every cited contradiction reconciled at the source and
re-projected. Post-repair: full Go suite exit 0, `gate check --full`
68/68, all validators and probe/goldens bats green, projections current,
gemini byte-identity restored. Focused re-check verdict recorded in the
session report.

## Follow-ups (Train 2, already planned)

Seeded-defect probes for the judgment spine (every ledger row citing a
passing headroom pre-screen) and the gate-hardening pair
(`Gate-Loosen-Reason` tightening ratchet; mechanical
grounding-validation over evidence docs). Plus, surfaced by this train:
a latent `valid_keys`/schema divergence in `validate-skill-schema.sh`
(two keys the schema defines are absent from the script's allowlist —
pre-existing).
2026-08-26 23:16:51 +00:00

13 KiB

Codex Skill API Contract

Source of truth for what the Codex runtime actually supports. All converter output and validation must conform to this contract. Orientation contract: AGENTS.md.

Official docs:


SKILL.md Frontmatter

Every generated AgentOps Codex projection emits only the two fields Codex uses for discovery:

---
name: skill-name
description: 'Explain when this skill triggers and when it does not.'
---

The portable Agent Skills specification requires name and description and also permits license, compatibility, string-to-string metadata, and the experimental space-separated allowed-tools field. The AgentOps generator does not currently emit those optional fields. AgentOps-only fields such as skill_api_version, context, model, user-invocable, and output_contract must be stripped from Codex output.


Optional: agents/openai.yaml

Codex skills may include agents/openai.yaml for display metadata and policy:

interface:
  display_name: "User-facing name"
  short_description: "Brief description"
  icon_small: "./assets/small-logo.svg"
  icon_large: "./assets/large-logo.png"
  brand_color: "#3B82F6"
  default_prompt: "Optional surrounding prompt"

policy:
  allow_implicit_invocation: false

dependencies:
  tools:
    - type: "mcp"
      value: "toolName"
      description: "Tool description"
      transport: "streamable_http"
      url: "https://example.com"
Field Purpose
interface.display_name User-visible name in Codex UI
interface.short_description Brief description for skill browser
policy.allow_implicit_invocation false prevents auto-activation (explicit $skill only)
dependencies.tools MCP server dependencies

Skill Discovery And Installation

The portable Codex skill root is .agents/skills/ at repository or parent scope and $HOME/.agents/skills/ at user scope. Current Codex desktop installs may also index $HOME/.codex/skills/; on Bo's development fleet both user roots are symlinks to this repository's canonical skills/ tree.

Scope Path Use Case
Repo (nearest) .agents/skills/ from CWD Folder-specific workflows
Repo (parent) ../.agents/skills/ Nested repo organization
Repo (root) $REPO_ROOT/.agents/skills/ Organization-wide skills
User $HOME/.agents/skills/ Portable personal skill collection
Host compatibility $HOME/.codex/skills/ Codex-host symlink root when configured
Admin /etc/codex/skills/ System-wide defaults
System Bundled with Codex Built-in skills

Development symlinks may point directly at skills/<name>/; checked-in skills-codex/ packages are the portable Agent Skills release projection. Canonical skills/ frontmatter is intentionally host-extended and is evaluated against the AgentOps profile, not mislabeled as portable. The release gate scripts/validate-codex-api-conformance.sh validates every projected package against the portable field, type, identity, body, and relative-resource-link contract before applying Codex-specific checks. Plugin caches are neither source nor an installation target and may be deleted without affecting the canonical repository skills.


Skill Invocation

Method Syntax Description
Explicit $skill-name or /skills menu User directly requests the skill
Implicit Automatic Codex matches task to skill description

Skills are loaded via progressive disclosure: metadata first (name, description), full SKILL.md only when activated. Because the description is the implicit-routing surface, a projection must preserve the source's complete Triggers: clause while keeping its introductory prose within the catalog budget; a generic summary or dropped trigger is a behavioral defect, even when the body remains byte-complete.


Multi-Agent (Sub-Agents)

Current Codex releases enable subagent workflows by default. Codex only spawns subagents when the operator or parent agent explicitly asks it to do so. Subagents inherit the parent sandbox policy and live runtime overrides.

Agent Roles

Codex ships with these built-in roles:

Role Purpose
default General-purpose fallback
worker Execution-focused implementation
explorer Read-heavy codebase exploration

Custom agents live as standalone TOML files under $HOME/.codex/agents/ for personal agents or .codex/agents/ for project-scoped agents. Each custom agent file must define name, description, and developer_instructions. Optional fields such as nickname_candidates, model, model_reasoning_effort, sandbox_mode, mcp_servers, and skills.config inherit from the parent session when omitted.

Global subagent limits stay in the [agents] section of config.toml:

[agents]
max_threads = 6
max_depth = 1
job_max_runtime_seconds = 1800

[agents.reviewer]
description = "Code review specialist"
config_file = "codex-reviewer.toml"

agents.max_threads defaults to 6; agents.max_depth defaults to 1, which allows direct child agents but prevents deeper recursive fan-out.

Batch Processing

spawn_agents_on_csv processes batches of similar tasks:

Parameter Description
csv_path Source CSV file
instruction Worker prompt template with {column_name} placeholders
id_column Stable identifiers
output_schema Fixed JSON structure for worker results
output_csv_path Destination CSV containing row metadata and results
max_concurrency Parallel worker limit
max_runtime_seconds Worker timeout

Workers call report_agent_job_result exactly once.

sqlite_home controls where Codex stores the SQLite-backed state used for agent jobs and exported results.

Codex Built-in Tools

Tools available inside a Codex agent session:

Tool Purpose
read_file Read file contents
list_dir List directory contents
glob_file_search Find files by pattern
apply_patch Apply file edits (diff-based)
rg Ripgrep search
git Git operations
Shell/terminal tool Shell command execution
spawn_agent Create a focused sub-agent
send_input Send follow-up input to a sub-agent
wait_agent Wait for one or more sub-agents
close_agent Stop a stuck or no-longer-needed sub-agent

Claude → Codex Primitive Mapping

Claude Code Codex Equivalent Converter Action
Read tool read_file Map
Edit tool apply_patch Map
Grep tool rg Map
Glob tool glob_file_search Map
Agent(subagent_type="Explore") Explorer agent role Map
Skill(skill="name") $name invocation Map
TaskCreate / TaskList / TaskUpdate No equivalent (todo_write/update_plan not available — empirically verified) Strip
TeamCreate / TeamDelete No equivalent Strip
SendMessage send_input for brief follow-up only Rewrite or strip
EnterPlanMode / ExitPlanMode No equivalent Strip
EnterWorktree No equivalent Strip
context.window No equivalent Strip from frontmatter
context.sections.exclude No equivalent Strip from frontmatter
context.intel_scope Deprecated — ignored, no reader Does not exist
disable-model-invocation No equivalent — Codex has no model-invocation switch and always advertises a $name skill Strip from frontmatter

Skills referencing these primitives produce broken instructions in Codex.

disable-model-invocation is a Claude Code context-budget control, not a behavioral instruction, so stripping it loses nothing a Codex reader could act on. The consequence is intended and asymmetric: a human-only skill is invisible to Claude's implicit routing but stays $name-invocable under Codex. Requirement 1 below already strips it — the row exists so the divergence is declared rather than discovered.


Converter Requirements

When generating Codex skills from source skills:

  1. Strip all non-Codex frontmatter — emit only name + description, compacting introductory prose while preserving the complete source Triggers: clause as the activation catalog signal
  2. Map Claude tools to Codex tools — Read→read_file, Edit→apply_patch, Grep→rg, Glob→glob_file_search
  3. Rewrite Skill(skill="X") to $X — Codex uses dollar-prefix invocation
  4. Strip ALL task/team primitives — TaskCreate, TaskList, TeamCreate, SendMessage (none have working Codex equivalents as direct tool calls — todo_write/update_plan empirically unavailable, and send_input is follow-up-only)
  5. Fix paths — avoid host-specific installed-skill paths; use repository-relative links in shipped instructions
  6. Rewrite reference files — .md files in references/ pass through codex_rewrite_text() during copy
  7. Preserve skill body — the SKILL.md body (instructions) is the skill's value; keep it functional

Validation Criteria

A Codex-conformant skill must:

  1. Have frontmatter with only name and description
  2. Contain no Claude-only primitive names (TaskCreate, TeamCreate, SendMessage, etc.)
  3. Contain no Claude-specific paths and no dependency on one host's installed-skill root
  4. Have valid agents/openai.yaml if present
  5. Not reference non-existent Codex features (context controls, plan mode, etc.)

Generated inventories

skills/<name>/SKILL.md metadata owns the skill inventory and dependency graph. Run scripts/regen-all.sh after changing it. The CLI command surface is a separate executable projection generated from Cobra; no handwritten CLI-to-skill map is maintained.


Codex Skill Maintenance

Codex is a first-class runtime in this repo.

  • skills/<name>/SKILL.md is the canonical behavior contract.
  • skills-codex-overrides/<name>/ is the Codex-specific tailoring layer.
  • skills-codex-overrides/catalog.json is the machine-readable treatment map for the full catalog.
  • skills-codex/<name>/ is the generated checked-in Codex runtime artifact.

Editing an EXISTING parity skill regenerates its Codex twin — not just hashes. make regen-all / scripts/codex-sync.sh refresh parity-only twins from skills/<name>/ whenever their generated body, prompt, or mirrored references drift. scripts/regen-codex-hashes.sh remains the bookkeeping step after content is current. Manual edits under skills-codex/<name>/ are reserved for bespoke skills or deliberate Codex-only divergence recorded in skills-codex-overrides/catalog.json; otherwise fix the source skill or the codex-sync transform/template and regenerate.

The current catalog has no bespoke or pointer twins. Every live Codex package is a generated parity projection of skills/<name>/SKILL.md plus its linked local files. scripts/codex-sync.sh owns those packages; manual edits under skills-codex/<name>/ are drift and will be overwritten.

skills-codex-overrides/catalog.json remains the explicit treatment registry. If a future runtime-specific implementation is genuinely necessary, declare it there before editing a twin and add focused parity tests in the same change. Do not create an undocumented exception or a second skill inventory.

When a skill change affects Codex behavior, phrasing, orchestration, or UX:

  1. Update the source skill under skills/ when the shared contract changes.

  2. For parity-only skills, update source or the codex-sync transform/template and regenerate. Update skills-codex/<name>/SKILL.md directly only when the Codex runtime copy is bespoke, or update skills-codex-overrides/<name>/ when the Codex experience should differ from Claude.

    • Prompt/operator-layer changes belong in skills-codex-overrides/<name>/prompt.md.
    • Durable Codex-only body rewrites belong in skills-codex-overrides/<name>/SKILL.md.
  3. Run the semantic audit if the checked-in Codex body looks suspicious:

    bash scripts/audit-codex-parity.sh
    # or target one skill
    bash scripts/audit-codex-parity.sh --skill <name>
    
  4. Validate the checked-in Codex artifacts:

    bash scripts/audit-codex-parity.sh
    bash scripts/validate-codex-override-coverage.sh
    bash scripts/validate-codex-generated-artifacts.sh --scope worktree
    bash scripts/validate-headless-runtime-skills.sh
    python3 scripts/check-cathedral-cut-conformance.py
    

Think of skills/ as the shared contract, skills-codex-overrides/ as the durable Codex-only tailoring layer, and skills-codex/ as the checked-in Codex artifact shipped to users.