feat(provenance): #513 read_scope attestation + anchor-aware finalizer promotion (#568)

* feat(provenance): #513 read_scope attestation + anchor-aware finalizer promotion

- /ars-mark-read: optional --scope {full_text,sections,abstract_only,
  toc_only,unknown} + --locator (repeatable, requires sections) + --note;
  declaration-only, absent = unknown, contradictory attestations refused,
  attestation args rejected with --unmark, batch all-or-nothing preserved
- new sidecar schema shared/contracts/passport/human_read_log.schema.json
  (version_records precedent; test-time validation, CLI dependency-light)
- Cite-Time Provenance Finalizer: read-scope-aware LOW-WARN->ok promotion
  (legacy/unknown/full_text promote; abstract_only/toc_only keep LOW-WARN
  + coverage note; sections promote only on unambiguous locator
  containment; quote anchors need full coverage); degradation is always
  retained LOW-WARN + note, never a new WARN class or hard gate
- commands/ars-mark-read.md + formatter LOW-WARN remediation in lockstep;
  #528 orchestrator content lock re-pinned
- tests: +10 attestation cases incl. sidecar-schema validation

Closes #513

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EA3EvegVqKrkM62u7k9PHF

* fix(provenance): #513 round-1 cross-model review closures (5 P1)

- presence checks use 'is not None' (--note '' no longer bypasses the
  requires-scope rule or the unmark rejection)
- locator/note length bounds enforced at write time in lockstep with the
  sidecar schema (CLI can never produce a schema-rejected ledger)
- governing attestation rides the settled latest-timestamped-event-wins
  precedence (a latest rescind wins; no resurrection of older marks)
- idempotency evidence enumeration names the attestation explicitly
- partial acknowledgment resolves to draft-visible
  LOW-WARN-PARTIAL-COVERAGE (formatter passes it as acknowledged with
  the coverage note surfaced; same severity tier, suffixes unchanged)
- tests 10→14 attestation cases; #528 lock re-pinned

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EA3EvegVqKrkM62u7k9PHF

* fix(provenance): #513 round-2 cross-model review closures (3 P1)

- LOW-WARN-PARTIAL-COVERAGE added to every base-status enumeration:
  check_v3_10_policy _BASE_STATUS, hard-gate handoff, contamination /
  policy-stamp sections (suffixes + policy_hash apply as LOW-WARN;
  freshness gate never refuses partial markers)
- citation anchor joins the idempotency/revision-loop evidence
  enumeration (anchor edits under partial coverage re-resolve markers)
- sidecar schema mirrors locators-require-sections via if/then so
  audit-time validation matches the writer contract; +1 schema test
- #528 lock re-pinned

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EA3EvegVqKrkM62u7k9PHF

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Edward Cheng-I Wu
2026-07-20 09:48:02 +08:00
committed by GitHub
parent 7f878f7ad2
commit f5402b114d
11 changed files with 530 additions and 13 deletions
+2
View File
File diff suppressed because one or more lines are too long
+2 -2
View File
@@ -346,7 +346,7 @@ Before emitting any final converted artifact (LaTeX / DOCX / PDF), scan the inpu
1. A literal `[UNVERIFIED CITATION — NO ORIGINAL]` marker (HIGH-WARN; v3.7.1).
2. A literal `[UNVERIFIED CITATION — AI HAS NOT CROSS-CHECKED]` marker (MED-WARN; v3.7.1).
3. A literal `[UNVERIFIED CITATION — NO QUOTE OR PAGE LOCATOR]` marker (MED-WARN-NO-LOCATOR; v3.7.3).
4. Any `<!--ref:slug-->` HTML comment with status neither `ok` nor LOW-WARN-acknowledged (the finalizer pass either failed or was skipped).
4. Any `<!--ref:slug-->` HTML comment with status neither `ok`, LOW-WARN-acknowledged, nor `LOW-WARN-PARTIAL-COVERAGE` (the finalizer pass either failed or was skipped). `LOW-WARN-PARTIAL-COVERAGE` (#513) is the acknowledged-partial state — the user attested a read scope that does not cover this citation's anchor — and passes the gate as an acknowledged LOW-WARN variant, with the coverage note surfaced in `provenance_summary.md`, never refused.
5. **Any `<!--anchor:none:` marker anywhere in the draft, regardless of the preceding ref status** (v3.7.3 codex round-8 F20 closure). A stale or skipped finalizer pass can leave `<!--ref:slug ok--><!--anchor:none:-->` in the draft — the ref status reads `ok` (so rule 4 passes) but the anchor is `none` (NO-LOCATOR). Since v3.7.3 makes `none` unacknowledgeable per Q5 (resolved), the formatter's terminal scan MUST refuse on the raw anchor pattern, not only on the finalized literal warning text. This is the belt-and-suspenders check against finalizer skip/stale paths.
6. A literal `[HIGH-WARN-CLAIM-NOT-SUPPORTED]` annotation (v3.8 §3.6 8-row matrix; UNSUPPORTED + source-level defect_stage). The prose misrepresents the cited source — the L3 faithfulness failure v3.8 exists to catch. Mirrors v3.7.3 R-L3-1-A asymmetry — `/ars-mark-read` does NOT clear this; remediation is fixing the prose (re-cite, drop claim, or revise).
7. A literal `[HIGH-WARN-NEGATIVE-CONSTRAINT-VIOLATION` annotation (v3.8 §3.6; UNSUPPORTED + negative_constraint_violation). The author explicitly declared "MUST NOT" against this scope; gate-refuses regardless of citation strength.
@@ -362,7 +362,7 @@ When refusing, surface the unresolved markers to the user with their per-section
- HIGH-WARN (v3.7.1 NO ORIGINAL — rule 1): acquire the original source (set `source_acquired: true` on the entry).
- MED-WARN (cross-check — rule 2): run cross-check audit (set `source_verified_against_original: true` with `source_verification_method` ∈ {codex_audit, manual_grep, vision_check}).
- MED-WARN-NO-LOCATOR (rule 3): re-emit the citation with a `<!--anchor:<kind>:<value>-->` where `<kind>``none`. This is the ONLY remediation path. `/ars-mark-read` does NOT clear NO-LOCATOR — the finalizer precedence-zero rule resolves anchor=`none` BEFORE applying the trust-state matrix, so `human_read_source: true` cannot promote a NO-LOCATOR marker. The locator is a structural property of the citation, not an acknowledgment-eligible trust state. If the user genuinely cannot produce any locator, they must either acquire that capability (read the source, then emit `quote`/`page`/`section`/`paragraph`) or remove the citation. v3.7.3 codex review P2-2 closure.
- LOW-WARN (rule 4): run `/ars-mark-read <slug>` to acknowledge.
- LOW-WARN (rule 4): run `/ars-mark-read <slug>` to acknowledge — optionally with a #513 read-scope attestation (`--scope`, `--locator`, `--note`). A partial-coverage attestation (`abstract_only`/`toc_only`, or `sections` whose locators do not cover this citation's anchor) resolves the marker to `LOW-WARN-PARTIAL-COVERAGE` — an acknowledged state that passes the gate with the coverage note surfaced — instead of promoting to `ok`; read the relevant part of the source (or attest `full_text`) for full promotion.
- v3.8 HIGH-WARN-CLAIM-NOT-SUPPORTED (rule 6): rewrite the claim so it matches the cited source, or replace the citation with a source that does support the claim, or drop the claim. `/ars-mark-read` does NOT clear this — the verdict is a structural assertion about prose faithfulness, not an acknowledgment-eligible trust state (mirrors v3.7.3 R-L3-1-A asymmetry). v3.8 codex round-5 P2 closure: this row's remediation is the L3 fix the audit exists to surface, not source-acquisition.
- v3.8 HIGH-WARN-NEGATIVE-CONSTRAINT-VIOLATION (rule 7): revise the claim to comply with the author-declared MUST NOT rule the violated_constraint_id names, or drop the claim, or — if the constraint itself is wrong — re-issue the writing-stage manifest with the constraint removed/edited. `/ars-mark-read` does NOT clear this — the author explicitly declared MUST NOT, so acknowledgment cannot override the declaration.
- v3.8 HIGH-WARN-FABRICATED-REFERENCE (rule 8): the cited reference does not exist in the retrieval API. Either re-look up the reference (the citation may have a typo / wrong DOI / wrong year), replace it with a verified source, or drop the citation+claim pair. `/ars-mark-read` does NOT clear this — fabrication is the L3-1 failure mode v3.8 exists to surface.
@@ -706,16 +706,18 @@ When `academic-pipeline` mode is active, the orchestrator runs the **Cite-Time P
**Idempotency:** the finalizer pass is idempotent on the join of `(literature_corpus[]` row, read-log row`)` for each slug — re-running on a resolved marker with byte-identical input evidence yields byte-identical output. The matrix is re-applied to every `<!--ref:slug ...-->` on every pass; resolution tracks the current evidence, not a sticky historical state. Concretely:
- When the joined evidence (`source_acquired`, `source_verified_against_original`, derived `human_read_source`) is unchanged between passes, the marker's resolved form is byte-identical to the prior pass.
- When the joined evidence (`source_acquired`, `source_verified_against_original`, derived `human_read_source`, and — #513 — the governing mark's `read_scope` attestation together with the citation's own `<!--anchor:<kind>:<value>-->`, since scope-aware promotion is a function of the anchor too: a revision that moves an anchor from a covered to an uncovered locator re-resolves the marker even when corpus and ledger are unchanged) is unchanged between passes, the marker's resolved form is byte-identical to the prior pass.
- When the joined evidence changes between passes (user acquires / verifies the source, runs `/ars-mark-read <refcode>`, or runs `/ars-unmark-read <refcode>` to rescind a prior mark), the next finalizer pass re-applies the matrix from the new triple and re-emits the resolved form. Promotion (e.g. `LOW-WARN` → `ok` after `/ars-mark-read`) and demotion (e.g. `ok` → `LOW-WARN` after `/ars-unmark-read`, since spec §3.6 line 325/340 makes the most recent timestamped event win) are both possible.
In other words: the resolved status is a pure function of the current input triple; user-facing remediation and rescind affordances both round-trip through the matrix.
**Revision loops:** on revision loops (Stage 4 → reviewer → Stage 4 revise; or `academic-paper` Phase 6 → Phase 4 loops), the finalizer re-runs against the current draft, resolves any newly-emitted bare `<!--ref:slug-->` comments introduced in the revision pass, and re-applies the matrix to existing resolved markers per the idempotency rule above. Resolved markers **do not invalidate** in the sense that nothing about the revision-loop mechanism itself perturbs them — only a change in the joined evidence (acquire / verify / `/ars-mark-read` / `/ars-unmark-read`) can move a marker. When evidence is unchanged across a revision pass, every marker is preserved byte-identical.
**Revision loops:** on revision loops (Stage 4 → reviewer → Stage 4 revise; or `academic-paper` Phase 6 → Phase 4 loops), the finalizer re-runs against the current draft, resolves any newly-emitted bare `<!--ref:slug-->` comments introduced in the revision pass, and re-applies the matrix to existing resolved markers per the idempotency rule above. Resolved markers **do not invalidate** in the sense that nothing about the revision-loop mechanism itself perturbs them — only a change in the joined evidence (acquire / verify / `/ars-mark-read` / `/ars-unmark-read`, or — #513 — a revision editing the citation's anchor under a partial-coverage attestation) can move a marker. When evidence is unchanged across a revision pass, every marker is preserved byte-identical.
**LOW-WARN promotion:** when the user runs `/ars-mark-read <refcode>` between finalizer passes, the next pass observes `human_read_source: true` for that slug via the read-log join and resolves the marker to row 4 (`<!--ref:slug ok-->`). The finalizer does not delete the LOW-WARN entry from the per-section checklist artifact; that artifact is informational and the user clears it manually (or it falls out at the next checklist regeneration).
**Hard-gate handoff:** the finalizer never blocks pipeline progress on its own. It mutates the draft in place, then the orchestrator advances to Stage 5 where `formatter_agent` carries the hard-gate refusal rule (any `[UNVERIFIED CITATION ...]` literal or any unresolved `<!--ref:slug-->` whose status is neither `ok` nor LOW-WARN-acknowledged forces a refusal at format time per spec §3.3 line 185).
**Read-scope-aware promotion (#513).** The mark consumed by the row-3 → row-4 promotion may carry an optional `read_scope` attestation (`{level, locators[], note}`, schema `shared/contracts/passport/human_read_log.schema.json`). The governing signal follows the SAME latest-timestamped-event-wins rule as the binary case (spec §3.6): promotion is considered only when the slug's latest event overall is a mark — a latest rescind keeps row 3 regardless of any older non-rescinded marks — and the `read_scope` consulted is the one carried by that latest mark. Promotion then consults the attestation against the citation's own `<!--anchor:<kind>:<value>-->`: `level` absent or `unknown` promotes exactly as before (legacy marks impose no migration); `full_text` promotes; `abstract_only` / `toc_only` does NOT promote — the marker resolves to `<!--ref:slug LOW-WARN-PARTIAL-COVERAGE-->` (a draft-visible acknowledged-partial state, distinguishable from an unacknowledged `LOW-WARN`) and the per-section checklist entry gains an explicit coverage note (e.g. `read_scope abstract_only does not cover anchor page:12`); `sections` promotes ONLY when a `page` / `section` / `paragraph` anchor value falls unambiguously within a declared locator (locators are free text — promote on a clear containment match only; any ambiguity or no match resolves to `LOW-WARN-PARTIAL-COVERAGE` + coverage note), and `quote` anchors promote only under `full_text` or `unknown` (with partial coverage the finalizer cannot vouch that the quoted passage lies in a read section). `LOW-WARN-PARTIAL-COVERAGE` behaves as `LOW-WARN` for every downstream mechanism that keys on the base status (contamination suffixes attach the same way; severity tier unchanged; never a hard gate) — the formatter treats it as an acknowledged LOW-WARN variant and passes it with the coverage advisory surfaced. The degradation is always this retained LOW-WARN-tier state with a note — never a new severity, never below the pre-mark state. This paragraph refines WHEN row 3 promotes to row 4; the matrix rows and the join are unchanged, and the idempotency rule's evidence enumeration above explicitly includes the governing mark's attestation — a `read_scope` change between passes is an evidence change and re-resolves the marker.
**Hard-gate handoff:** the finalizer never blocks pipeline progress on its own. It mutates the draft in place, then the orchestrator advances to Stage 5 where `formatter_agent` carries the hard-gate refusal rule (any `[UNVERIFIED CITATION ...]` literal or any unresolved `<!--ref:slug-->` whose status is neither `ok`, LOW-WARN-acknowledged, nor `LOW-WARN-PARTIAL-COVERAGE` (#513 acknowledged-partial — passes, never refused) forces a refusal at format time per spec §3.3 line 185).
**Audit trail:** the finalizer's per-pass resolution counts (HIGH WARN / MED WARN / LOW WARN / OK / unresolved) are logged via `state_tracker` for the pipeline audit trail and surface in the Stage 4.5 integrity-check report.
@@ -739,7 +741,7 @@ NO-LOCATOR is MED severity (not HIGH) because the citation may still point at a
### Contamination annotation (L3-2)
After the 4-cell matrix resolves a citation to `ok` or `LOW-WARN`, the finalizer reads the entry's `contamination_signals` object from `literature_corpus[]` (if present) and appends an annotation suffix:
After the 4-cell matrix resolves a citation to `ok` or `LOW-WARN`, the finalizer reads the entry's `contamination_signals` object from `literature_corpus[]` (if present) and appends an annotation suffix. (#513: `LOW-WARN-PARTIAL-COVERAGE` behaves as `LOW-WARN` throughout this section's — and the v3.9.0/v3.10/v3.11 extensions' — `ok`/`LOW-WARN` base-status enumerations: suffixes attach the same way, `policy_hash` stamping applies the same way, and `scripts/check_v3_10_policy.py` recognizes it as a base status.)
| Base resolution | contamination_signals state | Annotated marker |
|---|---|---|
+2
View File
@@ -7,6 +7,8 @@ Acknowledge that the user has personally read the source(s) backing the named ci
The dispatching agent substitutes `<path>` below with the active Material Passport path from session context before executing (the quoting is preserved so paths containing spaces remain a single argument). The CLI handles validation (citation_key must exist in `literature_corpus[]`; on miss emit `[ARS-MARK-READ ERROR: citation_key '<slug>' not in literature_corpus[]]` and refuse to write), 4 fail-fast environment checks (no active passport / passport not found / parent unreadable / read-log unwritable), and append-only write per §3.6 firm rule 3.
Optional read-scope attestation (#513, declaration-only — pass through whatever the user states, never infer): `--scope {full_text,sections,abstract_only,toc_only,unknown}` records how much of the source was read; `--locator "<text>"` (repeatable, requires `--scope sections`) names the read sections/pages; `--note "<text>"` free text (requires `--scope`). Without `--scope`, no `read_scope` is recorded and downstream treats the mark as `unknown` (pre-#513 behavior). A partial-coverage attestation means the finalizer promotes LOW-WARN → ok only for anchors falling within declared coverage — see the Read-scope-aware promotion paragraph in `pipeline_orchestrator_agent.md`.
Implementation:
```bash
python3 scripts/ars_mark_read.py $ARGUMENTS --passport-path "<path>"
@@ -0,0 +1,132 @@
# #513 — `read_scope` honest-coverage attestation + anchor-aware finalizer promotion
**Date:** 2026-07-20 · **Issue:** #513 · **Status:** implemented in the same PR
## Problem
ARS records source possession, AI verification-against-original, and a binary human-read
mark — but nothing records **how much** of a source was actually read. A user who read
only the abstract is indistinguishable from one who read the whole paper, and the
Cite-Time Provenance Finalizer's LOW-WARN → `ok` promotion consumes the mark as a binary:
a TOC-only reading promotes a citation whose `page` anchor points at a chapter the user
never opened.
Placement constraint (verified in the #513 dual-track review): corpus entries are
adapter-owned and MUST NOT carry human-read state (v3.6.8 firm rule 3), so the peer
project's shape — a field on the bibliographic entry — is not available. The attestation
belongs on the **user-owned human-read ledger** (`<passport-stem>_human_read_log.yaml`).
Provenance: mechanism observed in kengo006/alexandria (mandatory note declaring actual
reading coverage); ranked P2 of three in the 2026-07-11 adoption review.
## Design
### Layer 1 — ledger field + CLI (`scripts/ars_mark_read.py`)
Ledger entries gain an optional `read_scope` object:
```yaml
human_read:
- citation_key: smith2024
marked_at: "2026-07-20T04:00:00Z"
read_scope: # optional — absent on legacy and scope-less marks
level: sections # full_text | sections | abstract_only | toc_only | unknown
locators: # only meaningful (and only accepted) with level: sections
- "pp. 10-24"
- "section 3"
note: "methods + results read closely; discussion skimmed"
```
`/ars-mark-read` gains optional arguments, all attestation-only (declaration, never
inference):
- `--scope <level>` — closed enum above. Absent ⇒ no `read_scope` written; consumers
treat absence as `unknown`. Nothing is fabricated or backfilled.
- `--locator <text>` — repeatable; **requires `--scope sections`** (locators name which
sections/pages were read; with `full_text` they are redundant and with
`abstract_only`/`toc_only` they contradict the level — a contradictory attestation is
refused, not recorded).
- `--note <text>` — free text; requires `--scope` (a note is part of an attestation).
- `--scope`/`--locator`/`--note` are rejected with `--unmark` (rescinding takes no
attestation).
- Batch semantics unchanged: one invocation's `read_scope` applies to every key in the
batch; validation stays all-or-nothing.
Errors use the existing canonical `[ARS-MARK-READ ERROR: ...]` surface.
### Layer 2 — ledger sidecar schema
New `shared/contracts/passport/human_read_log.schema.json`, following the
`rejection_log.schema.json` / `version_records.schema.json` sidecar precedent
(`additionalProperties: false` throughout, closed `level` enum, registered in
`shared/contracts/README.md`). The ledger stays adapter-free and user-owned; the schema
exists for audit/debugging and test-time validation — `ars_mark_read.py` itself stays
dependency-light (no jsonschema import at runtime).
### Layer 3 — anchor-aware finalizer promotion (prose, `pipeline_orchestrator_agent.md`)
The v3.7.1 finalizer block gains a read-scope-aware promotion paragraph (plain bold
paragraph, no nested heading — the `check_v3_6_8_cite_provenance_pipeline.py` block
extractor terminates at headings). The LOW-WARN → `ok` transition consults the mark's
`read_scope`:
The governing signal follows the existing latest-timestamped-event-wins rule (§3.6):
promotion is considered only when the slug's latest event overall is a mark — a latest
rescind keeps row 3 regardless of older non-rescinded marks — and the attestation
consulted is the one on that latest mark (codex r1: "most recent non-rescinded" would
have contradicted the settled precedence and resurrected rescinded promotions).
| `read_scope.level` | Promotion of the citation's anchor |
|---|---|
| absent / `unknown` | promotes (legacy marks keep their pre-#513 behavior — the optional field must not impose a de-facto migration) |
| `full_text` | promotes |
| `abstract_only` / `toc_only` | does NOT promote — the marker resolves to `LOW-WARN-PARTIAL-COVERAGE` and the per-section checklist entry carries an explicit coverage note (e.g. `read_scope abstract_only does not cover anchor page:12`) |
| `sections` | promotes ONLY when the anchor (`page` / `section` / `paragraph`) falls unambiguously within a declared locator; ambiguity or no match ⇒ `LOW-WARN-PARTIAL-COVERAGE` + coverage note. `quote` anchors promote only under `full_text` / `unknown` — with partial coverage the finalizer cannot vouch that the quoted passage lies in a read section |
`LOW-WARN-PARTIAL-COVERAGE` (codex r1: a partial acknowledgment that left the plain
`LOW-WARN` marker was indistinguishable from an unacknowledged citation at the terminal
gate, forcing the formatter to either refuse an acknowledged mark or pass unacknowledged
ones) is a draft-visible acknowledged-partial state: same severity tier as `LOW-WARN`,
contamination suffixes attach identically, and the formatter passes it as an
acknowledged LOW-WARN variant with the coverage note surfaced — never refused, never a
new severity. The v3.7.3 ref-marker grammar (`[\w-]+` status tokens) admits it without
lint changes. The idempotency rule's evidence enumeration now names the governing
mark's attestation explicitly — a `read_scope` change between passes is an evidence
change and re-resolves the marker. The judgment "falls unambiguously within" is
conservative by instruction: locators are free text; the finalizer promotes only on a
clear containment match.
CLI bounds (codex r1): `--locator` values 1-200 chars and `--note` 1-1000 chars are
enforced at write time — in lockstep with the sidecar schema — so the CLI can never
produce a ledger the committed schema rejects; presence checks use `is not None`, so an
explicitly supplied empty string is an invalid attestation argument, not an absent one.
### Doc surfaces updated in lockstep
`commands/ars-mark-read.md` (optional-arguments paragraph; pinned tokens preserved),
`academic-paper/agents/formatter_agent.md` LOW-WARN remediation line (mentions the scope
argument), and the #528 content lock on `pipeline_orchestrator_agent.md` re-pinned in
the same commit.
## Out of scope (deliberate, from the issue)
- Any new field on `literature_corpus[]` entries (adapter-owned; v3.6.5 consumer
protocol).
- Adapter inference of reading depth — declaration-only.
- Mandatory migration — legacy marks mean `unknown` and keep their behavior.
- A `source_sha256` join field toward the #512 preflight sidecar. The #512 spec names
the sidecar `sha256` as the natural future join key, but #513's scope is the
human-attestation channel; adding a hash field with no wired consumer would be the
same dead-metadata risk this issue exists to avoid. The schema is additive, so the
field can land with its consumer.
## Test plan
- `scripts/test_ars_mark_read.py`: scope happy path per level; locators/note persisted;
invalid level / `--locator` without `sections` / `--note` without `--scope` /
attestation args with `--unmark` all rejected batch-wide with canonical errors;
scope-less marks write no `read_scope` key (byte-shape backward compat); produced
ledgers validate against the new schema; legacy ledger entries untouched by new marks.
- `tests/test_mark_read_args.py`: dispatch-level pass-through of the new flags.
- Lints: `check_v3_6_8_*` all stay green; `check_pipeline_boundary_semantics.py` hash
re-pinned.
+96 -5
View File
@@ -20,6 +20,15 @@ Behavior summary:
`rescinded_at` to the matching entry, never deletes.
- First-time write creates the file with the YAML schema header. Not a
fail-fast condition.
- #513 read_scope attestation (declaration-only, never inferred): optional
`--scope {full_text,sections,abstract_only,toc_only,unknown}` records HOW
MUCH of the source was read; `--locator` (repeatable, requires
`--scope sections`) names the read sections/pages; `--note` free text
(requires `--scope`). Absent `--scope` writes no `read_scope` field
consumers treat absence as `unknown`; nothing is fabricated or backfilled.
Attestation args are rejected with `--unmark` (rescinding takes no
attestation). One invocation's attestation applies to every key in the
batch. Sidecar schema: shared/contracts/passport/human_read_log.schema.json.
"""
from __future__ import annotations
@@ -34,6 +43,15 @@ import yaml
ERR_PREFIX = "[ARS-MARK-READ ERROR:"
# #513 closed level enum + text bounds — keep in lockstep with
# shared/contracts/passport/human_read_log.schema.json (level enum; locators
# items minLength 1 / maxLength 200; note minLength 1 / maxLength 1000). The
# CLI enforces the bounds at write time so it can never produce a ledger the
# committed schema rejects (codex #513 r1 P1).
READ_SCOPE_LEVELS = ("full_text", "sections", "abstract_only", "toc_only", "unknown")
LOCATOR_MAX_LEN = 200
NOTE_MAX_LEN = 1000
def _err(msg: str) -> str:
return f"{ERR_PREFIX} {msg}]"
@@ -128,10 +146,64 @@ def _save_log(log_path: Path, data: dict) -> None:
yaml.safe_dump(data, f, sort_keys=False, allow_unicode=True)
def _mark(log: dict, citation_key: str) -> None:
log["human_read"].append(
{"citation_key": citation_key, "marked_at": _now_iso()}
)
def _mark(log: dict, citation_key: str, read_scope: dict | None = None) -> None:
entry: dict = {"citation_key": citation_key, "marked_at": _now_iso()}
if read_scope is not None:
entry["read_scope"] = read_scope
log["human_read"].append(entry)
def _build_read_scope(args: argparse.Namespace) -> dict | None:
"""Validate the #513 attestation flags and build the read_scope object.
Returns None when no attestation was given (legacy mark shape), the
read_scope dict when valid, or raises SystemExit(2) with the canonical
error surface on a contradictory attestation a contradictory or
partial attestation is refused, never recorded ambiguously."""
# Presence is `is not None`, NEVER truthiness: `--note ""` is an explicitly
# supplied (invalid) attestation argument, not an absent one — truthiness
# would let it bypass both the requires-scope rule and the unmark rejection
# (codex #513 r1 P1).
locator_given = args.locator is not None
note_given = args.note is not None
errors: list[str] = []
if args.unmark and (args.scope is not None or locator_given or note_given):
errors.append("--scope/--locator/--note cannot be combined with --unmark (rescinding takes no attestation)")
elif args.scope is None:
if locator_given:
errors.append("--locator requires --scope sections (an attestation needs a declared level)")
if note_given:
errors.append("--note requires --scope (an attestation needs a declared level)")
else:
if locator_given and args.scope != "sections":
errors.append(
f"--locator requires --scope sections; with --scope {args.scope} "
"locators are redundant or contradict the declared level"
)
if locator_given:
for loc in args.locator:
if not 1 <= len(loc) <= LOCATOR_MAX_LEN:
errors.append(
f"--locator value must be 1-{LOCATOR_MAX_LEN} characters "
f"(got {len(loc)}); the sidecar schema would reject the ledger"
)
if note_given and not 1 <= len(args.note) <= NOTE_MAX_LEN:
errors.append(
f"--note must be 1-{NOTE_MAX_LEN} characters (got {len(args.note)}); "
"the sidecar schema would reject the ledger"
)
if errors:
for e in errors:
print(_err(e), file=sys.stderr)
raise SystemExit(2)
if args.scope is None:
return None
read_scope: dict = {"level": args.scope}
if locator_given:
read_scope["locators"] = list(args.locator)
if note_given:
read_scope["note"] = args.note
return read_scope
def _unmark(log: dict, citation_key: str) -> bool:
@@ -164,8 +236,27 @@ def main(argv: list[str] | None = None) -> int:
action="store_true",
help="Rescind prior marks (write rescinded_at instead of marked_at).",
)
parser.add_argument(
"--scope",
choices=READ_SCOPE_LEVELS,
default=None,
help="#513 read_scope attestation: how much of the source was read. "
"Absent = no read_scope recorded (consumers treat as unknown).",
)
parser.add_argument(
"--locator",
action="append",
default=None,
help="Which sections/pages were read (repeatable; requires --scope sections).",
)
parser.add_argument(
"--note",
default=None,
help="Free-text attestation note (requires --scope).",
)
args = parser.parse_args(argv)
read_scope = _build_read_scope(args)
passport_path, log_path = _validate_passport_environment(args.passport_path)
corpus_keys = _load_corpus_keys(passport_path)
@@ -197,7 +288,7 @@ def main(argv: list[str] | None = None) -> int:
return 2
else:
for k in args.citation_keys:
_mark(log, k)
_mark(log, k, read_scope)
_save_log(log_path, log)
return 0
+1 -1
View File
@@ -71,7 +71,7 @@ REPO_ROOT = Path(__file__).resolve().parent.parent
# ---------------------------------------------------------------------------
CONTENT_LOCKS = {
"academic-pipeline/SKILL.md": "69c1a8bfe01ab252b13fe9685e778591d3078fd82a0d6d84763f1bae1d3ff510",
"academic-pipeline/agents/pipeline_orchestrator_agent.md": "9c9cdc58ded9f63e855c6d37697a4e02650f45b6f93c1f337b032cafd986cd99",
"academic-pipeline/agents/pipeline_orchestrator_agent.md": "a08c2446a84abfa23801f1727e96e694046fcf8ef73bb0c799150c28e3d2aa3d",
"academic-pipeline/agents/state_tracker_agent.md": "6528349959ae9ef9c126bab3117a57da894e9517104a83a0d1d7ad74daf36d5f",
"academic-pipeline/references/pipeline_state_machine.md": "d61ac458e39d8bd7f8ad0c85b01887a1ffc5657578b883af15989e2677d60e89",
"academic-pipeline/references/process_summary_protocol.md": "5c7053230d73b39d0a5d9d6f5e9f339c12570ae6d3aa2eae2eaf74f51d571e94",
+4 -1
View File
@@ -88,7 +88,10 @@ V3_10_FORMATTER_HEADER = "## Cite-Time Terminal Policy Gate (v3.10)"
# severity=HIGH-BLOCK policy=<p> reason=<r> mode=<m> policy_hash=<slug>-->
# Legacy v3.9.0 markers carry NO policy_hash and are recognized separately.
_REF_MARKER_RE = re.compile(r"<!--ref:(?P<inner>[^>]*?)-->")
_BASE_STATUS = ("ok", "LOW-WARN")
# #513: LOW-WARN-PARTIAL-COVERAGE is the acknowledged-partial state emitted by the
# read-scope-aware promotion — it behaves as LOW-WARN for every base-status
# mechanism (contamination suffixes, policy_hash stamping, terminal policies).
_BASE_STATUS = ("ok", "LOW-WARN", "LOW-WARN-PARTIAL-COVERAGE")
# Recognized key=value prefixes inside a finalized marker. A token that is
+204
View File
@@ -353,5 +353,209 @@ class TestReadLogUnwritableExistingFile(unittest.TestCase):
log_path.chmod(stat.S_IRUSR | stat.S_IWUSR)
class TestReadScopeAttestation(unittest.TestCase):
"""#513: optional read_scope attestation on ledger marks — declaration-only,
all-or-nothing batch validation, byte-shape backward compat when absent."""
def _mark(self, tmp, *extra, keys=("smith2024",)):
root = Path(tmp)
passport = root / "p.yaml"
_write_passport(passport, citation_keys=["smith2024", "lee2023"])
result = run_script(SCRIPT, *keys, "--passport-path", str(passport), *extra)
return passport, result
def test_scope_less_mark_writes_no_read_scope_key(self):
with TemporaryDirectory() as tmp:
passport, result = self._mark(tmp)
self.assertEqual(result.returncode, 0, result.stderr)
(entry,) = _read_log(passport)["human_read"]
self.assertNotIn("read_scope", entry)
def test_each_level_persisted(self):
for level in ("full_text", "abstract_only", "toc_only", "unknown"):
with self.subTest(level=level), TemporaryDirectory() as tmp:
passport, result = self._mark(tmp, "--scope", level)
self.assertEqual(result.returncode, 0, result.stderr)
(entry,) = _read_log(passport)["human_read"]
self.assertEqual(entry["read_scope"], {"level": level})
def test_sections_with_locators_and_note(self):
with TemporaryDirectory() as tmp:
passport, result = self._mark(
tmp, "--scope", "sections", "--locator", "pp. 10-24",
"--locator", "section 3", "--note", "methods read closely",
)
self.assertEqual(result.returncode, 0, result.stderr)
(entry,) = _read_log(passport)["human_read"]
self.assertEqual(
entry["read_scope"],
{"level": "sections", "locators": ["pp. 10-24", "section 3"],
"note": "methods read closely"},
)
def test_batch_applies_same_scope_to_every_key(self):
with TemporaryDirectory() as tmp:
passport, result = self._mark(
tmp, "--scope", "full_text", keys=("smith2024", "lee2023"),
)
self.assertEqual(result.returncode, 0, result.stderr)
entries = _read_log(passport)["human_read"]
self.assertEqual(len(entries), 2)
for entry in entries:
self.assertEqual(entry["read_scope"], {"level": "full_text"})
def test_invalid_level_rejected_no_write(self):
with TemporaryDirectory() as tmp:
passport, result = self._mark(tmp, "--scope", "skimmed")
self.assertNotEqual(result.returncode, 0)
log_path = passport.parent / f"{passport.stem}_human_read_log.yaml"
self.assertFalse(log_path.exists(), "invalid attestation must not write")
def test_locator_requires_sections_level(self):
for level in ("full_text", "abstract_only", "toc_only", "unknown"):
with self.subTest(level=level), TemporaryDirectory() as tmp:
passport, result = self._mark(
tmp, "--scope", level, "--locator", "pp. 1-2",
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("[ARS-MARK-READ ERROR:", result.stderr)
log_path = passport.parent / f"{passport.stem}_human_read_log.yaml"
self.assertFalse(log_path.exists())
def test_locator_or_note_without_scope_rejected(self):
for extra in (("--locator", "pp. 1-2"), ("--note", "read it")):
with self.subTest(extra=extra), TemporaryDirectory() as tmp:
passport, result = self._mark(tmp, *extra)
self.assertNotEqual(result.returncode, 0)
self.assertIn("[ARS-MARK-READ ERROR:", result.stderr)
def test_attestation_args_rejected_with_unmark(self):
with TemporaryDirectory() as tmp:
root = Path(tmp)
passport = root / "p.yaml"
_write_passport(passport, citation_keys=["smith2024"])
run_script(SCRIPT, "smith2024", "--passport-path", str(passport))
result = run_script(
SCRIPT, "smith2024", "--passport-path", str(passport),
"--unmark", "--scope", "full_text",
)
self.assertNotEqual(result.returncode, 0)
self.assertIn("[ARS-MARK-READ ERROR:", result.stderr)
(entry,) = _read_log(passport)["human_read"]
self.assertNotIn("rescinded_at", entry, "rejected unmark must not write")
def test_legacy_entries_untouched_by_new_scoped_mark(self):
with TemporaryDirectory() as tmp:
root = Path(tmp)
passport = root / "p.yaml"
_write_passport(passport, citation_keys=["smith2024", "lee2023"])
run_script(SCRIPT, "smith2024", "--passport-path", str(passport))
result = run_script(
SCRIPT, "lee2023", "--passport-path", str(passport),
"--scope", "abstract_only",
)
self.assertEqual(result.returncode, 0, result.stderr)
first, second = _read_log(passport)["human_read"]
self.assertNotIn("read_scope", first)
self.assertEqual(second["read_scope"], {"level": "abstract_only"})
def test_empty_string_note_is_present_not_absent(self):
# codex #513 r1: --note "" is an explicitly supplied attestation arg;
# truthiness checks would let it bypass the requires-scope rule.
with TemporaryDirectory() as tmp:
passport, result = self._mark(tmp, "--note", "")
self.assertNotEqual(result.returncode, 0)
self.assertIn("[ARS-MARK-READ ERROR:", result.stderr)
def test_empty_note_with_unmark_rejected(self):
with TemporaryDirectory() as tmp:
root = Path(tmp)
passport = root / "p.yaml"
_write_passport(passport, citation_keys=["smith2024"])
run_script(SCRIPT, "smith2024", "--passport-path", str(passport))
result = run_script(
SCRIPT, "smith2024", "--passport-path", str(passport),
"--unmark", "--note", "",
)
self.assertNotEqual(result.returncode, 0)
(entry,) = _read_log(passport)["human_read"]
self.assertNotIn("rescinded_at", entry)
def test_schema_bounds_enforced_at_write_time(self):
# codex #513 r1: the CLI must never produce a ledger the sidecar schema
# rejects — empty and oversize locator/note values are refused.
cases = [
("--scope", "sections", "--locator", ""),
("--scope", "sections", "--locator", "x" * 201),
("--scope", "full_text", "--note", "x" * 1001),
]
for extra in cases:
with self.subTest(extra=extra[-1][:10] or "(empty)"), TemporaryDirectory() as tmp:
passport, result = self._mark(tmp, *extra)
self.assertNotEqual(result.returncode, 0)
self.assertIn("[ARS-MARK-READ ERROR:", result.stderr)
log_path = passport.parent / f"{passport.stem}_human_read_log.yaml"
self.assertFalse(log_path.exists())
def test_max_length_boundary_values_accepted(self):
with TemporaryDirectory() as tmp:
passport, result = self._mark(
tmp, "--scope", "sections",
"--locator", "x" * 200, "--note", "y" * 1000,
)
self.assertEqual(result.returncode, 0, result.stderr)
(entry,) = _read_log(passport)["human_read"]
self.assertEqual(len(entry["read_scope"]["locators"][0]), 200)
self.assertEqual(len(entry["read_scope"]["note"]), 1000)
def test_schema_rejects_locators_on_non_sections_level(self):
# codex #513 r2: the sidecar schema mirrors the CLI's locators-require-
# sections rule so audit-time validation matches the writer contract.
import json
try:
import jsonschema
except ImportError:
self.skipTest("jsonschema not installed")
schema_path = (
Path(SCRIPT).resolve().parent.parent
/ "shared" / "contracts" / "passport" / "human_read_log.schema.json"
)
schema = json.loads(schema_path.read_text(encoding="utf-8"))
bad = {
"session_id": "s", "created_at": "2026-07-20T00:00:00Z",
"human_read": [{
"citation_key": "smith2024", "marked_at": "2026-07-20T00:00:00Z",
"read_scope": {"level": "full_text", "locators": ["pp. 1-2"]},
}],
}
with self.assertRaises(jsonschema.ValidationError):
jsonschema.validate(bad, schema)
bad["human_read"][0]["read_scope"] = {"level": "sections", "locators": ["pp. 1-2"]}
jsonschema.validate(bad, schema) # sections+locators stays valid
def test_ledger_validates_against_sidecar_schema(self):
import json
try:
import jsonschema
except ImportError:
self.skipTest("jsonschema not installed")
schema_path = (
Path(SCRIPT).resolve().parent.parent
/ "shared" / "contracts" / "passport" / "human_read_log.schema.json"
)
schema = json.loads(schema_path.read_text(encoding="utf-8"))
with TemporaryDirectory() as tmp:
passport, result = self._mark(
tmp, "--scope", "sections", "--locator", "pp. 1-9",
"--note", "intro only",
)
self.assertEqual(result.returncode, 0, result.stderr)
run_script(SCRIPT, "smith2024", "--passport-path", str(passport), "--unmark")
jsonschema.validate(_read_log(passport), schema)
if __name__ == "__main__":
unittest.main()
+6
View File
@@ -59,6 +59,12 @@ Schemas for Material Passport input ports.
`phase2_investigation/version_records.yaml` sidecar for academic citation version
families (preprint -> proceedings -> journal extension). This is deliberately a
sidecar: `literature_corpus_entry.schema.json` stays adapter-owned and unmodified.
- `passport/human_read_log.schema.json` (#513) — the user-owned human-read ledger
(`<passport-stem>_human_read_log.yaml`, written by `scripts/ars_mark_read.py`),
including the optional #513 `read_scope` honest-coverage attestation
(`level`/`locators`/`note`, declaration-only). Deliberately a sidecar for the same
reason as above: corpus entries MUST NOT carry human-read state (v3.6.8 firm rule 3).
Audit/test-time validation only — the CLI stays dependency-light at runtime.
## Audit artifact contracts (v3.6.7 Step 6)
@@ -0,0 +1,75 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/Imbad0202/academic-research-skills/shared/contracts/passport/human_read_log.schema.json",
"title": "ARS human-read ledger (peer file of the Material Passport)",
"description": "Schema for <passport-stem>_human_read_log.yaml, the user-owned human-read signal ledger written by scripts/ars_mark_read.py (v3.6.8 §3.6; #513 read_scope attestation). Deliberately a sidecar: literature_corpus_entry.schema.json stays adapter-owned and unmodified — corpus entries MUST NOT carry human-read state (firm rule 3). The ledger is append-only; unmark appends rescinded_at, never deletes. read_scope is declaration-only (#513): absent means unknown, never fabricated or backfilled. Exists for audit/debugging and test-time validation; the CLI itself stays dependency-light and does not validate at runtime.",
"type": "object",
"additionalProperties": false,
"required": ["session_id", "created_at", "human_read"],
"properties": {
"session_id": {
"type": "string",
"minLength": 1
},
"created_at": {
"type": "string",
"minLength": 1,
"description": "ISO 8601 UTC timestamp of ledger creation."
},
"human_read": {
"type": "array",
"items": { "$ref": "#/$defs/human_read_entry" }
}
},
"$defs": {
"human_read_entry": {
"type": "object",
"additionalProperties": false,
"required": ["citation_key", "marked_at"],
"properties": {
"citation_key": {
"type": "string",
"minLength": 1,
"description": "Joins against literature_corpus[].citation_key in the active passport."
},
"marked_at": {
"type": "string",
"minLength": 1
},
"rescinded_at": {
"type": "string",
"minLength": 1,
"description": "Appended in place by /ars-unmark-read; presence means this mark no longer counts. Most-recent-timestamp-wins per v3.6.8 §3.6."
},
"read_scope": { "$ref": "#/$defs/read_scope" }
}
},
"read_scope": {
"type": "object",
"additionalProperties": false,
"required": ["level"],
"description": "#513 honest-coverage attestation: how much of the source the user actually read. Declaration-only — adapters and agents never infer or backfill it. Locators require level: sections (the if/then below mirrors the CLI's write-time refusal of the contradictory combination).",
"if": { "required": ["locators"] },
"then": {
"properties": { "level": { "const": "sections" } }
},
"properties": {
"level": {
"type": "string",
"enum": ["full_text", "sections", "abstract_only", "toc_only", "unknown"]
},
"locators": {
"type": "array",
"minItems": 1,
"items": { "type": "string", "minLength": 1, "maxLength": 200 },
"description": "Free-text section/page declarations (e.g. 'pp. 10-24', 'section 3'). Only meaningful — and only accepted by the CLI — with level: sections."
},
"note": {
"type": "string",
"minLength": 1,
"maxLength": 1000
}
}
}
}
}