fix(citation-gate): take ref_slug from the prose join, not the corpus entry (#332) (#337)

* fix(citation-gate): take ref_slug from the prose join, not the corpus entry (#332)

verify_citation wrote summary["ref_slug"] = entry.get("ref_slug"), but
literature_corpus_entry.schema.json is additionalProperties:False with no
ref_slug property — a schema-valid corpus entry never carries one, so the normal
passport path emitted ref_slug: None and violated the summary contract (ref_slug
required string). The 133 tests missed it because both fixtures (the _entry
helper and the CLI fixture) illegally stuffed ref_slug INTO the corpus entry,
masking the production shape.

ref_slug is now an explicit prose-sourced param, parallel to the existing
`anchor` param (both joined upstream, never read off the entry). verify_passport
takes a {citation_key: ref_slug} join map and raises ValueError on a missing
join rather than emitting a contract-invalid summary. The standalone CLI has no
prose document, so it refuses by default with a clear error and offers an
explicit --synthetic-ref-slug citation_key escape hatch (stderr-warned,
diagnostic-only) instead of silently fabricating a slug.

P2: check_evals_gold_set's flat queried_by enum check under-enforced the
status<->queried_by coherence the summary schema requires (ran -> {id,title};
skipped/unreachable -> null; queried_by required present). Replaced it by
validating each resolver_outcome against the shipped summary-schema
$defs.resolver_outcome (single source of truth, matching the I9b reducer-recompute
philosophy), and removed the now-dead STATUS_ENUM/QUERIED_BY_ENUM constants.

Fixtures are now schema-valid against literature_corpus_entry.schema.json (added
authors/year/source_pointer, dropped ref_slug, fixed an invalid DOI) with a
guard test in each module so a future schema change can't silently re-mask.

Tests: ref_slug-not-read-off-entry anti-regression, missing-join raises,
end-to-end passport summaries validate against the schema, CLI refuse +
synthetic modes, 3 queried_by-coherence mutation tests. 112 passing across the
citation-gate + eval suites.

Closes #332

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(citation-gate): reject non-string / empty ref_slug at the emission point (#332)

verify_citation declared ref_slug as a required str but did not enforce it at
runtime: a caller passing an empty string or a non-string would stamp a summary
the schema rejects (type:string) or that joins to no <!--ref:slug--> prose
marker (empty). verify_passport only guarded `is None`, so a present-but-empty
slug slipped through.

Adds a shared _is_valid_ref_slug (non-empty string) so verify_citation (the
single emission point) and verify_passport (the join layer) agree on "bad slug"
and can't drift. verify_citation now raises ValueError on any invalid ref_slug;
verify_passport re-checks only to name the offending citation_key (context the
per-citation layer lacks). Closes the codex-flagged hardening gap on the #332
fix.

Tests: parametrized rejection over {None, "", 0, 123, list, dict}; mutation-
verified (accept-all stub fails all 7 guard tests). Full suite 2109 pass /
3 skip / 0 fail.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edward Cheng-I Wu
2026-06-05 20:22:45 +08:00
committed by GitHub
parent 3ab2b2247a
commit 6286fd0b66
7 changed files with 369 additions and 70 deletions
+5
View File
@@ -12,6 +12,11 @@ All notable changes to this project will be documented in this file.
- **Parallelize the OpenAlex + Crossref backfill lookups per entry in `migrate_literature_corpus_to_v3_9_0.py` (#138).** When both `openalex_unmatched` and `crossref_unmatched` are missing for an entry, the two independent resolver calls (different hosts, per-instance throttle state, monotonic timing) now run concurrently via a 2-worker `ThreadPoolExecutor` instead of one-after-the-other, roughly halving per-entry network wait on a full backfill. Scope is deliberately bounded: only the two calls within one entry overlap — the corpus loop stays sequential (cross-entry parallelism is out of scope; the clients' per-instance throttle assumes serial use), all passport mutation / report bookkeeping / degradation logging stays single-threaded on the orchestrator thread, and an already-set field still never consults its client. A single missing field skips the pool and calls directly. Behavior is otherwise byte-equivalent to the sequential version, including the omit-on-`Unavailable` partial-degradation contract (now surfaced via `Future.result()`). Adds 2 tests (barrier-verified parallel dispatch + the previously-untested API-down degradation path); the 6 existing migration tests pass unchanged.
### Fixed
- **`verification_gate` reads `ref_slug` from the prose join, not the corpus entry (#332).** `verify_citation`/`verify_passport` previously wrote `summary.ref_slug = entry.get("ref_slug")`, but `literature_corpus_entry.schema.json` is `additionalProperties: false` with no `ref_slug` property — so the normal (schema-valid) passport path emitted `ref_slug: null` and violated the summary contract (a required string). Two non-schema-conformant test fixtures masked it. `ref_slug` is now an explicit prose-sourced parameter parallel to `anchor`: `verify_citation(entry, clients, *, ref_slug, anchor=None, …)` and `verify_passport(passport, clients, *, ref_slug_by_key, anchors=None, …)`, with a `ValueError` on any invalid join — a missing key, or a present-but-empty/non-string slug (validated once at the `verify_citation` emission point via a shared `_is_valid_ref_slug` so the per-citation and passport layers can't drift; the passport layer re-checks only to name the offending `citation_key`) — rather than a contract-invalid summary. The standalone `verify_passport.py` CLI (which has no prose document) now refuses by default with a clear error and offers an explicit `--synthetic-ref-slug citation_key` diagnostic escape hatch instead of silently fabricating a slug. **API-stability note (C-V4):** these are new *required* keyword-only parameters. The spec's C-V4 freeze names v3.10.0, but #182 was specced-but-not-implemented in v3.10 (spec §0 amendment) and first shipped in the v3.11.0 minor release — so no v3.10.0 caller can depend on the old signature, and C-V4 itself permits a minor release to add required fields. The only in-repo callers (the CLI + the internal `verify_passport``verify_citation` call) are updated in lockstep.
- **`check_evals_gold_set` enforces `status``queried_by` coherence via the shipped schema (#332).** The gold validator's flat `queried_by ∈ {id, title, null}` enum check under-enforced the conditional coherence the summary schema requires (a ran resolver must carry `id`/`title`, a skipped/unreachable one must carry `null`, and `queried_by` must be present). It now validates each `resolver_outcome` against `citation_verification_summary.schema.json`'s `$defs.resolver_outcome` — single source of truth, matching the existing I9b reduce-and-compare philosophy — and the now-dead `STATUS_ENUM`/`QUERIED_BY_ENUM` constants are removed. The shipped gold set already satisfies the stricter check.
## [3.11.0] - 2026-06-04 — Deterministic citation verification gate (#182)
The v3.11.0 minor release ships **#182 — a deterministic citation-existence verification gate**
+29 -12
View File
@@ -29,11 +29,12 @@ from citation_verification_summary import ( # noqa: E402
)
LABEL_ENUM = {"true", "false", "unresolvable"}
QUERIED_BY_ENUM = {"id", "title", None}
KIND_ENUM = {"valid_doi", "valid_arxiv", "manual_exempt", "fabricated",
"fabricated_title_only", "valid_unindexed"}
RESOLVER_NAMES = ("crossref", "openalex", "semantic_scholar", "arxiv")
STATUS_ENUM = {"matched", "unmatched", "unreachable", "skipped"}
# status + queried_by enums (and their status↔queried_by coherence) are now
# enforced by _RESOLVER_OUTCOME_VALIDATOR against the shipped summary-schema $def,
# not local constants (#332 P2).
_CORPUS_ENTRY_SCHEMA_PATH = (
Path(__file__).resolve().parent.parent
@@ -44,6 +45,23 @@ _CORPUS_ENTRY_VALIDATOR = Draft202012Validator(
format_checker=Draft202012Validator.FORMAT_CHECKER,
)
# Validate each resolver_outcome against the SHIPPED summary-schema $def rather
# than a hand-rolled coherence check, so the gold validator can't drift from the
# contract it pins (the I9b single-source-of-truth philosophy, applied to shape).
# The $def carries the status↔queried_by allOf coherence (ran → {id,title};
# skipped/unreachable → null) plus the required-present rule that a flat enum
# check silently under-enforced (#332 P2).
_SUMMARY_SCHEMA_PATH = (
Path(__file__).resolve().parent.parent
/ "shared" / "contracts" / "passport" / "citation_verification_summary.schema.json"
)
# Safe to validate the extracted $def in isolation because resolver_outcome is
# $ref-less today. If it ever gains a $ref into a sibling $def, build the validator
# from the full schema document (with a registry) instead of the slice.
_RESOLVER_OUTCOME_VALIDATOR = Draft202012Validator(
json.loads(_SUMMARY_SCHEMA_PATH.read_text(encoding="utf-8"))["$defs"]["resolver_outcome"]
)
def _load_json_strict(path: Path) -> Any:
"""Load JSON; raise on duplicate keys (I4)."""
@@ -206,17 +224,16 @@ def validate(root: Path) -> list[str]:
if entry is None:
errors.append(f"I9: {tid} resolver_outcomes missing resolver {resolver!r}")
continue
status = entry.get("status")
if status not in STATUS_ENUM:
# Validate the resolver_outcome against the shipped summary-schema
# $def: status enum, queried_by enum, queried_by required-present, AND
# the status↔queried_by coherence allOf (ran → {id,title};
# skipped/unreachable → null). A flat enum check under-enforced the
# last two (#332 P2).
for verr in _RESOLVER_OUTCOME_VALIDATOR.iter_errors(entry):
errors.append(
f"I9: {tid} resolver_outcomes.{resolver}.status={status!r} "
f"not in {sorted(STATUS_ENUM)}"
)
queried_by = entry.get("queried_by")
if queried_by not in QUERIED_BY_ENUM:
errors.append(
f"I9: {tid} resolver_outcomes.{resolver}.queried_by={queried_by!r} "
f"not in {{'id', 'title', null}}"
f"I9: {tid} resolver_outcomes.{resolver} violates "
f"citation_verification_summary $defs.resolver_outcome: "
f"{verr.message}"
)
recomputed = _reduce_lookup_verified(ros)
+42
View File
@@ -194,6 +194,48 @@ def test_i9_invalid_queried_by_enum_caught(tmp_path):
assert any("I9" in e for e in errors), f"I9 not caught; errors: {errors}"
def test_i9_queried_by_missing_caught(tmp_path):
"""I9 coherence (#332 P2): queried_by is REQUIRED present. An entry that omits
it must be caught the summary schema marks queried_by required, and a missing
key is load-bearing (an absent key could let an ambiguous unmatched silently
reduce to unresolvable instead of false)."""
target = _copy_clean(tmp_path)
exp = json.loads((target / "expected_outcomes.json").read_text(encoding="utf-8"))
del exp["001-valid-doi-test"]["resolver_outcomes"]["crossref"]["queried_by"]
(target / "expected_outcomes.json").write_text(json.dumps(exp))
errors = check_evals_gold_set.validate(target)
assert any("I9" in e for e in errors), f"I9 not caught; errors: {errors}"
def test_i9_ran_resolver_with_null_queried_by_caught(tmp_path):
"""I9 coherence (#332 P2): a ran resolver (matched/unmatched) must carry
queried_by {id, title}, never null the summary schema's allOf forces it.
A matched row claiming queried_by=null is incoherent and must be caught."""
target = _copy_clean(tmp_path)
exp = json.loads((target / "expected_outcomes.json").read_text(encoding="utf-8"))
ros = exp["001-valid-doi-test"]["resolver_outcomes"]
# crossref is matched in the clean fixture; null queried_by is incoherent.
ros["crossref"]["queried_by"] = None
(target / "expected_outcomes.json").write_text(json.dumps(exp))
errors = check_evals_gold_set.validate(target)
assert any("I9" in e for e in errors), f"I9 not caught; errors: {errors}"
def test_i9_skipped_resolver_with_nonnull_queried_by_caught(tmp_path):
"""I9 coherence (#332 P2): a skipped/unreachable resolver must carry
queried_by=null there was no query to attribute. A skipped row claiming
queried_by='id' is incoherent (the summary schema's allOf forces null)."""
target = _copy_clean(tmp_path)
exp = json.loads((target / "expected_outcomes.json").read_text(encoding="utf-8"))
ros = exp["001-valid-doi-test"]["resolver_outcomes"]
# arxiv is skipped on a non-arXiv citation in the clean fixture.
assert ros["arxiv"]["status"] == "skipped", "fixture assumption changed"
ros["arxiv"]["queried_by"] = "id" # incoherent: skipped must be null
(target / "expected_outcomes.json").write_text(json.dumps(exp))
errors = check_evals_gold_set.validate(target)
assert any("I9" in e for e in errors), f"I9 not caught; errors: {errors}"
def test_i9b_false_with_only_title_only_unmatched_caught(tmp_path):
"""I9b (C-V6(a)): a `false`-labeled tuple whose unmatched are all title-only
(queried_by != id) is a mislabel narrowed-false requires an ID-keyed
+125 -26
View File
@@ -21,14 +21,22 @@ if str(REPO_ROOT / "scripts") not in sys.path:
sys.path.insert(0, str(REPO_ROOT / "scripts"))
_DEFAULT_REF_SLUG = "vaswani-2017-attention"
def _entry(**overrides):
# Production-shaped corpus entry: NO anchor field (the v3.7.3 anchor lives in
# writer prose, joined by ref_slug — passed to verify_citation as an explicit
# `anchor` param, never read off the corpus entry).
# Production-shaped corpus entry: schema-valid against
# literature_corpus_entry.schema.json (additionalProperties:False). It carries
# NEITHER ref_slug NOR anchor — both live in writer prose and are passed to
# verify_citation as explicit params, never read off the corpus entry. Carries
# all five required corpus fields (citation_key/title/authors/year/source_pointer)
# so the fixture matches the real shape the gate sees in production.
base = {
"citation_key": "vaswani2017",
"ref_slug": "vaswani-2017-attention",
"title": "Attention Is All You Need",
"authors": [{"family": "Vaswani", "given": "Ashish"}],
"year": 2017,
"source_pointer": "kb://refs/vaswani2017",
"doi": "10.5555/abc",
"obtained_via": "folder-scan",
}
@@ -65,7 +73,7 @@ def test_matched_yields_true_and_id_queried():
cr = MagicMock()
cr.doi_lookup_with_title_check.return_value = {"title": ["X"]} # match
outcome = verify_citation(_entry(), _clients(crossref=cr))
outcome = verify_citation(_entry(), _clients(crossref=cr), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["lookup_verified"] == "true"
assert outcome["resolver_outcomes"]["crossref"]["status"] == "matched"
@@ -75,7 +83,7 @@ def test_matched_yields_true_and_id_queried():
def test_id_keyed_unmatched_yields_false():
from verification_gate import verify_citation
# All resolvers miss; entry has a DOI → ID-keyed unmatched → false.
outcome = verify_citation(_entry(), _clients())
outcome = verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["lookup_verified"] == "false"
assert outcome["resolver_outcomes"]["crossref"]["queried_by"] == "id"
@@ -83,7 +91,7 @@ def test_id_keyed_unmatched_yields_false():
def test_title_only_unmatched_yields_unresolvable():
from verification_gate import verify_citation
# No DOI → title-only unmatched everywhere → unresolvable (C-V6(a)).
outcome = verify_citation(_entry(doi=None), _clients())
outcome = verify_citation(_entry(doi=None), _clients(), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["lookup_verified"] == "unresolvable"
assert outcome["resolver_outcomes"]["crossref"]["queried_by"] == "title"
@@ -95,7 +103,7 @@ def test_resolver_outage_is_unreachable():
cr = MagicMock()
cr.doi_lookup_with_title_check.side_effect = CrossrefUnavailable("down")
# other three also miss (id-keyed) → false stands (anti-fabrication bias).
outcome = verify_citation(_entry(), _clients(crossref=cr))
outcome = verify_citation(_entry(), _clients(crossref=cr), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["resolver_outcomes"]["crossref"]["status"] == "unreachable"
assert outcome["resolver_outcomes"]["crossref"]["queried_by"] is None
assert outcome["lookup_verified"] == "false"
@@ -115,6 +123,7 @@ def test_all_unreachable_is_unresolvable():
outcome = verify_citation(
_entry(arxiv_id="1706.03762"),
_clients(crossref=cr, openalex=oa, semantic_scholar=s2, arxiv=ax),
ref_slug=_DEFAULT_REF_SLUG,
)
assert outcome["lookup_verified"] == "unresolvable"
for r in ("crossref", "openalex", "semantic_scholar", "arxiv"):
@@ -123,7 +132,7 @@ def test_all_unreachable_is_unresolvable():
def test_manual_entry_all_skipped_unresolvable():
from verification_gate import verify_citation
outcome = verify_citation(_entry(obtained_via="manual"), _clients())
outcome = verify_citation(_entry(obtained_via="manual"), _clients(), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["lookup_verified"] == "unresolvable"
for r in ("crossref", "openalex", "semantic_scholar", "arxiv"):
assert outcome["resolver_outcomes"][r]["status"] == "skipped"
@@ -132,28 +141,29 @@ def test_manual_entry_all_skipped_unresolvable():
def test_arxiv_skipped_on_non_arxiv_citation():
from verification_gate import verify_citation
cr = MagicMock(); cr.doi_lookup_with_title_check.return_value = {"title": ["X"]}
outcome = verify_citation(_entry(), _clients(crossref=cr)) # no arxiv_id
outcome = verify_citation(_entry(), _clients(crossref=cr), ref_slug=_DEFAULT_REF_SLUG) # no arxiv_id
assert outcome["resolver_outcomes"]["arxiv"]["status"] == "skipped"
def test_anchor_present_true_for_page_kind():
from verification_gate import verify_citation
# anchor is an EXPLICIT param (prose-sourced, joined by ref_slug upstream).
outcome = verify_citation(_entry(), _clients(), anchor=_PAGE_ANCHOR)
outcome = verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG, anchor=_PAGE_ANCHOR)
assert outcome["anchor_present"] is True
def test_anchor_present_false_for_none_kind():
from verification_gate import verify_citation
outcome = verify_citation(
_entry(), _clients(), anchor={"kind": "none", "value": None})
_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG,
anchor={"kind": "none", "value": None})
assert outcome["anchor_present"] is False
def test_anchor_present_false_when_anchor_omitted():
from verification_gate import verify_citation
# No anchor param (the prose join found no anchor for this ref_slug) → False.
outcome = verify_citation(_entry(), _clients())
outcome = verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["anchor_present"] is False
@@ -165,20 +175,26 @@ def test_anchor_is_not_read_off_the_corpus_entry():
not in literature_corpus)."""
from verification_gate import verify_citation
e = _entry(anchor={"kind": "page", "value": "99"}) # decoy on the entry
outcome = verify_citation(e, _clients()) # no anchor param
outcome = verify_citation(e, _clients(), ref_slug=_DEFAULT_REF_SLUG) # no anchor param
assert outcome["anchor_present"] is False
def test_outcome_carries_keys_and_timestamp():
from verification_gate import verify_citation
outcome = verify_citation(_entry(), _clients())
outcome = verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG)
assert outcome["citation_key"] == "vaswani2017"
assert outcome["ref_slug"] == "vaswani-2017-attention"
assert outcome["verification_timestamp"] # never null/empty
def test_outcome_validates_against_summary_schema():
"""The outcome must validate against citation_verification_summary.schema."""
"""The outcome must validate against citation_verification_summary.schema.
The fixture (_entry) is schema-valid against literature_corpus_entry.schema
(no ref_slug additionalProperties:False), so ref_slug comes ONLY from the
explicit param. Before #332 this passed spuriously because the fixture stuffed
an illegal ref_slug into the entry; with a production-shaped entry it now
actually exercises the contract (ref_slug must be a non-null string)."""
import json
from jsonschema import Draft202012Validator
from verification_gate import verify_citation
@@ -187,11 +203,39 @@ def test_outcome_validates_against_summary_schema():
REPO_ROOT / "shared" / "contracts" / "passport"
/ "citation_verification_summary.schema.json"
).read_text(encoding="utf-8"))
outcome = verify_citation(_entry(), _clients())
outcome = verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG)
errors = list(Draft202012Validator(schema).iter_errors(outcome))
assert errors == [], f"outcome must validate: {errors}"
def test_ref_slug_is_not_read_off_the_corpus_entry():
"""Anti-regression for #332: even if a corpus entry erroneously carries a
'ref_slug' key, it MUST be ignored the emitted ref_slug derives ONLY from
the explicit param (the ref_slug lives in writer prose, not in
literature_corpus, whose schema forbids the field). Pins that we don't
resurrect the entry.get('ref_slug') path that emitted ref_slug: None on every
schema-valid passport."""
from verification_gate import verify_citation
e = _entry(ref_slug="decoy-off-the-entry") # illegal decoy on the entry
outcome = verify_citation(e, _clients(), ref_slug="from-prose")
assert outcome["ref_slug"] == "from-prose"
def test_corpus_fixture_is_schema_valid():
"""Guard the guard: the _entry fixture must itself validate against
literature_corpus_entry.schema.json (the masking bug in #332 was a fixture
that carried a field the corpus schema forbids). If this fails, the schema
tests above are testing a shape production never sees."""
import json
from jsonschema import Draft202012Validator
schema = json.loads((
REPO_ROOT / "shared" / "contracts" / "passport"
/ "literature_corpus_entry.schema.json"
).read_text(encoding="utf-8"))
errors = list(Draft202012Validator(schema).iter_errors(_entry()))
assert errors == [], f"_entry fixture must be a valid corpus entry: {errors}"
# ---------- verify_passport ----------
@@ -201,13 +245,16 @@ def test_verify_passport_runs_each_entry():
cr = MagicMock(); cr.doi_lookup_with_title_check.return_value = {"title": ["X"]}
passport = {
"literature_corpus": [
_entry(citation_key="a", ref_slug="slug-a"),
_entry(citation_key="b", ref_slug="slug-b", doi=None),
_entry(citation_key="a"),
_entry(citation_key="b", doi=None),
]
}
outcomes = verify_passport(passport, clients=_clients(crossref=cr))
outcomes = verify_passport(
passport, clients=_clients(crossref=cr),
ref_slug_by_key={"a": "slug-a", "b": "slug-b"})
assert len(outcomes) == 2
assert {o["citation_key"] for o in outcomes} == {"a", "b"}
assert {o["ref_slug"] for o in outcomes} == {"slug-a", "slug-b"}
def test_verify_passport_joins_anchors_by_ref_slug():
@@ -217,12 +264,15 @@ def test_verify_passport_joins_anchors_by_ref_slug():
from verification_gate import verify_passport
passport = {
"literature_corpus": [
_entry(citation_key="a", ref_slug="slug-a"),
_entry(citation_key="b", ref_slug="slug-b"),
_entry(citation_key="a"),
_entry(citation_key="b"),
]
}
ref_slug_by_key = {"a": "slug-a", "b": "slug-b"}
anchors = {"slug-a": _PAGE_ANCHOR} # slug-b absent → anchor_present False
outcomes = verify_passport(passport, clients=_clients(), anchors=anchors)
outcomes = verify_passport(
passport, clients=_clients(),
ref_slug_by_key=ref_slug_by_key, anchors=anchors)
by_key = {o["citation_key"]: o for o in outcomes}
assert by_key["a"]["anchor_present"] is True
assert by_key["b"]["anchor_present"] is False
@@ -230,8 +280,42 @@ def test_verify_passport_joins_anchors_by_ref_slug():
def test_verify_passport_empty_corpus():
from verification_gate import verify_passport
assert verify_passport({"literature_corpus": []}, clients=_clients()) == []
assert verify_passport({}, clients=_clients()) == []
assert verify_passport(
{"literature_corpus": []}, clients=_clients(), ref_slug_by_key={}) == []
assert verify_passport({}, clients=_clients(), ref_slug_by_key={}) == []
def test_verify_passport_raises_on_missing_join():
"""An entry whose citation_key has no joined ref_slug is a caller error: the
summary contract requires a non-null string ref_slug, so verify_passport
raises rather than emitting a contract-invalid summary (#332). No silent
default to citation_key or empty string."""
from verification_gate import verify_passport
passport = {"literature_corpus": [_entry(citation_key="orphan")]}
with pytest.raises(ValueError, match="orphan"):
verify_passport(passport, clients=_clients(), ref_slug_by_key={})
def test_verify_passport_outputs_are_schema_valid():
"""End-to-end: a schema-valid passport + a proper join map produces summaries
that ALL validate against citation_verification_summary.schema (the #332 P1
was that the normal passport path produced schema-invalid ref_slug: None)."""
import json
from jsonschema import Draft202012Validator
from verification_gate import verify_passport
schema = json.loads((
REPO_ROOT / "shared" / "contracts" / "passport"
/ "citation_verification_summary.schema.json"
).read_text(encoding="utf-8"))
passport = {"literature_corpus": [
_entry(citation_key="a"), _entry(citation_key="b", doi=None)]}
outcomes = verify_passport(
passport, clients=_clients(),
ref_slug_by_key={"a": "slug-a", "b": "slug-b"})
validator = Draft202012Validator(schema)
for o in outcomes:
errors = list(validator.iter_errors(o))
assert errors == [], f"summary must validate: {errors}"
def test_cache_argument_is_honest_not_silent_noop():
@@ -240,4 +324,19 @@ def test_cache_argument_is_honest_not_silent_noop():
thinking caching took effect."""
from verification_gate import verify_citation
with pytest.raises(NotImplementedError):
verify_citation(_entry(), _clients(), cache=object())
verify_citation(_entry(), _clients(), ref_slug=_DEFAULT_REF_SLUG, cache=object())
@pytest.mark.parametrize("bad", [None, "", 0, 123, ["slug"], {"slug": 1}])
def test_verify_citation_rejects_non_string_or_empty_ref_slug(bad):
"""#332 hardening: ref_slug is the prose-join key that the summary schema
requires as a non-empty string. verify_citation is the single emission point
(verify_passport routes through it), so the contract is enforced here once:
a non-string or empty ref_slug would stamp a summary that the caller cannot
join to any <!--ref:slug--> marker (empty) or that fails the schema's
type:string (non-string). Both are caller errors raise rather than emit a
join-broken / contract-invalid summary. Guarding at the entry also closes the
verify_passport `is None`-only gap (it now rejects "" and non-str too)."""
from verification_gate import verify_citation
with pytest.raises((ValueError, TypeError)):
verify_citation(_entry(), _clients(), ref_slug=bad)
+54 -10
View File
@@ -35,29 +35,73 @@ def _no_network_clients():
("crossref", "openalex", "semantic_scholar", "arxiv")}
def _entry(**overrides):
# Schema-valid corpus entry: all five required corpus fields, NO ref_slug
# (literature_corpus_entry.schema is additionalProperties:False — the ref_slug
# lives in writer prose, not the corpus).
base = {
"citation_key": "a", "title": "T",
"authors": [{"family": "Doe", "given": "J"}], "year": 2020,
"source_pointer": "kb://refs/a",
"doi": "10.5555/x", "obtained_via": "folder-scan",
}
base.update(overrides)
return base
def _write_passport(tmp_path, corpus):
p = tmp_path / "passport.yaml"
p.write_text(yaml.safe_dump({"literature_corpus": corpus}), encoding="utf-8")
return p
def test_cli_emits_json_summary(tmp_path, capsys):
from verify_passport import run
def test_cli_fixture_is_schema_valid():
"""Guard the guard: the _entry fixture must validate against
literature_corpus_entry.schema.json, so the CLI tests exercise the real
production shape (the #332 masking bug was a fixture carrying a forbidden
ref_slug field)."""
import json
from jsonschema import Draft202012Validator
schema = json.loads((
REPO_ROOT / "shared" / "contracts" / "passport"
/ "literature_corpus_entry.schema.json"
).read_text(encoding="utf-8"))
errors = list(Draft202012Validator(schema).iter_errors(_entry()))
assert errors == [], f"_entry fixture must be a valid corpus entry: {errors}"
# Production-shaped corpus entry: no anchor field (the anchor lives in writer
# prose, not in literature_corpus). The ad-hoc CLI has no prose document, so
# anchor_present is honestly False — a prose-join is a later-batch concern.
passport = _write_passport(tmp_path, [
{"citation_key": "a", "ref_slug": "slug-a", "title": "T",
"doi": "10.5/x", "obtained_via": "folder-scan"},
])
def test_cli_refuses_without_prose_join(tmp_path, capsys):
"""Default behavior: a passport-only CLI cannot honestly emit a
citation_verification_summary, because the summary contract requires the
prose-sourced ref_slug join that a passport alone does not carry. The CLI
refuses (nonzero) with a clear error rather than fabricating ref_slug (#332)."""
from verify_passport import run
passport = _write_passport(tmp_path, [_entry()])
rc = run([str(passport)], clients_factory=_no_network_clients)
assert rc != 0
err = capsys.readouterr().err
assert "ref_slug" in err or "prose" in err # explains why it refused
def test_cli_synthetic_ref_slug_uses_citation_key(tmp_path, capsys):
"""Explicit escape hatch: --synthetic-ref-slug citation_key synthesizes
ref_slug from citation_key so the ad-hoc tool can still produce output. It
warns on stderr that the result is diagnostic (not prose-join-safe) so no
downstream consumer mistakes it for a real prose join (#332)."""
from verify_passport import run
passport = _write_passport(tmp_path, [_entry(citation_key="a")])
rc = run([str(passport), "--synthetic-ref-slug", "citation_key"],
clients_factory=_no_network_clients)
assert rc == 0
out = json.loads(capsys.readouterr().out)
captured = capsys.readouterr()
out = json.loads(captured.out)
assert len(out) == 1
assert out[0]["citation_key"] == "a"
assert out[0]["ref_slug"] == "a" # synthesized from citation_key
assert out[0]["lookup_verified"] == "false" # id-keyed unmatched
assert out[0]["anchor_present"] is False # no prose anchor available
# the synthetic mode must say so on stderr (diagnostic, not prose-join-safe)
assert "synthetic" in captured.err.lower() or "diagnostic" in captured.err.lower()
def test_cli_missing_file_errors(tmp_path):
+77 -17
View File
@@ -2,14 +2,18 @@
"""verification_gate — citation existence verification API (Delta 5).
Public functions:
- verify_citation(entry, clients, cache=None) -> CitationVerificationOutcome
- verify_passport(passport, clients, cache=None) -> list[outcome]
- verify_citation(entry, clients, *, ref_slug, anchor=None, cache=None)
-> CitationVerificationOutcome
- verify_passport(passport, clients, *, ref_slug_by_key, anchors=None,
cache=None) -> list[outcome]
Composes the four resolvers (crossref / openalex / semantic_scholar / arxiv),
maps each resolver's execution to a {status, queried_by} outcome, derives the
3-class lookup_verified via the Delta 4 reducer (narrowed-false, C-V6(a)),
reads anchor_present from the v3.7.3 anchor marker, and stamps
verification_timestamp. Does NOT duplicate the v3.8 audit pipeline it composes
verification_timestamp. Both ref_slug and anchor are prose-sourced inputs joined
upstream by citation_key NEVER read off the corpus entry, whose schema forbids
them (#332). Does NOT duplicate the v3.8 audit pipeline — it composes
the same lower-layer resolvers and writes the unified summary schema (Delta 4).
The returned dict validates against
@@ -60,6 +64,15 @@ except ImportError: # pragma: no cover - dual-path import
_ANCHOR_PRESENT_KINDS = frozenset({"quote", "page", "section", "paragraph"})
def _is_valid_ref_slug(ref_slug: Any) -> bool:
"""A ref_slug is valid iff it is a non-empty string: the summary schema
requires ref_slug as a string, and an empty slug joins to no
<!--ref:slug--> prose marker. Single definition so verify_citation (the
emission point) and verify_passport (the join layer) agree on "bad slug"
(#332)."""
return isinstance(ref_slug, str) and bool(ref_slug)
def _outcome(status: str, queried_by: str | None,
response_summary: str | None = None) -> dict[str, Any]:
return {"status": status, "queried_by": queried_by,
@@ -122,14 +135,21 @@ def verify_citation(
entry: Mapping[str, Any],
clients: Mapping[str, Any],
*,
ref_slug: str,
anchor: Mapping[str, Any] | None = None,
cache=None,
) -> dict[str, Any]:
"""Verify one citation's existence across the four resolvers.
`entry` carries citation_key, ref_slug, title, optional doi / arxiv_id,
obtained_via. `clients` is a mapping {crossref, openalex, semantic_scholar,
arxiv} of resolver clients (injected so callers control network / cache).
`entry` carries citation_key, title, authors, year, source_pointer, optional
doi / arxiv_id, obtained_via. `clients` is a mapping {crossref, openalex,
semantic_scholar, arxiv} of resolver clients (injected so callers control
network / cache).
`ref_slug` is the writer-prose `<!--ref:slug-->` marker this citation renders
under, supplied by the caller never read off the corpus entry (same
provenance rule as `anchor`; the entry schema forbids the field, #332).
`anchor` is the v3.7.3 anchor marker ({kind, value}) for this citation's
ref_slug, already parsed from writer prose and joined upstream (None when no
anchor marker exists for the ref_slug). It is a SEPARATE input, not an entry
@@ -151,6 +171,17 @@ def verify_citation(
"cache-through at the verification_gate layer is not yet wired "
"(#182 Delta-2 follow-up); pass cache=None"
)
if not _is_valid_ref_slug(ref_slug):
# ref_slug is the prose-join key stamped verbatim into the summary, which
# the schema requires as a non-empty string. This is the single emission
# point (verify_passport routes through here), so the contract is enforced
# once: a non-string fails the schema's type:string, and an empty string
# joins to no <!--ref:slug--> marker. Either is a caller (prose-join) error
# — refuse rather than emit a contract-invalid / join-broken summary (#332).
raise ValueError(
f"ref_slug must be a non-empty string (the writer-prose join key), "
f"got {ref_slug!r}; corpus entries do not carry ref_slug (#332)"
)
if entry.get("obtained_via") == "manual":
# v3.7.3 manual exemption: no resolver runs — all four skipped (checked
# once here rather than re-checked inside each resolver helper).
@@ -170,7 +201,7 @@ def verify_citation(
}
return {
"citation_key": entry.get("citation_key"),
"ref_slug": entry.get("ref_slug"),
"ref_slug": ref_slug,
"lookup_verified": reduce_lookup_verified(resolver_outcomes),
"anchor_present": _anchor_present(anchor),
"verification_timestamp": datetime.now(timezone.utc).isoformat(),
@@ -182,19 +213,48 @@ def verify_passport(
passport: Mapping[str, Any],
clients: Mapping[str, Any],
*,
ref_slug_by_key: Mapping[str, str],
anchors: Mapping[str, Mapping[str, Any]] | None = None,
cache=None,
) -> list[dict[str, Any]]:
"""Batch helper: run verify_citation over every entry in the passport's
literature_corpus[]. `anchors` is the {ref_slug: anchor-marker} join map
parsed from writer prose (the v3.7.3 <!--anchor:kind:value--> markers); each
entry's anchor is looked up by its ref_slug (absent → anchor_present False).
Threading the join here keeps verify_citation a pure per-citation unit."""
literature_corpus[].
`ref_slug_by_key` is the {citation_key: ref_slug} join map: corpus entries are
keyed by citation_key, writer prose is keyed by ref_slug, and the join across
them is the caller's (Stage 4→5 pipeline's) responsibility the corpus entry
never carries ref_slug itself (#332). An entry whose citation_key has no joined
ref_slug raises ValueError rather than emitting a contract-invalid summary;
the per-entry summary contract requires a non-null string ref_slug, so a
missing join is a caller error, not a silently-defaulted field.
`anchors` is the {ref_slug: anchor-marker} join map parsed from writer prose
(the v3.7.3 <!--anchor:kind:value--> markers); each entry's anchor is looked
up by its (joined) ref_slug (absent anchor_present False). Threading both
joins here keeps verify_citation a pure per-citation unit.
ref_slug_by_key is 1:1 (one summary row per corpus entry); per-prose-occurrence
verification (one entry under several ref slugs) is a different API shape, out
of scope here.
"""
corpus = passport.get("literature_corpus") or []
anchors = anchors or {}
return [
verify_citation(
entry, clients, anchor=anchors.get(entry.get("ref_slug")),
cache=cache)
for entry in corpus
]
outcomes: list[dict[str, Any]] = []
for entry in corpus:
citation_key = entry.get("citation_key")
ref_slug = ref_slug_by_key.get(citation_key)
if not _is_valid_ref_slug(ref_slug):
# Missing join (None) OR a present-but-empty/non-string slug — both
# fail the per-summary contract. Caught here (not just in
# verify_citation) so the error names the offending citation_key,
# which the per-citation layer doesn't have (#332).
raise ValueError(
f"no valid ref_slug joined for citation_key {citation_key!r} "
f"(got {ref_slug!r}): the citation_key→ref_slug prose join must "
"cover every corpus entry with a non-empty string "
"(corpus entries do not carry ref_slug; #332)"
)
outcomes.append(verify_citation(
entry, clients, ref_slug=ref_slug,
anchor=anchors.get(ref_slug), cache=cache))
return outcomes
+37 -5
View File
@@ -8,10 +8,13 @@ literature_corpus[], and prints the list of per-citation summaries as JSON. A
standalone entry point for ad-hoc verification, separate from the Stage 4->5
audit pipeline.
Anchor note: the v3.7.3 anchor lives in writer prose (the <!--anchor:...-->
markers), not in literature_corpus. This ad-hoc CLI has no prose document to
join against, so every `anchor_present` is False. The prose-marker join (passing
an {ref_slug: anchor} map into verify_passport) is wired by the Stage 4->5
ref_slug note (#332): both the ref_slug AND the anchor live in writer prose (the
<!--ref:slug--> / <!--anchor:...--> markers), not in literature_corpus. The
summary contract REQUIRES a non-null string ref_slug, so a passport-only CLI
cannot honestly emit a summary by default it REFUSES (nonzero exit). Pass
`--synthetic-ref-slug citation_key` to synthesize ref_slug from citation_key for
DIAGNOSTIC output (warned on stderr, not a real prose join). The real
{citation_key: ref_slug} + {ref_slug: anchor} joins are wired by the Stage 4->5
pipeline / formatter batch, not by this standalone tool.
Spec: docs/design/2026-05-21-v3.10-182-promote-citation-gate-spec.md §2 Delta 5.
@@ -60,6 +63,11 @@ def run(argv: list[str] | None = None, *, clients_factory=_real_clients) -> int:
description="Verify citation existence across a Material Passport.",
)
parser.add_argument("passport", help="Path to the passport YAML file.")
parser.add_argument(
"--synthetic-ref-slug", choices=["citation_key"], default=None,
help="Synthesize ref_slug from each entry's citation_key for DIAGNOSTIC "
"output (the tool refuses by default; this is not a real prose "
"join, #332).")
args = parser.parse_args(argv)
path = Path(args.passport)
@@ -74,7 +82,31 @@ def run(argv: list[str] | None = None, *, clients_factory=_real_clients) -> int:
file=sys.stderr)
return 1
outcomes = verify_passport(passport, clients=clients_factory())
# Refuse by default — a passport alone carries no prose <!--ref:slug--> join,
# so it cannot produce a contract-valid summary (see module docstring, #332).
corpus = passport.get("literature_corpus") or []
if args.synthetic_ref_slug is None:
print(
"[verify_passport ERROR] cannot emit citation_verification_summary "
"from a passport alone: ref_slug is a prose-sourced join "
"(<!--ref:slug--> markers) that a passport does not carry. Run the "
"Stage 4->5 pipeline (which supplies the prose join), or pass "
"--synthetic-ref-slug citation_key for diagnostic output.",
file=sys.stderr)
return 2
# synthetic mode: ref_slug := citation_key. Diagnostic only.
ref_slug_by_key = {
e.get("citation_key"): e.get("citation_key") for e in corpus
}
print(
"[verify_passport WARNING] --synthetic-ref-slug citation_key: ref_slug "
"synthesized from citation_key. Output is DIAGNOSTIC, not a real prose "
"join — do NOT feed it to a consumer that expects prose-joined ref_slugs.",
file=sys.stderr)
outcomes = verify_passport(
passport, clients=clients_factory(), ref_slug_by_key=ref_slug_by_key)
print(json.dumps(outcomes, indent=2))
return 0