mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
Extend the v3.9.0 cross-index triangulation contamination matrix from three indexes (k=0..3) to four (adds arxiv; k=0..4). Data layer shipped earlier (#313/#315); this is the rendering layer only — advisory tokens, no gate change. - orchestrator finalizer matrix: k=0..4, four new advisory tokens (CONTAMINATED-ARXIV-UNMATCHED, -QUADRANGULATION-UNMATCHED + 2 PREPRINT compositions). arxiv carve-out fires only at k=1/k_max=1 (mirrors the s2 legacy carve-out); QUADRANGULATION reserved for k=4/k_max=4; k=3/k_max=4 is PARTIAL-UNMATCH. Terminal gate / refusal list unchanged (R-L3-2-E). - formatter pass-through allowlist 9 -> 13 (refusal semantics unchanged). - lint (check_v3_9_0_triangulation.py) rule 1 upgraded from subsection token presence to a matrix-row oracle: each Delta-1 token must sit on the table row carrying its exact (k, k_max) cell, so deleting/mistokening an operational row fails even when the same token survives in surrounding prose. The cell map is the single source of truth for the Delta-1 token set. - tests: row-oracle mutation (row-only mistoken must fail; prose-only edit must pass), arxiv carve-out pinned, allowlist set-equality 13. - CI manifest: backfill 5 data-layer test entries shipped earlier but never wired into the runner. Full suite 2078 passed / 3 skipped; lint + manifest drift guard green. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Edward Cheng-I Wu
committed by
GitHub
GitHub
parent
2a2df02c3e
commit
a0dca2c9b7
@@ -309,7 +309,7 @@ When refusing, surface the unresolved markers to the user with their per-section
|
||||
- v3.8 HIGH-WARN-CLAIM-AUDIT-ANCHORLESS (rule 9): defense-in-depth surface — the v3.7.3 finalizer should have caught this upstream. Remediation: same as MED-WARN-NO-LOCATOR (rule 3) — emit a `<!--anchor:<kind>:<value>-->` with `<kind>` ≠ `none`. `/ars-mark-read` does NOT clear this.
|
||||
- v3.8 HIGH-WARN-CONSTRAINT-VIOLATION-UNCITED (rule 10): same remediation as rule 7 (revise / drop / re-issue manifest). The entry-type split between cited (rule 7, claim_audit_result) and uncited (rule 10, constraint_violation) is a schema-integrity artifact only; the user-facing fix is identical.
|
||||
|
||||
**Contamination annotations (`CONTAMINATED-PREPRINT`, `CONTAMINATED-UNMATCHED`, `CONTAMINATED-PREPRINT+UNMATCHED`, `CONTAMINATED-COVERAGE-NOISE`, `CONTAMINATED-PARTIAL-UNMATCH`, `CONTAMINATED-TRIANGULATION-UNMATCHED`, `CONTAMINATED-PREPRINT+COVERAGE-NOISE`, `CONTAMINATED-PREPRINT+PARTIAL-UNMATCH`, `CONTAMINATED-PREPRINT+TRIANGULATION-UNMATCHED`) on `ok` or `LOW-WARN` markers DO NOT trigger refusal.** They are advisory per v3.5 Collaboration Depth Observer precedent + v3.7.3 R-L3-2-A + v3.9.0 R-L3-2-E — surface them in the output package's `provenance_summary.md`, but do not block the conversion. v3.9.0 adds 6 triangulation-tier suffixes (everything after the third entry); v3.7.3 added the first three. The advisory **suffix** never triggers refusal — the v3.9.0 pass-through allowlist is unchanged. (v3.10 separately adds rule 11, a generic `severity=HIGH-BLOCK` refusal: when a strict `terminal_policies` promotes a k=3 signal, the finalizer co-emits a `TERMINAL-BLOCK` token ALONGSIDE the advisory suffix; rule 11 refuses on that token, NOT on the suffix. The suffix stays on the advisory pass-through allowlist; the refusal-list is extended only by the one generic rule, never per-suffix — R-L3-2-E.)
|
||||
**Contamination annotations (`CONTAMINATED-PREPRINT`, `CONTAMINATED-UNMATCHED`, `CONTAMINATED-PREPRINT+UNMATCHED`, `CONTAMINATED-COVERAGE-NOISE`, `CONTAMINATED-PARTIAL-UNMATCH`, `CONTAMINATED-TRIANGULATION-UNMATCHED`, `CONTAMINATED-PREPRINT+COVERAGE-NOISE`, `CONTAMINATED-PREPRINT+PARTIAL-UNMATCH`, `CONTAMINATED-PREPRINT+TRIANGULATION-UNMATCHED`, `CONTAMINATED-ARXIV-UNMATCHED`, `CONTAMINATED-QUADRANGULATION-UNMATCHED`, `CONTAMINATED-PREPRINT+ARXIV-UNMATCHED`, `CONTAMINATED-PREPRINT+QUADRANGULATION-UNMATCHED`) on `ok` or `LOW-WARN` markers DO NOT trigger refusal.** They are advisory per v3.5 Collaboration Depth Observer precedent + v3.7.3 R-L3-2-A + v3.9.0 R-L3-2-E — surface them in the output package's `provenance_summary.md`, but do not block the conversion. v3.7.3 added the first three; v3.9.0 added the next 6 triangulation-tier suffixes; the v3.10/v3.11 Delta-1 arXiv four-index extension adds the final 4 (`CONTAMINATED-ARXIV-UNMATCHED`, `CONTAMINATED-QUADRANGULATION-UNMATCHED`, and their two PREPRINT compositions). The advisory **suffix** never triggers refusal — the pass-through allowlist grows in lockstep with the finalizer (3 → 9 → 13), but the **refusal semantics are unchanged**: no contamination suffix has ever been or will be added to the refusal list (R-L3-2-E). (v3.10 separately adds rule 11, a generic `severity=HIGH-BLOCK` refusal: when a strict `terminal_policies` promotes a k=3 signal, the finalizer co-emits a `TERMINAL-BLOCK` token ALONGSIDE the advisory suffix; rule 11 refuses on that token, NOT on the suffix. The suffix stays on the advisory pass-through allowlist; the refusal-list is extended only by the one generic rule, never per-suffix — R-L3-2-E.)
|
||||
|
||||
## Cite-Time Terminal Policy Gate (v3.10) — STAMP-ONLY freshness + rule 11
|
||||
|
||||
|
||||
@@ -761,9 +761,9 @@ Spec: `docs/design/2026-05-17-ars-v3.9.0-cross-index-triangulation-measurement-s
|
||||
|
||||
v3.9.0 extends the v3.7.3 contamination annotation channel with three new lookup-derived suffix shapes. The base 5-cell matrix is unchanged. The annotation rule expands as follows:
|
||||
|
||||
**Trigger:** annotation fires when (base resolution ∈ {`ok`, `LOW-WARN`}) AND (`preprint_post_llm_inflection` is true OR any of `semantic_scholar_unmatched` / `openalex_unmatched` / `crossref_unmatched` is true). Entries with `contamination_signals` present but all fields false (computed-clean) produce no suffix — v3.7.3 behavior preserved.
|
||||
**Trigger:** annotation fires when (base resolution ∈ {`ok`, `LOW-WARN`}) AND (`preprint_post_llm_inflection` is true OR any of `semantic_scholar_unmatched` / `openalex_unmatched` / `crossref_unmatched` / `arxiv_unmatched` is true). Entries with `contamination_signals` present but all fields false (computed-clean) produce no suffix — v3.7.3 behavior preserved.
|
||||
|
||||
**Compute k (triangulation count):** k = count of `*_unmatched` fields with value `true`, over fields that are present. Absent fields are excluded (per spec R-L3-2-C: absent ≠ false). k_max = count of `*_unmatched` fields that are present (0-3).
|
||||
**Compute k (triangulation count):** k = count of `*_unmatched` fields with value `true`, over fields that are present. Absent fields are excluded (per spec R-L3-2-C: absent ≠ false). k_max = count of `*_unmatched` fields that are present (0-4 — the v3.10/v3.11 Delta-1 `arxiv_unmatched` field is the fourth index; `arxiv_unmatched` is absent on citations with no arXiv ID, so k_max stays ≤ 3 for those, per the v3.9.0 absent≠false rule).
|
||||
|
||||
**Suffix shape table:**
|
||||
|
||||
@@ -773,24 +773,36 @@ v3.9.0 extends the v3.7.3 contamination annotation channel with three new lookup
|
||||
| `ok` / `LOW-WARN` | true | 0 | any | — | `CONTAMINATED-PREPRINT` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 1 | 1 | `semantic_scholar_unmatched` | `CONTAMINATED-UNMATCHED` (v3.7.3 legacy) |
|
||||
| `ok` / `LOW-WARN` | true | 1 | 1 | `semantic_scholar_unmatched` | `CONTAMINATED-PREPRINT+UNMATCHED` (v3.7.3 legacy) |
|
||||
| `ok` / `LOW-WARN` | false / absent | 1 | 1 | `arxiv_unmatched` | `CONTAMINATED-ARXIV-UNMATCHED` (v3.10/v3.11 Delta-1) |
|
||||
| `ok` / `LOW-WARN` | true | 1 | 1 | `arxiv_unmatched` | `CONTAMINATED-PREPRINT+ARXIV-UNMATCHED` (v3.10/v3.11 Delta-1) |
|
||||
| `ok` / `LOW-WARN` | false / absent | 1 | 1 | `openalex_unmatched` or `crossref_unmatched` | `CONTAMINATED-COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | true | 1 | 1 | `openalex_unmatched` or `crossref_unmatched` | `CONTAMINATED-PREPRINT+COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 1 | 2-3 | — | `CONTAMINATED-COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | true | 1 | 2-3 | — | `CONTAMINATED-PREPRINT+COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 2 | 2-3 | — | `CONTAMINATED-PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | true | 2 | 2-3 | — | `CONTAMINATED-PREPRINT+PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 1 | 2-4 | — | `CONTAMINATED-COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | true | 1 | 2-4 | — | `CONTAMINATED-PREPRINT+COVERAGE-NOISE` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 2 | 2-4 | — | `CONTAMINATED-PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | true | 2 | 2-4 | — | `CONTAMINATED-PREPRINT+PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 3 | 3 | — | `CONTAMINATED-TRIANGULATION-UNMATCHED` |
|
||||
| `ok` / `LOW-WARN` | true | 3 | 3 | — | `CONTAMINATED-PREPRINT+TRIANGULATION-UNMATCHED` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 3 | 4 | — | `CONTAMINATED-PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | true | 3 | 4 | — | `CONTAMINATED-PREPRINT+PARTIAL-UNMATCH` |
|
||||
| `ok` / `LOW-WARN` | false / absent | 4 | 4 | — | `CONTAMINATED-QUADRANGULATION-UNMATCHED` |
|
||||
| `ok` / `LOW-WARN` | true | 4 | 4 | — | `CONTAMINATED-PREPRINT+QUADRANGULATION-UNMATCHED` |
|
||||
|
||||
**Composition order:** `PREPRINT` token first, triangulation token second, joined by `+`. The canonical token order list is `[PREPRINT, UNMATCHED | COVERAGE-NOISE | PARTIAL-UNMATCH | TRIANGULATION-UNMATCHED]`.
|
||||
**v3.10/v3.11 Delta-1 extension (arXiv fourth index):** `arxiv_unmatched` is the fourth lookup field, present only on citations carrying an arXiv ID (absent ≠ false). Two new single-named tiers join the v3.9.0 tiers:
|
||||
- **`CONTAMINATED-ARXIV-UNMATCHED` (k=1, k_max=1, present field = `arxiv_unmatched`)** — the arxiv-only carve-out, mirroring the `semantic_scholar_unmatched` legacy carve-out exactly: it fires ONLY when arxiv is the SOLE present-and-unmatched index. An arxiv-only k=1 with k_max ≥ 2 (arxiv unmatched, other present indexes matched) stays `CONTAMINATED-COVERAGE-NOISE` like every other k=1 k_max ≥ 2 case — "single-index" means k_max=1, not merely k=1 (consistent with the v3.9.0 s2 carve-out being k_max=1-only).
|
||||
- **`CONTAMINATED-QUADRANGULATION-UNMATCHED` (k=4, k_max=4)** — all four indexes unmatched, the four-index analogue of `CONTAMINATED-TRIANGULATION-UNMATCHED` (which stays k=3 k_max=3, all-three-unmatched). A k=3 k_max=4 (three of four unmatched) is `CONTAMINATED-PARTIAL-UNMATCH`, NOT triangulation — the strong all-N name is reserved for k = k_max = N (the v3.9.0 "observation not inferred cause" rule extended to N=4).
|
||||
|
||||
**Gate semantics:** All v3.9.0 suffixes are advisory. The terminal gate refusal list is NOT extended. `formatter_agent.md` pass-through allowlist MUST extend from 3 v3.7.3 suffixes to 9 (3 legacy + 6 v3.9.0) per R-L3-2-E. `/ars-mark-read` behavior is unchanged.
|
||||
**Composition order:** `PREPRINT` token first, triangulation token second, joined by `+`. The canonical token order list is `[PREPRINT, UNMATCHED | ARXIV-UNMATCHED | COVERAGE-NOISE | PARTIAL-UNMATCH | TRIANGULATION-UNMATCHED | QUADRANGULATION-UNMATCHED]`.
|
||||
|
||||
**Gate semantics:** All v3.9.0 AND Delta-1 suffixes are advisory. The terminal gate refusal list is NOT extended. `formatter_agent.md` pass-through allowlist MUST extend from 3 v3.7.3 suffixes to 9 (v3.9.0) to 13 (Delta-1: + the 4 arXiv tokens) per R-L3-2-E. `/ars-mark-read` behavior is unchanged.
|
||||
|
||||
Example markers:
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-COVERAGE-NOISE-->` — single-index unmatched, k_max ≥ 2.
|
||||
- `<!--ref:smith2024 ok CONTAMINATED-PARTIAL-UNMATCH-->` — two-of-three unmatched.
|
||||
- `<!--ref:smith2024 ok CONTAMINATED-PARTIAL-UNMATCH-->` — two-of-three (or three-of-four) unmatched.
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-TRIANGULATION-UNMATCHED-->` — all three indexes unmatched.
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-PREPRINT+TRIANGULATION-UNMATCHED-->` — preprint heuristic + k=3.
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-ARXIV-UNMATCHED-->` — arxiv-only (k_max=1) unmatched.
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-QUADRANGULATION-UNMATCHED-->` — all four indexes unmatched.
|
||||
- `<!--ref:smith2024 LOW-WARN CONTAMINATED-PREPRINT+QUADRANGULATION-UNMATCHED-->` — preprint heuristic + k=4.
|
||||
|
||||
## Cite-Time Provenance Finalizer — v3.10 extension (terminal policy layer)
|
||||
|
||||
|
||||
@@ -137,3 +137,25 @@ path = "scripts/test_check_v3_10_134_write_scope.py"
|
||||
[[pytest]]
|
||||
id = "v3.11-citation-existence-policy"
|
||||
path = "scripts/test_citation_existence_policy.py"
|
||||
|
||||
# #182 Delta 4+5 data-layer tests (shipped via #313/#315 but not wired into the
|
||||
# manifest at the time; backfilled here so the runner actually executes them).
|
||||
[[pytest]]
|
||||
id = "v3.10-182-citation-verification-summary"
|
||||
path = "scripts/test_citation_verification_summary.py"
|
||||
|
||||
[[pytest]]
|
||||
id = "v3.10-182-verification-gate"
|
||||
path = "scripts/test_verification_gate.py"
|
||||
|
||||
[[pytest]]
|
||||
id = "v3.10-182-arxiv-client"
|
||||
path = "scripts/test_arxiv_client.py"
|
||||
|
||||
[[pytest]]
|
||||
id = "v3.10-182-verification-cache"
|
||||
path = "scripts/test_verification_cache.py"
|
||||
|
||||
[[pytest]]
|
||||
id = "v3.10-182-verify-passport-cli"
|
||||
path = "scripts/test_verify_passport_cli.py"
|
||||
|
||||
@@ -1,12 +1,21 @@
|
||||
#!/usr/bin/env python3
|
||||
"""v3.9.0 spec lint: verify finalizer suffix table contract + formatter allowlist + refusal-list guard.
|
||||
"""Cross-version contamination-suffix lint: finalizer suffix table contract +
|
||||
formatter allowlist + refusal-list guard.
|
||||
|
||||
Per spec v3.9.0 §3.8 rules 1-7:
|
||||
Rule 1 — marker syntax: 3 new v3.9.0 markers present in finalizer suffix table
|
||||
NOTE ON NAMING: this file is named for v3.9.0 (where the triangulation matrix
|
||||
landed) but is the CANONICAL contamination-suffix oracle across versions. It now
|
||||
also guards the v3.10/v3.11 Delta-1 arXiv four-index tokens (EXPECTED_DELTA1_ROWS).
|
||||
The v3.9.0 name is historical; renaming would churn spec-consistency.yml + the CI
|
||||
manifest + the test companion for no benefit.
|
||||
|
||||
Per spec v3.9.0 §3.8 rules 1-7, extended by #182 Delta 1:
|
||||
Rule 1 — matrix-row oracle: the 3 v3.9.0 markers present in the finalizer
|
||||
suffix-table rows; the 4 Delta-1 arXiv markers (incl. PREPRINT
|
||||
compositions) each pinned to their exact (k, k_max) table cell
|
||||
Rule 2 — preprint composition order: PREPRINT before triangulation token
|
||||
Rule 3 — v3.7.3 legacy compat: k=1 k_max=1 S2 row → CONTAMINATED-UNMATCHED (not COVERAGE-NOISE)
|
||||
Rule 4 — no *-BLOCK tokens in v3.9.0 finalizer subsection (v3.10 policy-layer scope only)
|
||||
Rule 5 — formatter pass-through allowlist set-equality (R-L3-2-E lint)
|
||||
Rule 4 — no *-BLOCK tokens in finalizer subsection (v3.10 policy-layer scope only)
|
||||
Rule 5 — formatter pass-through allowlist set-equality, 13 tokens (R-L3-2-E lint)
|
||||
Rule 6 — refusal-list-unchanged guard
|
||||
Rule 7 — CI integration (done in spec-consistency.yml)
|
||||
|
||||
@@ -41,7 +50,14 @@ EXPECTED_NEW_V3_9_0_SUFFIXES = {
|
||||
"CONTAMINATED-TRIANGULATION-UNMATCHED",
|
||||
}
|
||||
|
||||
# Canonical 9-suffix allowlist per spec v3.9.0 §3.8 rule 5.
|
||||
# The four v3.10/v3.11 Delta-1 markers are NOT declared as a bare set here — they
|
||||
# are the keys of EXPECTED_DELTA1_ROWS below, which pins each to its exact (k, k_max)
|
||||
# matrix cell. A separate presence-only set would be a parallel structure that could
|
||||
# drift out of sync (and presence is strictly weaker than the cell assertion anyway),
|
||||
# so the cell map is the single source of truth for the Delta-1 token set.
|
||||
|
||||
# Canonical 13-suffix allowlist (set-equality oracle, rule 5). This is the full
|
||||
# contamination-suffix set across versions: 3 v3.7.3 legacy + 6 v3.9.0 + 4 Delta-1.
|
||||
EXPECTED_ALLOWLIST_TOKENS = {
|
||||
# v3.7.3 legacy (3)
|
||||
"CONTAMINATED-PREPRINT",
|
||||
@@ -54,6 +70,11 @@ EXPECTED_ALLOWLIST_TOKENS = {
|
||||
"CONTAMINATED-PREPRINT+COVERAGE-NOISE",
|
||||
"CONTAMINATED-PREPRINT+PARTIAL-UNMATCH",
|
||||
"CONTAMINATED-PREPRINT+TRIANGULATION-UNMATCHED",
|
||||
# v3.10/v3.11 Delta-1 arXiv four-index (4)
|
||||
"CONTAMINATED-ARXIV-UNMATCHED",
|
||||
"CONTAMINATED-QUADRANGULATION-UNMATCHED",
|
||||
"CONTAMINATED-PREPRINT+ARXIV-UNMATCHED",
|
||||
"CONTAMINATED-PREPRINT+QUADRANGULATION-UNMATCHED",
|
||||
}
|
||||
|
||||
|
||||
@@ -78,15 +99,57 @@ def extract_v3_9_0_finalizer_subsection(orchestrator_text: str) -> str:
|
||||
return "\n".join(collected)
|
||||
|
||||
|
||||
# Matrix-row contract for the 4 Delta-1 tokens: each must appear in a suffix-table
|
||||
# row carrying its exact (k, k_max) cell. The bullet prose + example markers below
|
||||
# the table also mention these tokens, so a subsection-wide token scan would pass
|
||||
# even if the operational row were deleted/mistokened. Rule 1 therefore asserts the
|
||||
# matrix row, not mere subsection presence — the matrix table IS the contract.
|
||||
# Column order in the suffix table: | base | preprint | k | k_max | present | suffix |
|
||||
# so "| <k> | <k_max> |" locates the (k, k_max) cell exactly (same idiom as rule 3).
|
||||
EXPECTED_DELTA1_ROWS = {
|
||||
"CONTAMINATED-ARXIV-UNMATCHED": "| 1 | 1 |",
|
||||
"CONTAMINATED-PREPRINT+ARXIV-UNMATCHED": "| 1 | 1 |",
|
||||
"CONTAMINATED-QUADRANGULATION-UNMATCHED": "| 4 | 4 |",
|
||||
"CONTAMINATED-PREPRINT+QUADRANGULATION-UNMATCHED": "| 4 | 4 |",
|
||||
}
|
||||
|
||||
|
||||
def check_marker_syntax(subsection_text: str) -> list[str]:
|
||||
"""Rule 1: all 3 new v3.9.0 markers appear as backtick-quoted suffixes in subsection."""
|
||||
tokens = set(re.findall(r"`(CONTAMINATED-[A-Z+\-]+)`", subsection_text))
|
||||
missing = EXPECTED_NEW_V3_9_0_SUFFIXES - tokens
|
||||
"""Rule 1 (matrix-row oracle): each v3.9.0 + Delta-1 marker must appear as a
|
||||
backtick-quoted suffix in a TABLE ROW, and each Delta-1 marker must sit in the
|
||||
row carrying its exact (k, k_max) cell.
|
||||
|
||||
Scanning only `|`-delimited rows (not the explanatory bullets/examples below the
|
||||
table) is what makes this a contract oracle rather than a presence check: the
|
||||
matrix table IS the prompt contract, and deleting an operational row must fail
|
||||
even though the same token still appears in the surrounding prose.
|
||||
"""
|
||||
table_rows = [ln for ln in subsection_text.splitlines() if "|" in ln]
|
||||
table_text = "\n".join(table_rows)
|
||||
row_tokens = set(re.findall(r"`(CONTAMINATED-[A-Z+\-]+)`", table_text))
|
||||
failures = []
|
||||
# v3.9.0 tokens span k_max RANGES across multiple rows (e.g. COVERAGE-NOISE is
|
||||
# k=1 k_max=1 AND k=1 k_max=2-4), so they get a presence-in-table-rows check, not
|
||||
# a single-cell assertion (same family-by-family treatment as rule 3's s2 rows).
|
||||
missing = EXPECTED_NEW_V3_9_0_SUFFIXES - row_tokens
|
||||
if missing:
|
||||
return [
|
||||
f"rule 1 (marker syntax): missing in v3.9.0 finalizer subsection: {sorted(missing)}"
|
||||
]
|
||||
return []
|
||||
failures.append(
|
||||
f"rule 1 (marker syntax): missing v3.9.0 markers in finalizer "
|
||||
f"suffix-table rows: {sorted(missing)}"
|
||||
)
|
||||
# Delta-1 tokens are POINT cells, so the cell assertion is their sole check — it
|
||||
# is strictly stronger than presence (a token on the wrong tier, or absent
|
||||
# entirely, fails here), and EXPECTED_DELTA1_ROWS is their single source of truth.
|
||||
for token, cell in EXPECTED_DELTA1_ROWS.items():
|
||||
on_correct_row = any(
|
||||
f"`{token}`" in ln and cell in ln for ln in table_rows
|
||||
)
|
||||
if not on_correct_row:
|
||||
failures.append(
|
||||
f"rule 1 (matrix row): `{token}` is not on a suffix-table row "
|
||||
f"carrying its required {cell} (k, k_max) cell"
|
||||
)
|
||||
return failures
|
||||
|
||||
|
||||
def check_preprint_composition_order(subsection_text: str) -> list[str]:
|
||||
|
||||
@@ -19,7 +19,8 @@ def run_lint(args: list[str] | None = None) -> subprocess.CompletedProcess:
|
||||
|
||||
|
||||
def test_lint_passes_on_clean_repo():
|
||||
"""After T7 lands (formatter has all 9 allowlist entries), lint passes."""
|
||||
"""The real repo carries all 13 allowlist entries (9 v3.9.0 + 4 Delta-1) and
|
||||
lint passes."""
|
||||
result = run_lint()
|
||||
assert result.returncode == 0, f"Lint failed: stderr={result.stderr}\nstdout={result.stdout}"
|
||||
|
||||
@@ -224,6 +225,86 @@ def test_rule4_high_block_injection_fails(tmp_path):
|
||||
assert "rule 4" in result.stderr.lower() or "block" in result.stderr.lower()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Delta-1 (v3.10/v3.11) — arXiv four-index extension
|
||||
# ---------------------------------------------------------------------------
|
||||
# (Clean-repo pass for all 13 allowlist tokens is covered by
|
||||
# test_lint_passes_on_clean_repo above — not duplicated here.)
|
||||
|
||||
def test_delta1_missing_arxiv_allowlist_token_fails(tmp_path):
|
||||
"""Drop CONTAMINATED-ARXIV-UNMATCHED from the formatter allowlist — set-equality
|
||||
(rule 5) must fail (the Delta-1 token is load-bearing, not incidentally present)."""
|
||||
formatter = REPO_ROOT / "academic-paper/agents/formatter_agent.md"
|
||||
content = formatter.read_text()
|
||||
# Remove the bare ARXIV token from the allowlist sentence (keep the PREPRINT+ARXIV
|
||||
# composite so the failure is specifically the bare-token drop).
|
||||
broken = content.replace("`CONTAMINATED-ARXIV-UNMATCHED`, ", "", 1)
|
||||
p = tmp_path / "formatter_agent.md"
|
||||
p.write_text(broken)
|
||||
result = run_lint(["--formatter-path", str(p)])
|
||||
assert result.returncode == 1
|
||||
assert "CONTAMINATED-ARXIV-UNMATCHED" in result.stderr
|
||||
|
||||
|
||||
def test_delta1_missing_quadrangulation_matrix_row_fails(tmp_path):
|
||||
"""Mistoken CONTAMINATED-QUADRANGULATION-UNMATCHED in its MATRIX ROW ONLY (k=4
|
||||
k_max=4) while the explanatory bullet prose keeps the token verbatim — rule 1
|
||||
must still fail.
|
||||
|
||||
This is the mutation rule 1 exists to catch: the matrix table IS the prompt
|
||||
contract, so deleting/mistokening an operational row must fail even though the
|
||||
same token survives in surrounding prose. (A blanket str.replace of every
|
||||
occurrence would mask this — it strips the prose too, so a subsection-wide token
|
||||
scan would fail for the wrong reason rather than because the row is gone.)"""
|
||||
orch = REPO_ROOT / "academic-pipeline/agents/pipeline_orchestrator_agent.md"
|
||||
content = orch.read_text()
|
||||
# Mistoken the suffix in the k=4 k_max=4 (non-preprint) table row only.
|
||||
matrix_cell = "| 4 | 4 | — | `CONTAMINATED-QUADRANGULATION-UNMATCHED`"
|
||||
broken_cell = "| 4 | 4 | — | `CONTAMINATED-QUADXXX`"
|
||||
assert matrix_cell in content, "fixture drift: k=4 k_max=4 row shape changed"
|
||||
broken = content.replace(matrix_cell, broken_cell, 1)
|
||||
# The bullet prose still carries the verbatim token, proving the failure comes
|
||||
# from the row scan, not from blanket token absence.
|
||||
assert "`CONTAMINATED-QUADRANGULATION-UNMATCHED`" in broken
|
||||
p = tmp_path / "orch.md"
|
||||
p.write_text(broken)
|
||||
result = run_lint(["--orchestrator-path", str(p)])
|
||||
assert result.returncode == 1
|
||||
assert "QUADRANGULATION" in result.stderr or "Delta-1" in result.stderr
|
||||
|
||||
|
||||
def test_delta1_token_in_prose_only_still_passes_is_not_contract(tmp_path):
|
||||
"""Deleting a Delta-1 token from the bullet PROSE while keeping its matrix row
|
||||
intact must still PASS — the prose is documentation, not the contract. This is
|
||||
the negative twin of the row-oracle test: it proves rule 1 keys off the matrix
|
||||
row, not subsection-wide presence (so prose edits don't cause false failures)."""
|
||||
orch = REPO_ROOT / "academic-pipeline/agents/pipeline_orchestrator_agent.md"
|
||||
content = orch.read_text()
|
||||
# The arxiv carve-out bullet starts with the bolded token; neutralise the bolded
|
||||
# prose mention without touching its `| 1 | 1 |` matrix row.
|
||||
prose_bullet = "- **`CONTAMINATED-ARXIV-UNMATCHED` (k=1, k_max=1"
|
||||
assert prose_bullet in content, "fixture drift: arxiv carve-out bullet shape changed"
|
||||
broken = content.replace(prose_bullet, "- **The arxiv carve-out (k=1, k_max=1", 1)
|
||||
# Matrix row for the token is untouched.
|
||||
assert "| 1 | 1 | `arxiv_unmatched` | `CONTAMINATED-ARXIV-UNMATCHED`" in broken
|
||||
p = tmp_path / "orch.md"
|
||||
p.write_text(broken)
|
||||
result = run_lint(["--orchestrator-path", str(p)])
|
||||
assert result.returncode == 0, f"prose-only edit should pass: stderr={result.stderr}"
|
||||
|
||||
|
||||
def test_delta1_arxiv_is_kmax1_carveout_documented():
|
||||
"""The arxiv carve-out semantics (fires only at k_max=1, not any k=1) must be
|
||||
documented verbatim in the orchestrator subsection so a future edit cannot
|
||||
silently widen it (the resolved reading of the ambiguous 'single-index' phrase:
|
||||
single-index means k_max=1, not merely k=1). This pins the prompt contract."""
|
||||
orch = (REPO_ROOT / "academic-pipeline/agents/pipeline_orchestrator_agent.md"
|
||||
).read_text()
|
||||
# The disambiguating sentence must be present.
|
||||
assert "single-index" in orch and "k_max=1" in orch
|
||||
assert "CONTAMINATED-COVERAGE-NOISE" in orch # the k=1 k_max>=2 arxiv fallback
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Invocation error handling
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
Reference in New Issue
Block a user