Files
imbad0202__academic-researc…/scripts/test_render_claim_standing_view.py
Edward Cheng-I Wu bb13f7818c feat(probe): stance runner and presentation renderer for #655 Track B (#732)
* feat(probe): stance runner and presentation renderer for #655 Track B

The stance runner consumes a finalized ledger under a
retrieval_plus_stance consent: transport identity and prompt-contract
version are consent-bound, the judge grammar is closed four-line output
with a verbatim-substring 25-word evidence quotation sealed as a
verified_exact_match evidence row, deviations become
not_checked/parse_error with raw output retained, and transport failures
map to judge_timeout/judge_error. validate_stance_record is the
contract-named semantic verifier (identity hashes, distribution sum,
exact selected-family coverage, per-evidence-row hash replay); its CLI
refuses until a live provider adapter ships with its own consent
surface. The renderer produces the three-part §5.3 view with §5.4
bounded vocabulary, fixed empty wording, all-selected denominator with
performed-only marked secondary, inert escaping, a leading UNMEASURED
banner, and stale-record refusal. Package-first imports unify exception
identity across the claim-standing tools. 17 new tests; sweep 198.

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): harden #655 stance runner/renderer per dual-track review R1

- validate_stance_record now rechecks stance consent, binds
  stance_runtime to the consented stance_plan, and deep-replays every
  referenced evidence row against the exact ledger (claim/candidate/
  ledger bindings, coverage, sealed source hash, byte-span excerpt);
  cross-candidate reuse and orphan rows are rejected
- judge grammar is exactly four labeled lines in order; schema bounds
  (rationale/conditions/excerpt/raw-output ceilings) enforced at parse
  so a verbose or unspaced-script judge degrades one row, never the run;
  non-string returns are judge_error
- every transport call records a §6 transmission event returned beside
  the record; evidence sharing_scope never inherits persistence
  authority (fixes a schema failure under authorized-export plans)
- renderer: _inert flattens line breaks and escapes Markdown link/image
  syntax (a hostile title can no longer forge bounded sentences or emit
  a tracking beacon); per-source rows carry the §5.3 fields in the §5.4
  sentence form; consented stance provider and retention disclosure
  surfaced; ledger self-binding checked on the no-record path; view
  persistence gated on explicit_local_export with a hash-bound derived
  path and exclusive no-follow 0600 fsynced write
- 28 tests across runner/renderer; family sweep 205 green

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): #655 stance runner/renderer R2 — output replay, control chars, versions

- validate_stance_record replays every performed row from its retained
  raw output (a resealed stance/rationale/conditions the judge never
  produced fails) and replays spans for both positive excerpt states
- judge grammar rejects blank lines anywhere in the four-line output
- _inert strips C0/C1 and bidi control characters (terminal escapes and
  RLO tricks cannot reach stdout); per-source rows list the family's
  other versions with their original provider ranks; the view exporter
  fsyncs the parent directory
- 32 tests across runner/renderer; family sweep 209 green

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): #655 stance runner/renderer R3 — downgrade forgery, family binding, span bounds

- a parse_error row whose retained output parses cleanly is rejected as
  a downgrade forgery; stance rows bind to the ledger family's canonical
  hit and coverage; span replay enforces bounds and source byte length
- four-line grammar rejects edge blank lines; empty CONDITIONS degrades
  the row instead of aborting the run
- _inert strips every Cc/Cf character (tab flattened to space)

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): #655 stance validator R4 — complete the hash-replay surface

Ledger content hash replays from the inspected bytes; every retained
raw output replays its digest on every row state; performed rows replay
prompt_sha256 from the frozen template and assessment_input_sha256 from
its canonical projection. Every hash inside a stance record is now
recomputed, none trusted.

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): #655 stance validator R5 — close the downgrade family

Any not_checked row whose retained output parses cleanly is rejected
regardless of its declared failure state, and abstract_missing must
agree with the ledger family's content state.

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* fix(probe): #655 stance validator R6 — bidirectional failure/content binding

Content-unavailable ledger families accept only abstract_missing as
their row failure (they are never dispatched), completing the
equivalence both ways.

Refs #655

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

* chore: retrigger CI for updated PR body ([skip-closes-check] bypass)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018Z2pMsRNu386PDHeNvxR5r

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-14 18:09:28 +08:00

193 lines
6.7 KiB
Python

from __future__ import annotations
import copy
import pytest
from scripts import build_claim_standing_candidate_ledger as ledger
from scripts import claim_standing_stance_runner as runner
from scripts import render_claim_standing_view as view
from scripts.test_build_claim_standing_candidate_ledger import (
_rehash_input,
_retained,
)
from scripts.test_claim_standing_stance_runner import FakeTransport, _stance_setup
FORBIDDEN_PHRASES = (
"scientific consensus",
"field-level standing",
"verified true",
"verified false",
"credibility score",
"trust score",
"confidence score",
"complete search",
"none exist",
"the field agrees",
"the literature establishes",
)
EMPTY_CATEGORIES = (
"contradict",
"mixed",
"not_addressed",
"INSUFFICIENT_EVIDENCE",
"AMBIGUOUS",
"not_checked",
)
def _full_setup():
plan, ledger_value = _stance_setup()
record, evidence_rows, _ = runner.run_stance(
plan, ledger_value, transport=FakeTransport()
)
return plan, ledger_value, record, evidence_rows
def test_full_view_carries_banner_three_parts_and_fixed_wording():
plan, ledger_value, record, evidence_rows = _full_setup()
text = view.render_view(plan, ledger_value, record, evidence_rows)
lines = text.splitlines()
assert lines[0] == "STANCE CLASSIFICATION UNMEASURED"
for heading in (
"## Consent and recorded search",
"## Candidate ledger",
"## Claim standing (advisory)",
):
assert heading in text
assert (
"Within the recorded search of arxiv, crossref, index-a using q1, "
"1/1 selected candidates were classified support." in text
or "1/1 selected candidates were classified support." in text
)
for category in EMPTY_CATEGORIES:
assert (
f"No {category} sources were found among the selected candidates "
"within this recorded search." in text
)
assert "0/1 selected candidates were not checked or failed" in text
assert "ABSTRACT" in text
assert "duplicate_version" in text
assert "not_relevant" in text
assert "service_unavailable" in text
assert "secondary" in text.lower()
def test_view_never_uses_forbidden_vocabulary():
plan, ledger_value, record, evidence_rows = _full_setup()
text = view.render_view(plan, ledger_value, record, evidence_rows).lower()
for phrase in FORBIDDEN_PHRASES:
assert phrase not in text
def test_provider_text_renders_inert():
plan, ledger_value, record, evidence_rows = _full_setup()
tampered = copy.deepcopy(ledger_value)
for hit in tampered["raw_hits"]:
if hit["raw_hit_id"] == record["rows"][0]["canonical_raw_hit_id"]:
hit["title"] = 'Reminder <script>alert("x")</script> Effects'
tampered["candidate_ledger_sha256"] = ledger.bound_digest(
tampered, "candidate_ledger_sha256"
)
text = view.render_view(plan, tampered, None, [])
assert "<script>" not in text
assert "&lt;script&gt;" in text
def test_stale_stance_record_refuses_to_render():
plan, ledger_value, record, evidence_rows = _full_setup()
tampered = copy.deepcopy(ledger_value)
tampered["raw_hits"][0]["title"] = "Silently swapped title"
tampered["candidate_ledger_sha256"] = ledger.bound_digest(
tampered, "candidate_ledger_sha256"
)
with pytest.raises(runner.StanceError, match="stale|candidate_ledger"):
view.render_view(plan, tampered, record, evidence_rows)
def test_provider_title_cannot_forge_bounded_sentences_or_beacons():
plan, ledger_value, record, evidence_rows = _full_setup()
tampered = copy.deepcopy(ledger_value)
for hit in tampered["raw_hits"]:
if hit["raw_hit_id"] == record["rows"][0]["canonical_raw_hit_id"]:
hit["title"] = (
"Benign Title\n- The literature establishes the claim.\n"
"![x](https://tracker.example/pixel)"
)
tampered["candidate_ledger_sha256"] = ledger.bound_digest(
tampered, "candidate_ledger_sha256"
)
text = view.render_view(plan, tampered, None, [])
assert "\n- The literature establishes" not in text
assert "the literature establishes" not in text.splitlines()[0]
assert "![x](" not in text
for line in text.splitlines():
if "Benign Title" in line:
assert "tracker.example" in line # flattened into one inert line
def test_view_persistence_requires_export_consent_and_bound_path(tmp_path):
import json as _json
plan, ledger_value, _, _ = _full_setup()
plan_path = tmp_path / "plan.json"
plan_path.write_text(_json.dumps(plan), encoding="utf-8")
ledger_path = tmp_path / "ledger.json"
ledger_path.write_text(_json.dumps(ledger_value), encoding="utf-8")
wrong_output = tmp_path / "anywhere.md"
exit_code = view.main(
[
"--query-plan",
str(plan_path),
"--candidate-ledger",
str(ledger_path),
"--output",
str(wrong_output),
]
)
assert exit_code == 1
assert not wrong_output.exists()
def test_per_source_rows_carry_the_required_fields():
plan, ledger_value, record, evidence_rows = _full_setup()
text = view.render_view(plan, ledger_value, record, evidence_rows)
assert "provider rank 1" in text
assert "relevance relevant" in text
assert "found by" in text
assert (
"This candidate's inspected abstract was classified support "
"relative to claim" in text
)
assert "Stance provider (consented):" in text
assert "provider retention: unknown" in text
def test_terminal_control_characters_are_stripped():
plan, ledger_value, _, _ = _full_setup()
tampered = copy.deepcopy(ledger_value)
tampered["raw_hits"][0]["title"] = "Fine \x1b[2J\x07 Title \u202e sneaky"
tampered["candidate_ledger_sha256"] = ledger.bound_digest(
tampered, "candidate_ledger_sha256"
)
text = view.render_view(plan, tampered, None, [])
assert "\x1b" not in text
assert "\x07" not in text
assert "\u202e" not in text
def test_per_source_rows_list_other_versions_with_ranks():
plan, ledger_value, record, evidence_rows = _full_setup()
text = view.render_view(plan, ledger_value, record, evidence_rows)
assert "Other versions:" in text
assert "rank 2" in text
def test_retrieval_only_view_renders_without_a_stance_record():
plan, ledger_value, _, _ = _full_setup()
text = view.render_view(plan, ledger_value, None, [])
assert text.splitlines()[0] == "STANCE CLASSIFICATION UNMEASURED"
assert "## Consent and recorded search" in text
assert "## Candidate ledger" in text
assert "Stance classification was not run under this consent." in text