Files
imbad0202__academic-researc…/scripts/test_claim_standing_pipeline_wiring.py
Edward Cheng-I Wu cdb8485ec2 feat(probe): pipeline wiring for #655 PR-C — trigger, consent surface, freshness, transmission ledger (#733)
* feat(probe): pipeline wiring for #655 PR-C — trigger, consent surface, freshness, transmission ledger

- claim_verification_protocol.md gains the Claim-Standing Probe Offer section:
  user-requested advisory view after the Phase E Claim Registry at Stage
  2.5/4.5; HIGH-IMPACT-only trigger; ambiguity ineligible until researcher
  confirmation; never part of the integrity result (gate 12 boundary).
- scripts/build_claim_standing_query_plan.py: §3.1 eligibility (gate 1) +
  §3.2 propose/bind consent surface (gate 2); emits schema-valid query-plan
  1.0/1.1 or an explicit not_checked declination on cancel.
- scripts/check_claim_standing_freshness.py: §7 probe-identity drift ->
  stale with closed reasons (gate 13); corrupt digests are errors.
- shared/contracts/claim_standing/transmission_ledger.schema.json +
  scripts/check_claim_standing_transmissions.py: closed §6 per-event
  accounting; consented allowlist/roster fail-closed (gate 14).
- 82 new tests across four files; CI manifest + integration-guard markers,
  protocol doc, contracts README, CHANGELOG aligned.

Zero-spend: no live index, model, or network call anywhere; every probe
surface still says STANCE CLASSIFICATION UNMEASURED. #655 stays open.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

* fix(probe): PR-C review round 2 — consent binds full projection, honest declinations, freshness coverage, transmission completeness

Applies the converged three-track findings (codex 5 P1 + 2 P2, security
2 P2 + 6 P3, simplify/reuse/altitude):

- Gate 1: Stage 2.5 HIGH-IMPACT tier alone is the registry witness; E1 now
  records the five-part high_impact_basis; basis provenance recorded on the
  consent surface, never in the registry.
- Gate 2: consent surface embeds the complete consentable-plan projection
  (surface hash covers every receipt-bound field; bind asserts digest
  equality); absence/invalidation/cancel each produce an explicit local
  not_checked declination (consent_absent/consent_invalidated/
  consent_cancelled); bind --output is consent-gated to the derived
  .query-plan.json path.
- Gate 13: freshness consumes the runner's single expected_identity
  authority, requires the record's ledger, compares consented
  provider/model, and declares assessed/unassessed bindings.
- Gate 14: schema + self-digest validation of both inputs, duplicate
  attempt and off-target index refusal, explicit stance-transmission
  requirement, stance-record cross-check, event-kind fixed field-by-field,
  conservative-accounting semantics documented; uniqueItems on events.
- Gate 12: attribute-level containment of the discovery module, no direct
  write calls, CLI paths included in the no-file-creation run; doc pins
  consolidated into the #655 integration guard (incl. Phase E surface and
  a forbidden-stale-sentence check).
- Substrate: validator memoization, shared export-consent helpers
  (trailing-separator refusal), exported trigger constants pinned to the
  plan schema; falsey decisions refused, deep copies via copy.deepcopy.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

* fix(probe): PR-C review round 3 — intake invariants in gate 14, cancel-flow fidelity, complete artifact roster

codex R2 (1 P1 + 4 P2 + 1 P3) and security R2 (6 P3) convergence:

- Gate 14: adopt substrate.validate_input so a re-sealed retrieval input
  omitting a planned attempt is refused; stance-record cross-check now
  schema-validates + digest-replays the record and compares per-row prompt
  hash and result state (closed row->event mapping).
- Gate 2: an explicit cancel records consent_cancelled even after a
  retrieval_plus_stance surface was proposed (checked before the surface
  hash comparison); falsey filter lists refused at proposal; bind CLI
  exits 3 on declinations.
- Consent surface: complete consented-artifact roster from new
  substrate.ARTIFACT_SUFFIXES (test-pinned to every owning module's suffix
  constant); trailing-separator authorized paths refused at proposal, in
  validate_plan, and via the shared helper now used by discovery and the
  renderer too.
- Freshness: fail-closed parity guard between the runner's
  expected_identity and the stale-reason map; eligibility verdict gains a
  dispatchable flag distinct from the tier witness.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

* fix(probe): PR-C review round 4 — failed stance calls bind their prompt hash

codex R3's remaining P2: a judge_timeout / judge_error / oversized-output
row left prompt_sha256 null, so the gate-14 cross-check could not hold a
tampered failure event to the hash actually sent. The runner now binds
prompt_sha256 + assessment_input_sha256 into the row BEFORE the transport
call; validate_stance_record replays those hashes on transport-reaching
failure rows (and requires them null on undispatched abstract_missing
rows); the transmission cross-check requires a non-null, exactly-matching
prompt hash for every transport-reaching row.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

* test(probe): close the remaining capability-scan evasion forms

Dotted from-imports of discovery symbols, discovery-alias rebinding, and
getattr() dynamic lookup now fail the gate-12 wiring scan (security R2
P3-6 residue).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

* chore: ignore the maintainer-local deliverables/ directory

Local working documents are not part of the public project; keep the
directory permanently out of version control on every checkout.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-08-17 11:02:59 +08:00

291 lines
10 KiB
Python

"""#655 PR-C: pipeline-wiring noninterference tests (design §9 gate 12).
Running the wiring layer changes no Phase E verdict, gate, manuscript,
citation, or read-ledger byte — proven here as (a) a static capability scan
of the three wiring modules (import allowlist, no network / subprocess /
Phase E / read-ledger reach, attribute-level containment of the discovery
module to its pure roster constant, no direct file-writing calls), and (b)
an end-to-end run of both the library seams and the CLI entry points that
creates no files and mutates no inputs. The prose-surface markers for the
wiring live in `scripts/check_claim_standing_candidate_ledger_integration.py`
(single doc-pin owner), not here.
"""
from __future__ import annotations
import ast
import copy
import json
from pathlib import Path
from scripts import build_claim_standing_candidate_ledger as substrate
from scripts import build_claim_standing_query_plan as plan_builder
from scripts import check_claim_standing_freshness as freshness
from scripts import check_claim_standing_transmissions as transmissions
from scripts import claim_standing_stance_runner as runner
from scripts.test_build_claim_standing_candidate_ledger import (
_rehash_input,
_rehash_plan,
_retained,
)
from scripts.test_build_claim_standing_query_plan import (
_bound_decisions,
_decisions,
_registry_row,
)
from scripts.test_claim_standing_stance_contracts import _plan_v1_1
from scripts.test_claim_standing_stance_runner import FakeTransport
ROOT = Path(__file__).resolve().parents[1]
WIRING_MODULES = (
ROOT / "scripts/build_claim_standing_query_plan.py",
ROOT / "scripts/check_claim_standing_freshness.py",
ROOT / "scripts/check_claim_standing_transmissions.py",
)
ALLOWED_IMPORTS = {
"__future__",
"argparse",
"copy",
"json",
"sys",
"pathlib",
"typing",
"scripts",
"build_claim_standing_candidate_ledger",
"claim_standing_discovery",
"claim_standing_stance_runner",
}
FORBIDDEN_CAPABILITY_MODULES = {
"urllib",
"http",
"socket",
"ssl",
"subprocess",
"ctypes",
"importlib",
"os",
}
PHASE_E_AND_READ_LEDGER_MODULES = {"evidence_rows", "retraction_status"}
# The discovery module holds live transports; the wiring layer may reach
# only its pure declared-roster constant through it.
ALLOWED_DISCOVERY_ATTRIBUTES = {"provider_roster_defaults"}
WRITE_CAPABLE_ATTRIBUTES = {
"write_text",
"write_bytes",
"unlink",
"rmdir",
"chmod",
"mkdir",
"rename",
"replace",
"symlink_to",
"touch",
}
def _module_ast(path: Path) -> ast.Module:
return ast.parse(path.read_text(encoding="utf-8"))
def _imported_names(tree: ast.Module) -> set[str]:
names: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
names.update(alias.name.split("."))
elif isinstance(node, ast.ImportFrom):
if node.module is not None:
names.update(node.module.split("."))
names.update(
alias.name.split(".")[0]
for alias in node.names
if node.module is not None
and node.module.split(".")[0] == "scripts"
)
return names
def _discovery_aliases(tree: ast.Module) -> set[str]:
aliases: set[str] = set()
for node in ast.walk(tree):
if isinstance(node, ast.Import):
for alias in node.names:
if alias.name.split(".")[-1] == "claim_standing_discovery":
aliases.add(alias.asname or alias.name.split(".")[0])
elif isinstance(node, ast.ImportFrom):
for alias in node.names:
if alias.name == "claim_standing_discovery":
aliases.add(alias.asname or alias.name)
return aliases
# --- gate 12: static capability scan ----------------------------------------
def test_wiring_modules_import_only_the_declared_allowlist() -> None:
assert not ALLOWED_IMPORTS & (
FORBIDDEN_CAPABILITY_MODULES | PHASE_E_AND_READ_LEDGER_MODULES
)
for path in WIRING_MODULES:
names = _imported_names(_module_ast(path))
assert names <= ALLOWED_IMPORTS, (
f"{path.name} imports outside the wiring allowlist: "
f"{sorted(names - ALLOWED_IMPORTS)}"
)
def test_discovery_reach_is_limited_to_the_pure_roster_constant() -> None:
for path in WIRING_MODULES:
tree = _module_ast(path)
aliases = _discovery_aliases(tree)
for node in ast.walk(tree):
if (
isinstance(node, ast.Attribute)
and isinstance(node.value, ast.Name)
and node.value.id in aliases
):
assert node.attr in ALLOWED_DISCOVERY_ATTRIBUTES, (
f"{path.name} reaches discovery.{node.attr}; only "
f"{sorted(ALLOWED_DISCOVERY_ATTRIBUTES)} is allowed"
)
# An alias must not escape the attribute scan by rebinding or
# dynamic lookup.
if (
isinstance(node, ast.Assign)
and isinstance(node.value, ast.Name)
and node.value.id in aliases
):
raise AssertionError(
f"{path.name} rebinds a discovery alias"
)
if (
isinstance(node, ast.Call)
and isinstance(node.func, ast.Name)
and node.func.id == "getattr"
):
raise AssertionError(
f"{path.name} uses getattr(); the capability scan "
"requires static attribute access"
)
def test_wiring_modules_perform_no_direct_writes_or_read_ledger_access() -> None:
for path in WIRING_MODULES:
tree = _module_ast(path)
for node in ast.walk(tree):
if (
isinstance(node, ast.Call)
and isinstance(node.func, ast.Name)
and node.func.id == "open"
):
raise AssertionError(f"{path.name} calls open() directly")
if (
isinstance(node, ast.Attribute)
and node.attr in WRITE_CAPABLE_ATTRIBUTES
):
raise AssertionError(
f"{path.name} uses write-capable attribute {node.attr!r}; "
"all persistence must flow through the substrate writer"
)
source = path.read_text(encoding="utf-8")
for marker in ("human_read_log", "human_read_source", "read_scope"):
assert marker not in source, f"{path.name} touches {marker}"
# --- gate 12: end-to-end run leaves everything untouched --------------------
def _write_json(path: Path, value: object) -> Path:
path.write_text(json.dumps(value, ensure_ascii=False), encoding="utf-8")
return path
def test_end_to_end_wiring_creates_no_files_and_mutates_no_inputs(
tmp_path, monkeypatch, capsys
) -> None:
inputs = tmp_path / "inputs"
inputs.mkdir()
workdir = tmp_path / "work"
workdir.mkdir()
monkeypatch.chdir(workdir)
row = _registry_row()
decisions = _bound_decisions(row, _decisions())
row_before = copy.deepcopy(row)
decisions_before = copy.deepcopy(decisions)
built_plan = plan_builder.bind_plan(row, decisions)
substrate.validate_plan(built_plan)
stance_plan = _plan_v1_1(stance=True)
stance_plan["stance_plan"]["prompt_contract_version"] = (
runner.PROMPT_CONTRACT_VERSION
)
_rehash_plan(stance_plan)
retained = _retained()
_rehash_input(retained, stance_plan)
ledger_value = substrate.build_ledger(stance_plan, retained)
record, _, events = runner.run_stance(
stance_plan, ledger_value, transport=FakeTransport()
)
inputs_before = copy.deepcopy((stance_plan, retained, ledger_value, record))
transmissions.build_transmission_ledger(
stance_plan, retained, stance_transmissions=events, stance_record=record
)
freshness.assess_freshness(
current_claim_text=stance_plan["claim"]["claim_text"],
plan=stance_plan,
candidate_ledger=ledger_value,
stance_record=record,
)
row_file = _write_json(inputs / "row.json", row)
decisions_file = _write_json(inputs / "decisions.json", decisions)
plan_file = _write_json(inputs / "plan.json", stance_plan)
input_file = _write_json(inputs / "input.json", retained)
events_file = _write_json(inputs / "events.json", events)
record_file = _write_json(inputs / "record.json", record)
claim_file = inputs / "claim.txt"
claim_file.write_text(
stance_plan["claim"]["claim_text"], encoding="utf-8"
)
assert plan_builder.main(
["propose", "--registry-claim", str(row_file), "--decisions", str(decisions_file)]
) == 0
assert plan_builder.main(
["bind", "--registry-claim", str(row_file), "--decisions", str(decisions_file)]
) == 0
assert transmissions.main(
[
"build",
"--query-plan", str(plan_file),
"--retrieval-input", str(input_file),
"--stance-transmissions", str(events_file),
"--stance-record", str(record_file),
]
) == 0
assert freshness.main(
[
"--current-claim-file", str(claim_file),
"--query-plan", str(plan_file),
"--candidate-ledger", str(_write_json(inputs / "ledger.json", ledger_value)),
"--stance-record", str(record_file),
]
) == 0
capsys.readouterr()
assert row == row_before
assert decisions == decisions_before
assert (stance_plan, retained, ledger_value, record) == inputs_before
assert list(workdir.iterdir()) == []
def test_declination_record_carries_no_verdict_or_gate_vocabulary() -> None:
row = _registry_row()
decisions = _bound_decisions(
row, _decisions(decision="cancel", recorded_at="2026-08-14T01:05:00Z")
)
record = plan_builder.bind_plan(row, decisions)
rendered = json.dumps(record).lower()
for forbidden in ("verdict", "severity", "pass", "fail", "score"):
assert forbidden not in rendered