mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
cdb8485ec2
* feat(probe): pipeline wiring for #655 PR-C — trigger, consent surface, freshness, transmission ledger - claim_verification_protocol.md gains the Claim-Standing Probe Offer section: user-requested advisory view after the Phase E Claim Registry at Stage 2.5/4.5; HIGH-IMPACT-only trigger; ambiguity ineligible until researcher confirmation; never part of the integrity result (gate 12 boundary). - scripts/build_claim_standing_query_plan.py: §3.1 eligibility (gate 1) + §3.2 propose/bind consent surface (gate 2); emits schema-valid query-plan 1.0/1.1 or an explicit not_checked declination on cancel. - scripts/check_claim_standing_freshness.py: §7 probe-identity drift -> stale with closed reasons (gate 13); corrupt digests are errors. - shared/contracts/claim_standing/transmission_ledger.schema.json + scripts/check_claim_standing_transmissions.py: closed §6 per-event accounting; consented allowlist/roster fail-closed (gate 14). - 82 new tests across four files; CI manifest + integration-guard markers, protocol doc, contracts README, CHANGELOG aligned. Zero-spend: no live index, model, or network call anywhere; every probe surface still says STANCE CLASSIFICATION UNMEASURED. #655 stays open. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m * fix(probe): PR-C review round 2 — consent binds full projection, honest declinations, freshness coverage, transmission completeness Applies the converged three-track findings (codex 5 P1 + 2 P2, security 2 P2 + 6 P3, simplify/reuse/altitude): - Gate 1: Stage 2.5 HIGH-IMPACT tier alone is the registry witness; E1 now records the five-part high_impact_basis; basis provenance recorded on the consent surface, never in the registry. - Gate 2: consent surface embeds the complete consentable-plan projection (surface hash covers every receipt-bound field; bind asserts digest equality); absence/invalidation/cancel each produce an explicit local not_checked declination (consent_absent/consent_invalidated/ consent_cancelled); bind --output is consent-gated to the derived .query-plan.json path. - Gate 13: freshness consumes the runner's single expected_identity authority, requires the record's ledger, compares consented provider/model, and declares assessed/unassessed bindings. - Gate 14: schema + self-digest validation of both inputs, duplicate attempt and off-target index refusal, explicit stance-transmission requirement, stance-record cross-check, event-kind fixed field-by-field, conservative-accounting semantics documented; uniqueItems on events. - Gate 12: attribute-level containment of the discovery module, no direct write calls, CLI paths included in the no-file-creation run; doc pins consolidated into the #655 integration guard (incl. Phase E surface and a forbidden-stale-sentence check). - Substrate: validator memoization, shared export-consent helpers (trailing-separator refusal), exported trigger constants pinned to the plan schema; falsey decisions refused, deep copies via copy.deepcopy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m * fix(probe): PR-C review round 3 — intake invariants in gate 14, cancel-flow fidelity, complete artifact roster codex R2 (1 P1 + 4 P2 + 1 P3) and security R2 (6 P3) convergence: - Gate 14: adopt substrate.validate_input so a re-sealed retrieval input omitting a planned attempt is refused; stance-record cross-check now schema-validates + digest-replays the record and compares per-row prompt hash and result state (closed row->event mapping). - Gate 2: an explicit cancel records consent_cancelled even after a retrieval_plus_stance surface was proposed (checked before the surface hash comparison); falsey filter lists refused at proposal; bind CLI exits 3 on declinations. - Consent surface: complete consented-artifact roster from new substrate.ARTIFACT_SUFFIXES (test-pinned to every owning module's suffix constant); trailing-separator authorized paths refused at proposal, in validate_plan, and via the shared helper now used by discovery and the renderer too. - Freshness: fail-closed parity guard between the runner's expected_identity and the stale-reason map; eligibility verdict gains a dispatchable flag distinct from the tier witness. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m * fix(probe): PR-C review round 4 — failed stance calls bind their prompt hash codex R3's remaining P2: a judge_timeout / judge_error / oversized-output row left prompt_sha256 null, so the gate-14 cross-check could not hold a tampered failure event to the hash actually sent. The runner now binds prompt_sha256 + assessment_input_sha256 into the row BEFORE the transport call; validate_stance_record replays those hashes on transport-reaching failure rows (and requires them null on undispatched abstract_missing rows); the transmission cross-check requires a non-null, exactly-matching prompt hash for every transport-reaching row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m * test(probe): close the remaining capability-scan evasion forms Dotted from-imports of discovery symbols, discovery-alias rebinding, and getattr() dynamic lookup now fail the gate-12 wiring scan (security R2 P3-6 residue). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m * chore: ignore the maintainer-local deliverables/ directory Local working documents are not part of the public project; keep the directory permanently out of version control on every checkout. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KeJS63dMYTADV8FPR4Mx1m --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
291 lines
10 KiB
Python
291 lines
10 KiB
Python
"""#655 PR-C: pipeline-wiring noninterference tests (design §9 gate 12).
|
|
|
|
Running the wiring layer changes no Phase E verdict, gate, manuscript,
|
|
citation, or read-ledger byte — proven here as (a) a static capability scan
|
|
of the three wiring modules (import allowlist, no network / subprocess /
|
|
Phase E / read-ledger reach, attribute-level containment of the discovery
|
|
module to its pure roster constant, no direct file-writing calls), and (b)
|
|
an end-to-end run of both the library seams and the CLI entry points that
|
|
creates no files and mutates no inputs. The prose-surface markers for the
|
|
wiring live in `scripts/check_claim_standing_candidate_ledger_integration.py`
|
|
(single doc-pin owner), not here.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import ast
|
|
import copy
|
|
import json
|
|
from pathlib import Path
|
|
|
|
from scripts import build_claim_standing_candidate_ledger as substrate
|
|
from scripts import build_claim_standing_query_plan as plan_builder
|
|
from scripts import check_claim_standing_freshness as freshness
|
|
from scripts import check_claim_standing_transmissions as transmissions
|
|
from scripts import claim_standing_stance_runner as runner
|
|
from scripts.test_build_claim_standing_candidate_ledger import (
|
|
_rehash_input,
|
|
_rehash_plan,
|
|
_retained,
|
|
)
|
|
from scripts.test_build_claim_standing_query_plan import (
|
|
_bound_decisions,
|
|
_decisions,
|
|
_registry_row,
|
|
)
|
|
from scripts.test_claim_standing_stance_contracts import _plan_v1_1
|
|
from scripts.test_claim_standing_stance_runner import FakeTransport
|
|
|
|
ROOT = Path(__file__).resolve().parents[1]
|
|
WIRING_MODULES = (
|
|
ROOT / "scripts/build_claim_standing_query_plan.py",
|
|
ROOT / "scripts/check_claim_standing_freshness.py",
|
|
ROOT / "scripts/check_claim_standing_transmissions.py",
|
|
)
|
|
ALLOWED_IMPORTS = {
|
|
"__future__",
|
|
"argparse",
|
|
"copy",
|
|
"json",
|
|
"sys",
|
|
"pathlib",
|
|
"typing",
|
|
"scripts",
|
|
"build_claim_standing_candidate_ledger",
|
|
"claim_standing_discovery",
|
|
"claim_standing_stance_runner",
|
|
}
|
|
FORBIDDEN_CAPABILITY_MODULES = {
|
|
"urllib",
|
|
"http",
|
|
"socket",
|
|
"ssl",
|
|
"subprocess",
|
|
"ctypes",
|
|
"importlib",
|
|
"os",
|
|
}
|
|
PHASE_E_AND_READ_LEDGER_MODULES = {"evidence_rows", "retraction_status"}
|
|
# The discovery module holds live transports; the wiring layer may reach
|
|
# only its pure declared-roster constant through it.
|
|
ALLOWED_DISCOVERY_ATTRIBUTES = {"provider_roster_defaults"}
|
|
WRITE_CAPABLE_ATTRIBUTES = {
|
|
"write_text",
|
|
"write_bytes",
|
|
"unlink",
|
|
"rmdir",
|
|
"chmod",
|
|
"mkdir",
|
|
"rename",
|
|
"replace",
|
|
"symlink_to",
|
|
"touch",
|
|
}
|
|
|
|
|
|
def _module_ast(path: Path) -> ast.Module:
|
|
return ast.parse(path.read_text(encoding="utf-8"))
|
|
|
|
|
|
def _imported_names(tree: ast.Module) -> set[str]:
|
|
names: set[str] = set()
|
|
for node in ast.walk(tree):
|
|
if isinstance(node, ast.Import):
|
|
for alias in node.names:
|
|
names.update(alias.name.split("."))
|
|
elif isinstance(node, ast.ImportFrom):
|
|
if node.module is not None:
|
|
names.update(node.module.split("."))
|
|
names.update(
|
|
alias.name.split(".")[0]
|
|
for alias in node.names
|
|
if node.module is not None
|
|
and node.module.split(".")[0] == "scripts"
|
|
)
|
|
return names
|
|
|
|
|
|
def _discovery_aliases(tree: ast.Module) -> set[str]:
|
|
aliases: set[str] = set()
|
|
for node in ast.walk(tree):
|
|
if isinstance(node, ast.Import):
|
|
for alias in node.names:
|
|
if alias.name.split(".")[-1] == "claim_standing_discovery":
|
|
aliases.add(alias.asname or alias.name.split(".")[0])
|
|
elif isinstance(node, ast.ImportFrom):
|
|
for alias in node.names:
|
|
if alias.name == "claim_standing_discovery":
|
|
aliases.add(alias.asname or alias.name)
|
|
return aliases
|
|
|
|
|
|
# --- gate 12: static capability scan ----------------------------------------
|
|
|
|
|
|
def test_wiring_modules_import_only_the_declared_allowlist() -> None:
|
|
assert not ALLOWED_IMPORTS & (
|
|
FORBIDDEN_CAPABILITY_MODULES | PHASE_E_AND_READ_LEDGER_MODULES
|
|
)
|
|
for path in WIRING_MODULES:
|
|
names = _imported_names(_module_ast(path))
|
|
assert names <= ALLOWED_IMPORTS, (
|
|
f"{path.name} imports outside the wiring allowlist: "
|
|
f"{sorted(names - ALLOWED_IMPORTS)}"
|
|
)
|
|
|
|
|
|
def test_discovery_reach_is_limited_to_the_pure_roster_constant() -> None:
|
|
for path in WIRING_MODULES:
|
|
tree = _module_ast(path)
|
|
aliases = _discovery_aliases(tree)
|
|
for node in ast.walk(tree):
|
|
if (
|
|
isinstance(node, ast.Attribute)
|
|
and isinstance(node.value, ast.Name)
|
|
and node.value.id in aliases
|
|
):
|
|
assert node.attr in ALLOWED_DISCOVERY_ATTRIBUTES, (
|
|
f"{path.name} reaches discovery.{node.attr}; only "
|
|
f"{sorted(ALLOWED_DISCOVERY_ATTRIBUTES)} is allowed"
|
|
)
|
|
# An alias must not escape the attribute scan by rebinding or
|
|
# dynamic lookup.
|
|
if (
|
|
isinstance(node, ast.Assign)
|
|
and isinstance(node.value, ast.Name)
|
|
and node.value.id in aliases
|
|
):
|
|
raise AssertionError(
|
|
f"{path.name} rebinds a discovery alias"
|
|
)
|
|
if (
|
|
isinstance(node, ast.Call)
|
|
and isinstance(node.func, ast.Name)
|
|
and node.func.id == "getattr"
|
|
):
|
|
raise AssertionError(
|
|
f"{path.name} uses getattr(); the capability scan "
|
|
"requires static attribute access"
|
|
)
|
|
|
|
|
|
def test_wiring_modules_perform_no_direct_writes_or_read_ledger_access() -> None:
|
|
for path in WIRING_MODULES:
|
|
tree = _module_ast(path)
|
|
for node in ast.walk(tree):
|
|
if (
|
|
isinstance(node, ast.Call)
|
|
and isinstance(node.func, ast.Name)
|
|
and node.func.id == "open"
|
|
):
|
|
raise AssertionError(f"{path.name} calls open() directly")
|
|
if (
|
|
isinstance(node, ast.Attribute)
|
|
and node.attr in WRITE_CAPABLE_ATTRIBUTES
|
|
):
|
|
raise AssertionError(
|
|
f"{path.name} uses write-capable attribute {node.attr!r}; "
|
|
"all persistence must flow through the substrate writer"
|
|
)
|
|
source = path.read_text(encoding="utf-8")
|
|
for marker in ("human_read_log", "human_read_source", "read_scope"):
|
|
assert marker not in source, f"{path.name} touches {marker}"
|
|
|
|
|
|
# --- gate 12: end-to-end run leaves everything untouched --------------------
|
|
|
|
|
|
def _write_json(path: Path, value: object) -> Path:
|
|
path.write_text(json.dumps(value, ensure_ascii=False), encoding="utf-8")
|
|
return path
|
|
|
|
|
|
def test_end_to_end_wiring_creates_no_files_and_mutates_no_inputs(
|
|
tmp_path, monkeypatch, capsys
|
|
) -> None:
|
|
inputs = tmp_path / "inputs"
|
|
inputs.mkdir()
|
|
workdir = tmp_path / "work"
|
|
workdir.mkdir()
|
|
monkeypatch.chdir(workdir)
|
|
|
|
row = _registry_row()
|
|
decisions = _bound_decisions(row, _decisions())
|
|
row_before = copy.deepcopy(row)
|
|
decisions_before = copy.deepcopy(decisions)
|
|
built_plan = plan_builder.bind_plan(row, decisions)
|
|
substrate.validate_plan(built_plan)
|
|
|
|
stance_plan = _plan_v1_1(stance=True)
|
|
stance_plan["stance_plan"]["prompt_contract_version"] = (
|
|
runner.PROMPT_CONTRACT_VERSION
|
|
)
|
|
_rehash_plan(stance_plan)
|
|
retained = _retained()
|
|
_rehash_input(retained, stance_plan)
|
|
ledger_value = substrate.build_ledger(stance_plan, retained)
|
|
record, _, events = runner.run_stance(
|
|
stance_plan, ledger_value, transport=FakeTransport()
|
|
)
|
|
inputs_before = copy.deepcopy((stance_plan, retained, ledger_value, record))
|
|
|
|
transmissions.build_transmission_ledger(
|
|
stance_plan, retained, stance_transmissions=events, stance_record=record
|
|
)
|
|
freshness.assess_freshness(
|
|
current_claim_text=stance_plan["claim"]["claim_text"],
|
|
plan=stance_plan,
|
|
candidate_ledger=ledger_value,
|
|
stance_record=record,
|
|
)
|
|
|
|
row_file = _write_json(inputs / "row.json", row)
|
|
decisions_file = _write_json(inputs / "decisions.json", decisions)
|
|
plan_file = _write_json(inputs / "plan.json", stance_plan)
|
|
input_file = _write_json(inputs / "input.json", retained)
|
|
events_file = _write_json(inputs / "events.json", events)
|
|
record_file = _write_json(inputs / "record.json", record)
|
|
claim_file = inputs / "claim.txt"
|
|
claim_file.write_text(
|
|
stance_plan["claim"]["claim_text"], encoding="utf-8"
|
|
)
|
|
assert plan_builder.main(
|
|
["propose", "--registry-claim", str(row_file), "--decisions", str(decisions_file)]
|
|
) == 0
|
|
assert plan_builder.main(
|
|
["bind", "--registry-claim", str(row_file), "--decisions", str(decisions_file)]
|
|
) == 0
|
|
assert transmissions.main(
|
|
[
|
|
"build",
|
|
"--query-plan", str(plan_file),
|
|
"--retrieval-input", str(input_file),
|
|
"--stance-transmissions", str(events_file),
|
|
"--stance-record", str(record_file),
|
|
]
|
|
) == 0
|
|
assert freshness.main(
|
|
[
|
|
"--current-claim-file", str(claim_file),
|
|
"--query-plan", str(plan_file),
|
|
"--candidate-ledger", str(_write_json(inputs / "ledger.json", ledger_value)),
|
|
"--stance-record", str(record_file),
|
|
]
|
|
) == 0
|
|
capsys.readouterr()
|
|
|
|
assert row == row_before
|
|
assert decisions == decisions_before
|
|
assert (stance_plan, retained, ledger_value, record) == inputs_before
|
|
assert list(workdir.iterdir()) == []
|
|
|
|
|
|
def test_declination_record_carries_no_verdict_or_gate_vocabulary() -> None:
|
|
row = _registry_row()
|
|
decisions = _bound_decisions(
|
|
row, _decisions(decision="cancel", recorded_at="2026-08-14T01:05:00Z")
|
|
)
|
|
record = plan_builder.bind_plan(row, decisions)
|
|
rendered = json.dumps(record).lower()
|
|
for forbidden in ("verdict", "severity", "pass", "fail", "score"):
|
|
assert forbidden not in rendered
|