Files
imbad0202__academic-researc…/scripts/inquiry_branch_ledger.py
T
Edward Cheng-I Wu f1a57bbcab fix: shared file-lock helper with msvcrt backend for the remaining fcntl sites (#845) (#847)
* fix: shared file-lock helper with msvcrt backend for the six fcntl sites (#845)

scripts/file_lock.py owns the backend choice (fcntl.flock on POSIX,
msvcrt.locking on byte 0 on Windows) and routes adjudication_activity,
inquiry_branch_ledger, review_criteria_binding, and ars_mark_read through
acquire()/release(). POSIX lock sequences are unchanged. Per-site Windows
decisions: adjudication reads degrade to exclusive with a 5 s bounded wait;
the review-criteria manifest lock is capped at 30 s on Windows only; the
inquiry ledger alpha keeps refusing non-POSIX hosts. Two finally blocks that
released an unacquired lock now release only what they acquired. SETUP docs
state the best-effort Windows posture; no Windows CI job is added.

Refs #845, #843, #844.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

* fix(file_lock): interrupted attempts honour the deadline; pin adjudication wait policy (#845)

Cross-model review round 1 (gpt-6-astra, xhigh): a persistent
InterruptedError could retry past the bound; the Windows-shape test did
not exercise adjudication's reader-waits / writer-does-not-wait policy;
the adjudication contention message now names LockTimeout instead of
BlockingIOError, recorded in the CHANGELOG rather than masked.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

* refactor(file_lock): held() context manager, single BACKEND source, one fake msvcrt (#845)

/simplify pass (four cleanup reviewers): the release-only-if-acquired
invariant moves into file_lock.held() and review_criteria_binding /
inquiry_branch_ledger use it; runtime branches key off BACKEND and
SHARED_LOCKS_SUPPORTED is dropped; EINTR joins the retryable errno set and
the unreachable EDEADLK entry goes; backend calls are deduplicated; all four
consumers try the sibling import first so one module instance is shared;
the Windows fake lives once in tests/fake_msvcrt.py; test scaffolding is
folded into a lock_pair fixture and a parametrized wait test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

* fix(file_lock): keep lock acquisition and the guarded body in separate try blocks (#845)

Cross-model review round 3 (gpt-6-astra, xhigh): wrapping the body in the
same handler that translates LockTimeout meant a contended inner lock inside
the body was reported as the outer manifest/passport lock failing. Both
consumers now acquire in their own try block and release only after a
successful acquire; held() is dropped from the helper. The subprocess test
pins that a LockTimeout raised inside the binding body surfaces as itself.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

* test(file_lock): let the body LockTimeout leave _locked() so the attribution check bites (#845)

Cross-model review round 4: the inner LockTimeout was caught inside the
binding body, so the erroneous outer translation would still have passed.
Verified by mutation: restoring the outer translation fails this test.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

* ci(673): whitelist scripts/test_file_lock.py as a non-consumer importer of the activity runtime (#845)

The shared file-lock test imports adjudication_activity in a subprocess to
exercise its lock backend under a fake msvcrt; it never reads or writes an
activity store. The exact-owner whitelist is the lint's route for that.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0131cZMWBPPeEFiqgEPFZ3X2

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-11 14:47:26 +08:00

2542 lines
100 KiB
Python

#!/usr/bin/env python3
"""Validate, replay, and durably publish inquiry-branch-ledger/1.0 (#743).
The ledger is an opt-in, event-sourced user-project artifact. This module is
deliberately manuscript-blind and model-free: callers supply exact profile
documents, timestamps, event payloads, workspace paths, and project identity.
It never infers a research family, branch, condition, or author decision.
Runtime interpretations needed to make the frozen design executable:
* profile bindings are resolved only through an explicit catalog of fully
validated research-workflow-profile/1.0 documents. A digest is not a
branch budget, and the current shipped fallback is never substituted for an
unresolved historical binding;
* a non-null parent_id must already exist when a branch is introduced;
* downstream_refs are identifier lists, so duplicates are rejected. This
makes "one stale event per identifier" and merge/supersession order unique;
* a newly emitted reopen-condition signal must name a condition still present
on a non-terminal branch. Removed condition ids remain retired and can be
resolved in historical events, but cannot motivate a new event;
* a branch_reopened event is followed immediately, in current downstream-ref
order, by exactly one system artifact_marked_stale event per identifier.
A missing, delayed, reordered, or orphan system event fails replay;
* repeated supersession resolutions for distinct stale causes of the same
artifact must name the same replacement. Replacing an identifier with
itself is not a replacement;
* persisted publication requires at least two introduced branches, matching
the frozen rule that the ledger/pointer do not materialize on the simple
zero-or-one-branch path. In-memory replay remains available before that;
* ledger_path is an explicitly workspace-relative POSIX path and the ledger
must be beside the passport. No path is resolved relative to the process
working directory;
* two stable filesystem paths cannot be renamed atomically together. A
shared passport-domain sidecar lock, full-byte compare-and-swap checks, and
a durable recovery journal make the two-file update atomic for cooperating
ARS readers. Without a valid recoverable journal, an unreadable/mismatched
pointer is surfaced as
LEDGER-BINDING-BROKEN and is never silently continued.
The hash chain is tamper-evident relative to the separately trusted passport
pointer. It is not authentication: a party able to rewrite both artifacts
can recompute both hashes, just as the design's within-session author labels
do not authenticate a human identity.
"""
from __future__ import annotations
import argparse
import copy
import hashlib
import io
import json
import math
import os
import re
import stat
import sys
import unicodedata
import uuid
from contextlib import contextmanager
from datetime import datetime
from pathlib import Path, PurePosixPath
from typing import Any, Callable, Iterable, Iterator, Mapping, NoReturn, Sequence
try: # Dual-path import: script invocation vs package import under pytest.
import file_lock
from research_workflow_profile import (
JCS_SAFE_INTEGER_MAX,
ContractError as ProfileContractError,
canonical_bytes as _profile_canonical_bytes,
load_profile,
profile_binding,
validate_profile,
)
except ImportError: # pragma: no cover - package-import path
from scripts import file_lock # type: ignore[no-redef]
from scripts.research_workflow_profile import (
JCS_SAFE_INTEGER_MAX,
ContractError as ProfileContractError,
canonical_bytes as _profile_canonical_bytes,
load_profile,
profile_binding,
validate_profile,
)
LEDGER_SCHEMA_VERSION = "inquiry-branch-ledger/1.0"
POINTER_LEDGER_VERSION = LEDGER_SCHEMA_VERSION
TRANSACTION_SCHEMA_VERSION = "inquiry-ledger-transaction/1.0"
ZERO_SHA256 = "0" * 64
ENV_FLAG = "ARS_INQUIRY_LEDGER"
EVENT_KINDS = (
"branch_created",
"facet_surfaced",
"branch_adopted",
"branch_annotated",
"branch_parked",
"branch_rejected",
"branch_reopened",
"branch_merged",
"branch_archived",
"reopen_condition_signal",
"profile_rebound",
"artifact_marked_stale",
"artifact_reconfirmed",
"artifact_superseded",
)
AI_EVENT_KINDS = frozenset({"facet_surfaced", "reopen_condition_signal"})
SYSTEM_EVENT_KINDS = frozenset({"artifact_marked_stale"})
ARTIFACT_EVENT_KINDS = frozenset(
{"artifact_marked_stale", "artifact_reconfirmed", "artifact_superseded"}
)
NULL_BRANCH_EVENT_KINDS = ARTIFACT_EVENT_KINDS | {"profile_rebound"}
LIVE_STATUSES = frozenset({"active", "reopened"})
TERMINAL_STATUSES = frozenset({"merged", "archived"})
BRANCH_STATUSES = (
"active",
"parked",
"rejected",
"reopened",
"merged",
"archived",
)
AUTHOR_OWNED_PROVENANCE = frozenset({"author_originated", "author_adopted"})
AUTHOR_REOPENABLE_STATUSES = frozenset({"parked", "rejected"})
PROVENANCE_VALUES = (
"author_originated",
"ai_surfaced_facet",
"author_adopted",
)
ANNOTATION_FIELDS = (
"assumptions",
"evidence_sought",
"reopen_conditions",
"downstream_refs",
)
SUMMARY_MOMENTS = (
"design_freeze",
"stage_2_5",
"stage_4_5",
"reopen_condition_signal",
)
_SUMMARY_TEXT_LIMIT = 160
_MARKDOWN_META = frozenset("\\`*_[]<>|")
_LEDGER_FIELDS = {
"schema_version",
"project_ref",
"initial_profile_binding",
"events",
}
_EVENT_FIELDS = {
"event_id",
"recorded_at",
"actor",
"kind",
"branch_id",
"payload",
"prev_event_sha256",
}
_EVENT_INPUT_FIELDS = {
"recorded_at",
"actor",
"kind",
"branch_id",
"payload",
}
_BINDING_FIELDS = {"profile_id", "profile_version", "content_sha256"}
_POINTER_FIELDS = {"ledger_path", "ledger_version", "content_sha256"}
_SLUG_RE = re.compile(r"^[a-z0-9][a-z0-9_-]*$")
_SEMVER_RE = re.compile(
r"^(0|[1-9][0-9]*)\."
r"(0|[1-9][0-9]*)\."
r"(0|[1-9][0-9]*)"
r"(?:-((?:0|[1-9][0-9]*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*)"
r"(?:\.(?:0|[1-9][0-9]*|[0-9A-Za-z-]*[A-Za-z-][0-9A-Za-z-]*))*))?"
r"(?:\+([0-9A-Za-z-]+(?:\.[0-9A-Za-z-]+)*))?$"
)
_SHA256_RE = re.compile(r"^[0-9a-f]{64}$")
_RFC3339_RE = re.compile(
r"^[0-9]{4}-[0-9]{2}-[0-9]{2}[Tt]"
r"(?:[01][0-9]|2[0-3]):[0-5][0-9]:[0-5][0-9]"
r"(?:\.[0-9]+)?(?:[Zz]|[+-](?:[01][0-9]|2[0-3]):[0-5][0-9])$"
)
_FORBIDDEN_PATH_CHARS_RE = re.compile(
"[\\x00-\\x1f\\x7f-\\x9f"
"\u00ad\u0600-\u0605\u061c\u06dd\u070f\u0890-\u0891\u08e2"
"\u180e\u200b-\u200f\u202a-\u202e\u2060-\u2064\u2066-\u206f"
"\ufeff\ufff9-\ufffb]"
)
class ContractError(ValueError):
"""Raised when ledger input or replay violates the frozen contract."""
class LedgerBindingError(ContractError):
"""Raised for the design's visible LEDGER-BINDING-BROKEN load state."""
def __init__(self, message: str):
super().__init__(f"LEDGER-BINDING-BROKEN: {message}")
class TransactionRecoveryError(ContractError):
"""Raised when a durable journal exists but cannot be safely recovered."""
def __init__(self, message: str):
super().__init__(f"LEDGER-TRANSACTION-RECOVERY-REQUIRED: {message}")
def _fail(path: str, message: str) -> NoReturn:
raise ContractError(f"{path}: {message}")
def canonical_bytes(value: Any) -> bytes:
"""Restricted JCS bytes shared byte-for-byte with the #742 runtime."""
try:
return _profile_canonical_bytes(value)
except ProfileContractError as exc:
raise ContractError(str(exc)) from exc
def _reject_constant(value: str) -> NoReturn:
raise ContractError(f"non-finite JSON number is not allowed: {value}")
def _reject_duplicate_pairs(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
result: dict[str, Any] = {}
for key, value in pairs:
if key in result:
raise ContractError(f"duplicate JSON key is not allowed: {key}")
result[key] = value
return result
def _parse_json_bytes(raw: bytes, *, source: str) -> dict[str, Any]:
try:
text = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise ContractError(f"JSON input is not UTF-8: {source}") from exc
try:
value = json.loads(
text,
object_pairs_hook=_reject_duplicate_pairs,
parse_constant=_reject_constant,
)
except (json.JSONDecodeError, ContractError) as exc:
raise ContractError(f"cannot parse JSON input {source}: {exc}") from exc
if not isinstance(value, dict):
raise ContractError(f"top-level JSON value must be an object: {source}")
return value
def load_ledger(path: Path | str, *, require_canonical: bool = True) -> dict[str, Any]:
"""Load a ledger with duplicate-key rejection and exact canonical storage."""
ledger_path = Path(path)
try:
raw = ledger_path.read_bytes()
except OSError as exc:
raise ContractError(f"cannot read ledger {ledger_path}: {exc}") from exc
value = _parse_json_bytes(raw, source=str(ledger_path))
if require_canonical and raw != canonical_bytes(value):
raise ContractError(
f"ledger file must be stored as exact JSON Canonical Form bytes: {ledger_path}"
)
return value
def _object(
value: Any,
path: str,
required: set[str],
optional: set[str] | None = None,
) -> Mapping[str, Any]:
if not isinstance(value, Mapping):
_fail(path, "must be an object")
optional = optional or set()
keys = set(value)
missing = required - keys
extra = keys - required - optional
if missing:
_fail(path, f"missing field(s): {', '.join(sorted(missing))}")
if extra:
_fail(path, f"undeclared field(s): {', '.join(sorted(extra))}")
return value
def _text(value: Any, path: str) -> str:
if not isinstance(value, str) or not value.strip():
_fail(path, "must be a non-empty string")
return value
def _slug(value: Any, path: str) -> str:
text = _text(value, path)
if _SLUG_RE.fullmatch(text) is None:
_fail(path, "must be a lowercase slug using letters, digits, '_' or '-'")
return text
def _semver(value: Any, path: str) -> str:
text = _text(value, path)
if _SEMVER_RE.fullmatch(text) is None:
_fail(path, "must be a SemVer 2.0.0 version")
return text
def _sha256(value: Any, path: str) -> str:
text = _text(value, path)
if _SHA256_RE.fullmatch(text) is None:
_fail(path, "must be a lowercase 64-hex SHA-256 digest")
return text
def _positive_int(value: Any, path: str) -> int:
if (
isinstance(value, bool)
or not isinstance(value, int)
or value < 1
or value > JCS_SAFE_INTEGER_MAX
):
_fail(path, f"must be an integer from 1 through {JCS_SAFE_INTEGER_MAX}")
return value
def _nonnegative_int(value: Any, path: str) -> int:
if (
isinstance(value, bool)
or not isinstance(value, int)
or value < 0
or value > JCS_SAFE_INTEGER_MAX
):
_fail(path, f"must be an integer from 0 through {JCS_SAFE_INTEGER_MAX}")
return value
def _rfc3339(value: Any, path: str) -> str:
text = _text(value, path)
if _RFC3339_RE.fullmatch(text) is None:
_fail(path, "must be an ISO 8601 date-time with offset")
normalized = text[:10] + "T" + text[11:]
if normalized.endswith(("Z", "z")):
normalized = normalized[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(normalized)
except ValueError as exc:
raise ContractError(f"{path}: must be an ISO 8601 date-time with offset") from exc
if parsed.tzinfo is None or parsed.utcoffset() is None:
_fail(path, "must be an ISO 8601 date-time with offset")
return text
def _enum(value: Any, path: str, choices: Sequence[str]) -> str:
if value not in choices:
_fail(path, f"must be one of: {', '.join(choices)}")
return value
def _nullable_slug(value: Any, path: str) -> str | None:
if value is None:
return None
return _slug(value, path)
def _text_list(
value: Any, path: str, *, unique: bool = False
) -> list[str]:
if not isinstance(value, list):
_fail(path, "must be an array")
result = [_text(item, f"{path}[{index}]") for index, item in enumerate(value)]
if unique and len(set(result)) != len(result):
_fail(path, "must not contain duplicate identifiers")
return result
def _validate_binding(value: Any, path: str) -> dict[str, str]:
binding = _object(value, path, _BINDING_FIELDS)
return {
"profile_id": _slug(binding["profile_id"], f"{path}.profile_id"),
"profile_version": _semver(
binding["profile_version"], f"{path}.profile_version"
),
"content_sha256": _sha256(
binding["content_sha256"], f"{path}.content_sha256"
),
}
def _validate_conditions(value: Any, path: str) -> list[dict[str, str]]:
if not isinstance(value, list):
_fail(path, "must be an array")
result: list[dict[str, str]] = []
seen: set[str] = set()
for index, raw in enumerate(value):
item_path = f"{path}[{index}]"
condition = _object(
raw,
item_path,
{"condition_id", "statement"},
{"evidence_pointer"},
)
condition_id = _text(condition["condition_id"], f"{item_path}.condition_id")
if condition_id in seen:
_fail(path, f"duplicate condition_id: {condition_id}")
seen.add(condition_id)
normalized = {
"condition_id": condition_id,
"statement": _text(condition["statement"], f"{item_path}.statement"),
}
if "evidence_pointer" in condition:
normalized["evidence_pointer"] = _text(
condition["evidence_pointer"], f"{item_path}.evidence_pointer"
)
result.append(normalized)
return result
def _validate_event_shape(raw_event: Any, path: str) -> None:
event = _object(raw_event, path, _EVENT_FIELDS)
_positive_int(event["event_id"], f"{path}.event_id")
_rfc3339(event["recorded_at"], f"{path}.recorded_at")
actor = _enum(event["actor"], f"{path}.actor", ("author", "ai", "system"))
kind = _enum(event["kind"], f"{path}.kind", EVENT_KINDS)
_sha256(event["prev_event_sha256"], f"{path}.prev_event_sha256")
expected_actor = "ai" if kind in AI_EVENT_KINDS else (
"system" if kind in SYSTEM_EVENT_KINDS else "author"
)
if actor != expected_actor:
_fail(f"{path}.actor", f"must equal {expected_actor} for {kind}")
if kind in NULL_BRANCH_EVENT_KINDS:
if event["branch_id"] is not None:
_fail(f"{path}.branch_id", f"must be null for {kind}")
else:
_slug(event["branch_id"], f"{path}.branch_id")
payload_path = f"{path}.payload"
payload = event["payload"]
if kind == "branch_created":
value = _object(
payload,
payload_path,
{
"parent_id",
"statement",
"assumptions",
"evidence_sought",
"reopen_conditions",
"downstream_refs",
},
)
_nullable_slug(value["parent_id"], f"{payload_path}.parent_id")
_text(value["statement"], f"{payload_path}.statement")
_text_list(value["assumptions"], f"{payload_path}.assumptions")
_text_list(value["evidence_sought"], f"{payload_path}.evidence_sought")
_validate_conditions(
value["reopen_conditions"], f"{payload_path}.reopen_conditions"
)
_text_list(
value["downstream_refs"],
f"{payload_path}.downstream_refs",
unique=True,
)
elif kind == "facet_surfaced":
value = _object(payload, payload_path, {"parent_id", "surfaced_text"})
_nullable_slug(value["parent_id"], f"{payload_path}.parent_id")
_text(value["surfaced_text"], f"{payload_path}.surfaced_text")
elif kind == "branch_adopted":
value = _object(
payload,
payload_path,
{"source_event_id", "surfaced_text", "author_formulation"},
)
_positive_int(value["source_event_id"], f"{payload_path}.source_event_id")
surfaced = _text(value["surfaced_text"], f"{payload_path}.surfaced_text")
formulation = _text(
value["author_formulation"], f"{payload_path}.author_formulation"
)
if formulation == surfaced:
_fail(
f"{payload_path}.author_formulation",
"must not be byte-identical to surfaced_text",
)
if formulation.strip().casefold() == "ok":
_fail(
f"{payload_path}.author_formulation",
"a bare 'ok' is not an adoption formulation",
)
elif kind == "branch_annotated":
value = _object(payload, payload_path, {"field", "value"})
field = _enum(value["field"], f"{payload_path}.field", ANNOTATION_FIELDS)
if field == "reopen_conditions":
_validate_conditions(value["value"], f"{payload_path}.value")
else:
_text_list(
value["value"],
f"{payload_path}.value",
unique=field == "downstream_refs",
)
elif kind in {"branch_parked", "branch_rejected", "branch_archived"}:
value = _object(payload, payload_path, {"reason"})
_text(value["reason"], f"{payload_path}.reason")
elif kind == "branch_reopened":
value = _object(
payload,
payload_path,
{"reason"},
{"condition_id", "evidence_pointer"},
)
_text(value["reason"], f"{payload_path}.reason")
if "condition_id" in value:
_text(value["condition_id"], f"{payload_path}.condition_id")
if "evidence_pointer" in value:
_text(value["evidence_pointer"], f"{payload_path}.evidence_pointer")
elif kind == "branch_merged":
value = _object(payload, payload_path, {"merged_into", "reason"})
_slug(value["merged_into"], f"{payload_path}.merged_into")
_text(value["reason"], f"{payload_path}.reason")
elif kind == "reopen_condition_signal":
value = _object(
payload,
payload_path,
{"branch_id_ref", "condition_id", "evidence_pointer"},
)
_slug(value["branch_id_ref"], f"{payload_path}.branch_id_ref")
_text(value["condition_id"], f"{payload_path}.condition_id")
_text(value["evidence_pointer"], f"{payload_path}.evidence_pointer")
if value["branch_id_ref"] != event["branch_id"]:
_fail(
f"{payload_path}.branch_id_ref",
"must equal the event-level branch_id",
)
elif kind == "profile_rebound":
value = _object(
payload,
payload_path,
_BINDING_FIELDS | {"selection_receipt_ref"},
)
_validate_binding(
{key: value[key] for key in _BINDING_FIELDS},
payload_path,
)
_text(value["selection_receipt_ref"], f"{payload_path}.selection_receipt_ref")
elif kind == "artifact_marked_stale":
value = _object(
payload, payload_path, {"artifact_ref", "reopening_event_id"}
)
_text(value["artifact_ref"], f"{payload_path}.artifact_ref")
_positive_int(
value["reopening_event_id"], f"{payload_path}.reopening_event_id"
)
elif kind == "artifact_reconfirmed":
value = _object(
payload,
payload_path,
{"artifact_ref", "resolves_stale_event_id", "note"},
)
_text(value["artifact_ref"], f"{payload_path}.artifact_ref")
_positive_int(
value["resolves_stale_event_id"],
f"{payload_path}.resolves_stale_event_id",
)
_text(value["note"], f"{payload_path}.note")
elif kind == "artifact_superseded":
value = _object(
payload,
payload_path,
{"artifact_ref", "resolves_stale_event_id", "note", "replaced_by"},
)
artifact_ref = _text(value["artifact_ref"], f"{payload_path}.artifact_ref")
_positive_int(
value["resolves_stale_event_id"],
f"{payload_path}.resolves_stale_event_id",
)
_text(value["note"], f"{payload_path}.note")
replaced_by = _text(value["replaced_by"], f"{payload_path}.replaced_by")
if replaced_by == artifact_ref:
_fail(f"{payload_path}.replaced_by", "must differ from artifact_ref")
def validate_ledger_shape(ledger: Mapping[str, Any]) -> None:
"""Validate only the closed document/event shapes (not replay semantics)."""
value = _object(ledger, "ledger", _LEDGER_FIELDS)
if value["schema_version"] != LEDGER_SCHEMA_VERSION:
_fail("ledger.schema_version", f"must equal {LEDGER_SCHEMA_VERSION}")
_text(value["project_ref"], "ledger.project_ref")
_validate_binding(value["initial_profile_binding"], "ledger.initial_profile_binding")
events = value["events"]
if not isinstance(events, list):
_fail("ledger.events", "must be an array")
for index, event in enumerate(events):
_validate_event_shape(event, f"ledger.events[{index}]")
def _binding_key(binding: Mapping[str, Any]) -> tuple[str, str, str]:
return (
str(binding["profile_id"]),
str(binding["profile_version"]),
str(binding["content_sha256"]),
)
def build_profile_catalog(
profiles: Iterable[Mapping[str, Any]] | Mapping[Any, Mapping[str, Any]],
) -> dict[tuple[str, str, str], Mapping[str, Any]]:
"""Build an exact-binding catalog; never supplies an implicit fallback."""
if isinstance(profiles, Mapping) and "schema_version" in profiles:
candidates: Iterable[Mapping[str, Any]] = [profiles]
elif isinstance(profiles, Mapping):
candidates = profiles.values()
else:
candidates = profiles
catalog: dict[tuple[str, str, str], Mapping[str, Any]] = {}
for index, profile in enumerate(candidates):
try:
validate_profile(profile)
binding = profile_binding(profile)
except (ProfileContractError, TypeError, ValueError) as exc:
raise ContractError(f"profiles[{index}]: invalid profile: {exc}") from exc
key = _binding_key(binding)
if key in catalog and canonical_bytes(catalog[key]) != canonical_bytes(profile):
_fail(f"profiles[{index}]", "duplicate binding has different content")
catalog[key] = profile
if not catalog:
_fail("profiles", "must explicitly supply at least one exact profile document")
return catalog
def load_profile_catalog(
paths: Sequence[Path | str],
) -> dict[tuple[str, str, str], Mapping[str, Any]]:
if not paths:
_fail("profiles", "at least one --profile path is required")
loaded: list[Mapping[str, Any]] = []
for raw_path in paths:
try:
loaded.append(load_profile(Path(raw_path)))
except (ProfileContractError, OSError) as exc:
raise ContractError(f"cannot load profile {raw_path}: {exc}") from exc
return build_profile_catalog(loaded)
def _resolve_profile(
binding: Mapping[str, Any],
catalog: Mapping[tuple[str, str, str], Mapping[str, Any]],
path: str,
) -> Mapping[str, Any]:
key = _binding_key(binding)
profile = catalog.get(key)
if profile is None:
_fail(
path,
"binding is unresolved in the explicit profile catalog "
f"({key[0]}@{key[1]} {key[2]})",
)
return profile
def _is_terminal(branch: Mapping[str, Any]) -> bool:
if branch["status"] in TERMINAL_STATUSES:
return True
return (
branch["status"] == "rejected"
and branch["provenance"] == "ai_surfaced_facet"
)
def _require_branch(
branches: Mapping[str, dict[str, Any]], branch_id: Any, path: str
) -> dict[str, Any]:
branch = branches.get(str(branch_id))
if branch is None:
_fail(path, f"unknown branch_id: {branch_id}")
return branch
def _condition_map(branch: Mapping[str, Any]) -> dict[str, Mapping[str, Any]]:
return {
condition["condition_id"]: condition
for condition in branch["reopen_conditions"]
}
def _dedupe_ordered(values: Iterable[str]) -> list[str]:
return list(dict.fromkeys(values))
def _public_branch(branch: Mapping[str, Any]) -> dict[str, Any]:
return {
key: copy.deepcopy(value)
for key, value in branch.items()
if not key.startswith("_")
}
def replay_ledger(
ledger: Mapping[str, Any],
profiles: Iterable[Mapping[str, Any]] | Mapping[Any, Mapping[str, Any]],
*,
expected_project_ref: str | None = None,
require_materialized: bool = False,
) -> dict[str, Any]:
"""Purely replay a ledger and return its deterministic projection.
No file, environment, clock, model, or network state is consulted.
"""
validate_ledger_shape(ledger)
if expected_project_ref is not None:
_text(expected_project_ref, "expected_project_ref")
if ledger["project_ref"] != expected_project_ref:
_fail(
"ledger.project_ref",
f"does not equal expected project_ref {expected_project_ref!r}",
)
catalog = build_profile_catalog(profiles)
initial_binding = _validate_binding(
ledger["initial_profile_binding"], "ledger.initial_profile_binding"
)
current_profile = _resolve_profile(
initial_binding, catalog, "ledger.initial_profile_binding"
)
effective_binding = copy.deepcopy(initial_binding)
branches: dict[str, dict[str, Any]] = {}
artifacts: dict[str, dict[str, Any]] = {}
stale_causes: dict[int, dict[str, Any]] = {}
signals: list[dict[str, Any]] = []
supersession_targets: dict[str, str] = {}
expected_stale: list[tuple[str, int]] = []
introduced_count = 0
events = ledger["events"]
for index, raw_event in enumerate(events):
event = raw_event
path = f"ledger.events[{index}]"
expected_id = index + 1
if event["event_id"] != expected_id:
_fail(f"{path}.event_id", f"must equal dense event id {expected_id}")
expected_prev = (
ZERO_SHA256
if index == 0
else hashlib.sha256(canonical_bytes(events[index - 1])).hexdigest()
)
if event["prev_event_sha256"] != expected_prev:
_fail(
f"{path}.prev_event_sha256",
"does not equal the canonical SHA-256 of the previous event",
)
kind = event["kind"]
branch_id = event["branch_id"]
payload = event["payload"]
if expected_stale:
expected_artifact, reopening_id = expected_stale.pop(0)
if kind != "artifact_marked_stale":
_fail(
path,
"must be the contiguous artifact_marked_stale event "
f"for {expected_artifact!r} after reopening event {reopening_id}",
)
if payload["artifact_ref"] != expected_artifact:
_fail(
f"{path}.payload.artifact_ref",
f"must equal {expected_artifact!r} in downstream_refs order",
)
if payload["reopening_event_id"] != reopening_id:
_fail(
f"{path}.payload.reopening_event_id",
f"must equal reopening event {reopening_id}",
)
artifact = artifacts.setdefault(
expected_artifact,
{
"artifact_ref": expected_artifact,
"outstanding_stale_causes": [],
"resolution_history": [],
},
)
artifact["outstanding_stale_causes"].append(event["event_id"])
stale_causes[event["event_id"]] = {
"artifact_ref": expected_artifact,
"reopening_event_id": reopening_id,
}
elif kind == "artifact_marked_stale":
_fail(path, "orphan artifact_marked_stale event")
elif kind in {"branch_created", "facet_surfaced"}:
assert isinstance(branch_id, str)
if branch_id in branches:
_fail(f"{path}.branch_id", "branch ids are stable and cannot be reused")
parent_id = payload["parent_id"]
if parent_id is not None and parent_id not in branches:
_fail(
f"{path}.payload.parent_id",
"must name a branch introduced by an earlier event",
)
introduced_count += 1
if kind == "branch_created":
conditions = copy.deepcopy(payload["reopen_conditions"])
branches[branch_id] = {
"branch_id": branch_id,
"parent_id": parent_id,
"provenance": "author_originated",
"statement": payload["statement"],
"surfaced_text": None,
"adoption_receipt": None,
"assumptions": copy.deepcopy(payload["assumptions"]),
"evidence_sought": copy.deepcopy(payload["evidence_sought"]),
"status": "active",
"disposition_reason": None,
"reopen_conditions": conditions,
"downstream_refs": copy.deepcopy(payload["downstream_refs"]),
"merged_into": None,
"_origin_event_id": event["event_id"],
"_condition_history": {
item["condition_id"]: item["statement"] for item in conditions
},
"_retired_condition_ids": set(),
}
else:
branches[branch_id] = {
"branch_id": branch_id,
"parent_id": parent_id,
"provenance": "ai_surfaced_facet",
"statement": None,
"surfaced_text": payload["surfaced_text"],
"adoption_receipt": None,
"assumptions": [],
"evidence_sought": [],
"status": "parked",
"disposition_reason": None,
"reopen_conditions": [],
"downstream_refs": [],
"merged_into": None,
"_origin_event_id": event["event_id"],
"_condition_history": {},
"_retired_condition_ids": set(),
}
elif kind == "branch_adopted":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if branch["provenance"] != "ai_surfaced_facet" or branch["status"] != "parked":
_fail(path, "branch_adopted is lawful only on a parked unadopted facet")
if payload["source_event_id"] != branch["_origin_event_id"]:
_fail(
f"{path}.payload.source_event_id",
"must name the facet_surfaced event that introduced this branch",
)
if payload["surfaced_text"] != branch["surfaced_text"]:
_fail(
f"{path}.payload.surfaced_text",
"must retain the originating surfaced_text verbatim",
)
branch["provenance"] = "author_adopted"
branch["statement"] = payload["author_formulation"]
branch["adoption_receipt"] = copy.deepcopy(payload)
branch["status"] = "active"
elif kind == "branch_annotated":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if _is_terminal(branch):
_fail(path, "terminal branches cannot be annotated")
field = payload["field"]
replacement = copy.deepcopy(payload["value"])
if field == "reopen_conditions":
old = _condition_map(branch)
new = {item["condition_id"]: item for item in replacement}
history: dict[str, str] = branch["_condition_history"]
retired: set[str] = branch["_retired_condition_ids"]
for condition_id, condition in new.items():
if condition_id in retired:
_fail(
f"{path}.payload.value",
f"retired condition_id cannot be reused: {condition_id}",
)
historical = history.get(condition_id)
if historical is not None and historical != condition["statement"]:
_fail(
f"{path}.payload.value",
f"condition_id {condition_id!r} cannot be rebound to new text",
)
history.setdefault(condition_id, condition["statement"])
retired.update(set(old) - set(new))
branch[field] = replacement
elif kind == "branch_parked":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if branch["status"] not in {"active", "reopened"}:
_fail(path, "branch_parked requires active or reopened status")
branch["status"] = "parked"
branch["disposition_reason"] = payload["reason"]
elif kind == "branch_rejected":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if branch["status"] not in {"active", "reopened", "parked"}:
_fail(path, "branch_rejected requires active, reopened, or parked status")
branch["status"] = "rejected"
branch["disposition_reason"] = payload["reason"]
elif kind == "branch_reopened":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if branch["status"] not in {"parked", "rejected"}:
_fail(path, "branch_reopened requires parked or rejected status")
if branch["provenance"] not in {"author_originated", "author_adopted"}:
_fail(path, "an unadopted AI facet cannot be reopened")
condition_id = payload.get("condition_id")
if condition_id is not None and condition_id not in _condition_map(branch):
_fail(
f"{path}.payload.condition_id",
"must name a currently stored reopen condition",
)
branch["status"] = "reopened"
branch["disposition_reason"] = payload["reason"]
expected_stale = [
(artifact_ref, event["event_id"])
for artifact_ref in branch["downstream_refs"]
]
elif kind == "branch_merged":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if branch["status"] not in {"active", "reopened"}:
_fail(path, "branch_merged requires active or reopened source status")
target_id = payload["merged_into"]
if target_id == branch_id:
_fail(f"{path}.payload.merged_into", "a branch cannot merge into itself")
target = _require_branch(branches, target_id, f"{path}.payload.merged_into")
if target["status"] not in LIVE_STATUSES:
_fail(f"{path}.payload.merged_into", "must name a currently-live branch")
target["downstream_refs"] = _dedupe_ordered(
[*target["downstream_refs"], *branch["downstream_refs"]]
)
branch["status"] = "merged"
branch["disposition_reason"] = payload["reason"]
branch["merged_into"] = target_id
elif kind == "branch_archived":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if _is_terminal(branch):
_fail(path, "branch_archived requires a non-terminal branch")
branch["status"] = "archived"
branch["disposition_reason"] = payload["reason"]
elif kind == "reopen_condition_signal":
branch = _require_branch(branches, branch_id, f"{path}.branch_id")
if _is_terminal(branch):
_fail(path, "a terminal branch cannot receive a new reopen signal")
conditions = _condition_map(branch)
condition = conditions.get(payload["condition_id"])
if condition is None:
_fail(
f"{path}.payload.condition_id",
"must name a currently stored reopen condition",
)
signals.append(
{
"event_id": event["event_id"],
"branch_id": branch_id,
"condition_id": payload["condition_id"],
"condition_statement": condition["statement"],
"evidence_pointer": payload["evidence_pointer"],
}
)
elif kind == "profile_rebound":
rebound = _validate_binding(
{key: payload[key] for key in _BINDING_FIELDS},
f"{path}.payload",
)
if rebound == effective_binding:
_fail(path, "profile_rebound must change the effective profile binding")
replacement = _resolve_profile(rebound, catalog, f"{path}.payload")
live_count = sum(
branch["status"] in LIVE_STATUSES for branch in branches.values()
)
if live_count > replacement["branch_budget"]:
_fail(
path,
"profile_rebound would exceed the replacement profile's "
"branch_budget; dispose live branches first",
)
effective_binding = copy.deepcopy(rebound)
current_profile = replacement
elif kind in {"artifact_reconfirmed", "artifact_superseded"}:
cause_id = payload["resolves_stale_event_id"]
cause = stale_causes.get(cause_id)
if cause is None:
_fail(
f"{path}.payload.resolves_stale_event_id",
"does not name an earlier artifact_marked_stale event",
)
if cause["artifact_ref"] != payload["artifact_ref"]:
_fail(
f"{path}.payload.artifact_ref",
"does not match the named stale cause's artifact_ref",
)
artifact = artifacts[payload["artifact_ref"]]
if cause_id not in artifact["outstanding_stale_causes"]:
_fail(path, "the named stale cause has already been resolved")
artifact["outstanding_stale_causes"].remove(cause_id)
resolution = {
"event_id": event["event_id"],
"kind": kind,
"resolves_stale_event_id": cause_id,
"note": payload["note"],
}
if kind == "artifact_superseded":
old_ref = payload["artifact_ref"]
replacement_ref = payload["replaced_by"]
prior_target = supersession_targets.get(old_ref)
if prior_target is not None and prior_target != replacement_ref:
_fail(
f"{path}.payload.replaced_by",
f"must equal prior replacement {prior_target!r} for {old_ref!r}",
)
supersession_targets[old_ref] = replacement_ref
resolution["replaced_by"] = replacement_ref
for branch in branches.values():
if old_ref in branch["downstream_refs"]:
branch["downstream_refs"] = _dedupe_ordered(
replacement_ref if item == old_ref else item
for item in branch["downstream_refs"]
)
artifact["resolution_history"].append(resolution)
else: # pragma: no cover - shape validation closes the enum
_fail(path, f"unhandled event kind: {kind}")
live_count = sum(
branch["status"] in LIVE_STATUSES for branch in branches.values()
)
if live_count > current_profile["branch_budget"]:
_fail(
path,
f"post-event live branch count {live_count} exceeds effective "
f"branch_budget {current_profile['branch_budget']}",
)
if expected_stale:
artifact_ref, reopening_id = expected_stale[0]
_fail(
"ledger.events",
"ends before the required artifact_marked_stale event "
f"for {artifact_ref!r} after reopening event {reopening_id}",
)
if require_materialized and introduced_count < 2:
_fail(
"ledger.events",
"persisted ledger publication requires at least two introduced branches",
)
public_artifacts: list[dict[str, Any]] = []
for artifact in artifacts.values():
public_artifacts.append(
{
"artifact_ref": artifact["artifact_ref"],
"stale": bool(artifact["outstanding_stale_causes"]),
"outstanding_stale_causes": copy.deepcopy(
artifact["outstanding_stale_causes"]
),
"resolution_history": copy.deepcopy(artifact["resolution_history"]),
}
)
live_count = sum(
branch["status"] in LIVE_STATUSES for branch in branches.values()
)
head = ZERO_SHA256 if not events else hashlib.sha256(
canonical_bytes(events[-1])
).hexdigest()
return {
"project_ref": ledger["project_ref"],
"effective_profile_binding": copy.deepcopy(effective_binding),
"branch_budget": current_profile["branch_budget"],
"live_count": live_count,
"introduced_branch_count": introduced_count,
"branches": [_public_branch(branch) for branch in branches.values()],
"artifacts": public_artifacts,
"reopen_condition_signals": copy.deepcopy(signals),
"event_count": len(events),
"head_event_sha256": head,
}
def new_ledger(
project_ref: str, initial_profile: Mapping[str, Any]
) -> dict[str, Any]:
"""Create an in-memory empty ledger; publication waits for branch two."""
_text(project_ref, "project_ref")
try:
validate_profile(initial_profile)
binding = profile_binding(initial_profile)
except ProfileContractError as exc:
raise ContractError(f"initial_profile: {exc}") from exc
return {
"schema_version": LEDGER_SCHEMA_VERSION,
"project_ref": project_ref,
"initial_profile_binding": binding,
"events": [],
}
def _complete_event(
events: list[Mapping[str, Any]], event_input: Mapping[str, Any]
) -> dict[str, Any]:
_object(event_input, "event", _EVENT_INPUT_FIELDS)
completed = copy.deepcopy(dict(event_input))
completed["event_id"] = len(events) + 1
completed["prev_event_sha256"] = (
ZERO_SHA256
if not events
else hashlib.sha256(canonical_bytes(events[-1])).hexdigest()
)
# Put fields into one stable human-readable order. Canonical serialization
# itself sorts keys, but stable insertion order keeps replay output tidy.
result = {
"event_id": completed["event_id"],
"recorded_at": completed["recorded_at"],
"actor": completed["actor"],
"kind": completed["kind"],
"branch_id": completed["branch_id"],
"payload": completed["payload"],
"prev_event_sha256": completed["prev_event_sha256"],
}
_validate_event_shape(result, "event")
return result
def append_event(
ledger: Mapping[str, Any],
event: Mapping[str, Any],
profiles: Iterable[Mapping[str, Any]] | Mapping[Any, Mapping[str, Any]],
*,
expected_project_ref: str | None = None,
) -> dict[str, Any]:
"""Append one author/AI event and mechanically append reopen stale marks.
``event`` is the five-field pre-chain shape: recorded_at, actor, kind,
branch_id, payload. Direct system stale-mark appends are refused because
those events are emitted only as part of a branch_reopened operation.
"""
_object(event, "event", _EVENT_INPUT_FIELDS)
catalog = build_profile_catalog(profiles)
before = replay_ledger(
ledger, catalog, expected_project_ref=expected_project_ref
)
if event.get("kind") == "artifact_marked_stale":
_fail("event.kind", "artifact_marked_stale is emitted mechanically")
updated = copy.deepcopy(dict(ledger))
updated["events"] = copy.deepcopy(list(ledger["events"]))
completed = _complete_event(updated["events"], event)
updated["events"].append(completed)
if completed["kind"] == "branch_reopened":
branch = next(
(
item
for item in before["branches"]
if item["branch_id"] == completed["branch_id"]
),
None,
)
if branch is None:
# Final replay would fail too; this gives a local, comprehensible
# error before trying to derive system events from unknown state.
_fail("event.branch_id", "unknown branch_id")
for artifact_ref in branch["downstream_refs"]:
stale_input = {
"recorded_at": completed["recorded_at"],
"actor": "system",
"kind": "artifact_marked_stale",
"branch_id": None,
"payload": {
"artifact_ref": artifact_ref,
"reopening_event_id": completed["event_id"],
},
}
stale_event = _complete_event(updated["events"], stale_input)
updated["events"].append(stale_event)
replay_ledger(updated, catalog, expected_project_ref=expected_project_ref)
return updated
def inquiry_ledger_enabled(env: Mapping[str, str] | None = None) -> bool:
"""The alpha is enabled only by the exact value ``ARS_INQUIRY_LEDGER=1``."""
source = os.environ if env is None else env
return source.get(ENV_FLAG) == "1"
def _summary_text(value: Any, path: str) -> str:
"""Return one bounded line of inert Markdown-safe display text."""
text = _text(value, path)
collapsed = " ".join(text.split())
escaped: list[str] = []
escaped_length = 0
for character in collapsed:
if unicodedata.category(character).startswith("C"):
width = 4 if ord(character) <= 0xFFFF else 8
token = f"\\{'u' if width == 4 else 'U'}{ord(character):0{width}x}"
elif character in _MARKDOWN_META:
token = "\\" + character
else:
token = character
if escaped_length + len(token) > _SUMMARY_TEXT_LIMIT:
while escaped and escaped_length >= _SUMMARY_TEXT_LIMIT:
escaped_length -= len(escaped.pop())
return "".join(escaped).rstrip() + ""
escaped.append(token)
escaped_length += len(token)
return "".join(escaped)
def render_summary(
projection: Mapping[str, Any], *, signal_event_id: int | None = None
) -> str:
"""Render the compact, provenance-labelled branch summary."""
branches = projection.get("branches")
if not isinstance(branches, list):
_fail("projection.branches", "must be an array")
live_count = _nonnegative_int(projection.get("live_count"), "projection.live_count")
branch_budget = _positive_int(
projection.get("branch_budget"), "projection.branch_budget"
)
branch_by_id: dict[str, Mapping[str, Any]] = {}
for index, branch in enumerate(branches):
if not isinstance(branch, Mapping):
_fail(f"projection.branches[{index}]", "must be an object")
branch_id = _slug(
branch.get("branch_id"), f"projection.branches[{index}].branch_id"
)
if branch_id in branch_by_id:
_fail("projection.branches", f"duplicate branch_id: {branch_id}")
branch_by_id[branch_id] = branch
lines = [f"Inquiry branches (live {live_count}/{branch_budget}):"]
if signal_event_id is not None:
_positive_int(signal_event_id, "signal_event_id")
signals = projection.get("reopen_condition_signals")
if not isinstance(signals, list):
_fail("projection.reopen_condition_signals", "must be an array")
signal = None
for index, item in enumerate(signals):
signal_path = f"projection.reopen_condition_signals[{index}]"
value = _object(
item,
signal_path,
{
"event_id",
"branch_id",
"condition_id",
"condition_statement",
"evidence_pointer",
},
)
_positive_int(value["event_id"], f"{signal_path}.event_id")
if value["event_id"] == signal_event_id:
if signal is not None:
_fail("signal_event_id", "matches more than one replayed signal")
signal = value
if signal is None:
_fail("signal_event_id", "does not name a replayed reopen-condition signal")
signal_branch_id = _slug(signal["branch_id"], "signal.branch_id")
signal_branch = branch_by_id.get(signal_branch_id)
if signal_branch is None:
_fail("signal_event_id", "names a branch absent from the current projection")
status = _enum(signal_branch.get("status"), "signal.branch.status", BRANCH_STATUSES)
provenance = _enum(
signal_branch.get("provenance"),
"signal.branch.provenance",
PROVENANCE_VALUES,
)
if (
status not in AUTHOR_REOPENABLE_STATUSES
or provenance not in AUTHOR_OWNED_PROVENANCE
):
_fail(
"signal_event_id",
"branch is not currently eligible for an author-owned reopen",
)
conditions = _validate_conditions(
signal_branch.get("reopen_conditions"),
"signal.branch.reopen_conditions",
)
condition_id = _text(signal["condition_id"], "signal.condition_id")
current_condition = next(
(item for item in conditions if item["condition_id"] == condition_id),
None,
)
if current_condition is None:
_fail("signal_event_id", "condition_id is no longer current on the branch")
historical_statement = _text(
signal["condition_statement"], "signal.condition_statement"
)
if historical_statement != current_condition["statement"]:
_fail(
"signal_event_id",
"historical condition text does not equal the current condition text",
)
lines.append(
f"- {signal_branch_id} [{status}; {provenance}] — "
"AUTHOR JUDGMENT REQUIRED: "
f"{_summary_text(condition_id, 'signal.condition_id')}"
f"{_summary_text(current_condition['statement'], 'signal.condition_statement')} "
f"(evidence: {_summary_text(signal['evidence_pointer'], 'signal.evidence_pointer')})"
)
live = [branch for branch in branches if branch.get("status") in LIVE_STATUSES]
if not live:
lines.append("- No live branches.")
for branch in live:
statement = branch.get("statement") or branch.get("surfaced_text") or "(unstated)"
branch_id = _slug(branch["branch_id"], "branch.branch_id")
status = _enum(branch["status"], "branch.status", ("active", "reopened"))
provenance = _enum(
branch["provenance"], "branch.provenance", PROVENANCE_VALUES
)
lines.append(
f"- {branch_id} [{status}; {provenance}]: "
f"{_summary_text(statement, 'branch.statement')}"
)
artifacts = projection.get("artifacts", [])
if not isinstance(artifacts, list):
_fail("projection.artifacts", "must be an array")
stale_items: list[tuple[str, list[int]]] = []
for index, artifact in enumerate(artifacts):
if not isinstance(artifact, Mapping):
_fail(f"projection.artifacts[{index}]", "must be an object")
if not artifact.get("stale"):
continue
cause_ids = artifact.get("outstanding_stale_causes")
if not isinstance(cause_ids, list):
_fail(
f"projection.artifacts[{index}].outstanding_stale_causes",
"must be an array",
)
for cause_index, cause_id in enumerate(cause_ids):
_positive_int(
cause_id,
f"projection.artifacts[{index}].outstanding_stale_causes[{cause_index}]",
)
if not cause_ids:
_fail(
f"projection.artifacts[{index}].outstanding_stale_causes",
"a stale artifact must retain at least one outstanding cause",
)
stale_items.append(
(
_summary_text(
artifact.get("artifact_ref"),
f"projection.artifacts[{index}].artifact_ref",
),
cause_ids,
)
)
if stale_items:
lines.append(f"- Stale artifacts ({len(stale_items)}):")
for artifact_ref, cause_ids in stale_items:
lines.append(f" - {artifact_ref} [outstanding={len(cause_ids)}]")
lines.extend(f" - cause_event_id={cause_id}" for cause_id in cause_ids)
parked = sum(branch.get("status") == "parked" for branch in branches)
rejected = sum(branch.get("status") == "rejected" for branch in branches)
if parked or rejected:
lines.append(f"- Other dispositions: parked={parked}; rejected={rejected}.")
lines.append("Controls: skip | off | reset-to-simple-path (ledger is preserved).")
return "\n".join(lines)
def checkpoint_summary(
projection: Mapping[str, Any],
*,
moment: str,
env: Mapping[str, str] | None = None,
signal_event_id: int | None = None,
) -> str | None:
"""Return a summary only at the frozen moments and only when opt-in is on."""
_enum(moment, "moment", SUMMARY_MOMENTS)
if not inquiry_ledger_enabled(env):
return None
if int(projection.get("introduced_branch_count", 0)) <= 1:
return None
if moment == "reopen_condition_signal":
if signal_event_id is None:
_fail("signal_event_id", "is required at reopen_condition_signal moment")
elif signal_event_id is not None:
_fail("signal_event_id", "is lawful only at reopen_condition_signal moment")
return render_summary(projection, signal_event_id=signal_event_id)
def validate_pointer(pointer: Mapping[str, Any]) -> dict[str, str]:
value = _object(pointer, "inquiry_ledger_ref", _POINTER_FIELDS)
ledger_path = _workspace_relative_path(
value["ledger_path"], "inquiry_ledger_ref.ledger_path"
)
if value["ledger_version"] != POINTER_LEDGER_VERSION:
_fail(
"inquiry_ledger_ref.ledger_version",
f"must equal {POINTER_LEDGER_VERSION}",
)
return {
"ledger_path": ledger_path,
"ledger_version": POINTER_LEDGER_VERSION,
"content_sha256": _sha256(
value["content_sha256"], "inquiry_ledger_ref.content_sha256"
),
}
def _workspace_relative_path(value: Any, path: str) -> str:
text = _text(value, path)
if "\\" in text or _FORBIDDEN_PATH_CHARS_RE.search(text):
_fail(path, "must be a normalized relative POSIX path without unsafe characters")
pure = PurePosixPath(text)
if pure.is_absolute() or text.startswith("/"):
_fail(path, "must be workspace-relative")
if any(part in {"", ".", ".."} for part in pure.parts):
_fail(path, "must not contain empty, '.' or '..' path segments")
normalized = pure.as_posix()
if normalized != text or "//" in text or text.endswith("/"):
_fail(path, "must be a normalized relative POSIX path")
# PurePosixPath treats ``C:/x`` as relative; it is absolute on Windows and
# remains forbidden even though durable publication is POSIX-only.
if re.match(r"^[A-Za-z]:", text):
_fail(path, "must not be a platform-absolute path")
return text
def _workspace_root(path: Path | str) -> Path:
root = Path(path)
if not root.is_absolute():
_fail("workspace_root", "must be an explicit absolute path")
try:
resolved = root.resolve(strict=True)
except OSError as exc:
raise ContractError(f"workspace_root is not readable: {root}: {exc}") from exc
if not resolved.is_dir():
_fail("workspace_root", "must be a directory")
return resolved
def _authoritative_passport_path(
path: Path | str, root: Path
) -> tuple[str, Path]:
"""Resolve a passport without erasing evidence of a symlinked parent."""
raw = Path(path)
if raw.is_absolute():
try:
relative = raw.relative_to(root).as_posix()
except ValueError:
_fail("passport_path", "is outside workspace_root")
else:
relative = raw.as_posix()
normalized = _workspace_relative_path(relative, "passport_path")
passport = root.joinpath(*PurePosixPath(normalized).parts)
try:
resolved_parent = passport.parent.resolve(strict=True)
except OSError as exc:
raise ContractError(
f"passport_path: parent directory is not readable: {exc}"
) from exc
if resolved_parent != passport.parent:
_fail(
"passport_path",
"parent directory must not resolve through a symlink",
)
_require_regular_nonsymlink(passport, "passport_path")
return normalized, passport
def _contained_path(root: Path, relative: str, path: str) -> Path:
normalized = _workspace_relative_path(relative, path)
candidate = root.joinpath(*PurePosixPath(normalized).parts)
resolved = candidate.resolve(strict=False)
try:
resolved.relative_to(root)
except ValueError:
_fail(path, "resolves outside workspace_root")
return candidate
def _relative_to_workspace(path: Path | str, root: Path, field: str) -> str:
candidate = Path(path)
if candidate.is_absolute():
try:
# Preserve the caller's lexical target. Resolving here would erase
# a final-component or parent symlink before the non-symlink and
# beside-passport checks have a chance to reject it.
relative = candidate.relative_to(root).as_posix()
except ValueError:
_fail(field, "is outside workspace_root")
else:
relative = candidate.as_posix()
return _workspace_relative_path(relative, field)
def default_ledger_relative_path(
passport_path: Path | str, workspace_root: Path | str
) -> str:
"""Return the deterministic beside-passport orphan-discovery candidate."""
root = _workspace_root(workspace_root)
passport_relative = _relative_to_workspace(passport_path, root, "passport_path")
passport = PurePosixPath(passport_relative)
filename = f"{passport.stem}.inquiry-branch-ledger.json"
return (passport.parent / filename).as_posix()
def _require_regular_nonsymlink(path: Path, field: str, *, may_absent: bool = False) -> None:
try:
mode = path.lstat().st_mode
except FileNotFoundError:
if may_absent:
return
_fail(field, f"file does not exist: {path}")
except OSError as exc:
raise ContractError(f"{field}: cannot inspect {path}: {exc}") from exc
if stat.S_ISLNK(mode) or not stat.S_ISREG(mode):
_fail(field, "must be a regular non-symlink file")
def _yaml_runtime():
try:
from ruamel.yaml import YAML
except ImportError as exc: # pragma: no cover - dependency is CI-pinned
raise ContractError("ruamel.yaml is required for passport transactions") from exc
yaml = YAML(typ="rt")
yaml.preserve_quotes = True
yaml.allow_duplicate_keys = False
yaml.indent(mapping=2, sequence=4, offset=2)
return yaml
def _parse_passport_bytes(path: Path, raw: bytes) -> Any:
try:
text = raw.decode("utf-8")
except UnicodeDecodeError as exc:
raise ContractError(f"passport YAML is not UTF-8: {path}") from exc
yaml = _yaml_runtime()
try:
document = yaml.load(text)
except Exception as exc:
raise ContractError(f"cannot parse passport YAML {path}: {exc}") from exc
if not isinstance(document, Mapping):
_fail("passport", "top-level YAML value must be an object")
return document
def _load_passport_snapshot(path: Path) -> tuple[bytes, Any]:
_require_regular_nonsymlink(path, "passport_path")
try:
raw = path.read_bytes()
except OSError as exc:
raise ContractError(f"cannot read passport YAML {path}: {exc}") from exc
return raw, _parse_passport_bytes(path, raw)
def _load_passport_doc(path: Path) -> Any:
return _load_passport_snapshot(path)[1]
def _passport_bytes(document: Any) -> bytes:
yaml = _yaml_runtime()
stream = io.StringIO()
try:
yaml.dump(document, stream)
return stream.getvalue().encode("utf-8")
except Exception as exc:
raise ContractError(f"cannot serialize passport YAML: {exc}") from exc
def _passport_pointer(document: Mapping[str, Any]) -> dict[str, str] | None:
if "inquiry_ledger_ref" not in document:
return None
raw = document["inquiry_ledger_ref"]
if raw is None:
raise LedgerBindingError(
"inquiry_ledger_ref is present but null; omit the field for no ledger"
)
try:
return validate_pointer(raw)
except ContractError as exc:
raise LedgerBindingError(f"invalid inquiry_ledger_ref: {exc}") from exc
def _fsync_directory(path: Path) -> None:
flags = os.O_RDONLY | getattr(os, "O_DIRECTORY", 0)
fd = os.open(path, flags)
try:
os.fsync(fd)
finally:
os.close(fd)
def _write_exclusive_durable(path: Path, data: bytes, *, mode: int = 0o600) -> None:
flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL
fd = os.open(path, flags, mode)
try:
offset = 0
while offset < len(data):
written = os.write(fd, data[offset:])
if written <= 0: # pragma: no cover - defensive kernel/I/O guard
raise OSError(f"short write while staging transaction file: {path}")
offset += written
os.fsync(fd)
except BaseException:
try:
path.unlink()
except OSError:
pass
raise
finally:
os.close(fd)
def _sidecars(passport: Path) -> dict[str, Path]:
feature_base = f".{passport.name}.inquiry-ledger"
return {
# The lock is deliberately feature-neutral: every passport writer,
# including future companion transactions, shares this domain.
"lock": passport.parent / f".{passport.name}.lock",
"journal": passport.parent / f"{feature_base}.transaction.json",
"journal_temp": passport.parent / f"{feature_base}.transaction.tmp",
"passport_temp": passport.parent / f"{feature_base}.passport.tmp",
"ledger_temp": passport.parent / f"{feature_base}.ledger.tmp",
# Reserved now so a pre-#744 ledger can never occupy the companion
# register transaction's deterministic staged pathname.
"register_temp": passport.parent / f"{feature_base}.register.tmp",
}
def _ledger_temp(ledger: Path, passport: Path) -> Path:
if ledger.parent != passport.parent:
_fail("ledger_path", "must be in the same directory as the Material Passport")
return _sidecars(passport)["ledger_temp"]
def _assert_safe_ledger_target(passport: Path, ledger: Path) -> None:
def name_key(path: Path) -> str:
# APFS and HFS+ commonly compare names case-insensitively and with
# canonical-equivalence folding. Reject those aliases conservatively
# even when the current test filesystem happens to be case-sensitive.
return unicodedata.normalize("NFD", path.name).casefold()
ledger_key = name_key(ledger)
if ledger == passport or ledger_key == name_key(passport):
_fail("ledger_path", "must not equal passport_path")
if ledger.parent != passport.parent:
_fail("ledger_path", "must be beside the Material Passport")
collisions = [
name
for name, path in _sidecars(passport).items()
if ledger == path or ledger_key == name_key(path)
]
if collisions:
_fail(
"ledger_path",
"collides with reserved passport transaction path(s): "
+ ", ".join(sorted(collisions)),
)
@contextmanager
def _transaction_lock(passport: Path, *, timeout_seconds: float = 30.0) -> Iterator[None]:
# POSIX is the supported durable-publication platform for this alpha; the
# msvcrt backend has no CI coverage, so the ledger refuses rather than run
# its durable writes under an unverified lock (#845).
if file_lock.BACKEND != "fcntl":
raise ContractError(
"concurrency protection unavailable on this platform; refusing ledger access"
)
if isinstance(timeout_seconds, bool) or not isinstance(
timeout_seconds, (int, float)
):
_fail("lock_timeout", "must be a finite number between 0 and 60 seconds")
try:
normalized_timeout = float(timeout_seconds)
except (OverflowError, ValueError):
_fail("lock_timeout", "must be a finite number between 0 and 60 seconds")
if not math.isfinite(normalized_timeout) or not 0 <= normalized_timeout <= 60:
_fail("lock_timeout", "must be a finite number between 0 and 60 seconds")
lock_path = _sidecars(passport)["lock"]
if lock_path.exists() or lock_path.is_symlink():
_require_regular_nonsymlink(lock_path, "transaction_lock")
flags = os.O_RDWR | os.O_CREAT | getattr(os, "O_NOFOLLOW", 0)
fd = os.open(lock_path, flags, 0o600)
try:
if not stat.S_ISREG(os.fstat(fd).st_mode):
raise ContractError(f"transaction_lock: must be a regular file: {lock_path}")
try:
file_lock.acquire(fd, exclusive=True, timeout=normalized_timeout)
except file_lock.LockTimeout:
raise ContractError(
f"passport locked by another session: {passport}"
) from None
except BaseException:
os.close(fd)
raise
# Separate from acquisition: a LockTimeout raised inside the body must not
# be reported as "passport locked", and release runs only after a
# successful acquire (an unheld release is an error under msvcrt).
try:
yield
finally:
try:
file_lock.release(fd)
finally:
os.close(fd)
def _plain_pointer(pointer: Mapping[str, Any] | None) -> dict[str, str] | None:
if pointer is None:
return None
return {key: str(pointer[key]) for key in sorted(_POINTER_FIELDS)}
def _journal_shape(value: Mapping[str, Any]) -> Mapping[str, Any]:
fields = {
"schema_version",
"transaction_id",
"passport_path",
"ledger_path",
"passport_temp_path",
"ledger_temp_path",
"old_pointer",
"new_pointer",
"old_ledger_sha256",
"new_ledger_sha256",
"old_passport_sha256",
"new_passport_sha256",
}
journal = _object(value, "transaction", fields)
if journal["schema_version"] != TRANSACTION_SCHEMA_VERSION:
_fail("transaction.schema_version", f"must equal {TRANSACTION_SCHEMA_VERSION}")
_text(journal["transaction_id"], "transaction.transaction_id")
for field in (
"passport_path",
"ledger_path",
"passport_temp_path",
"ledger_temp_path",
):
_workspace_relative_path(journal[field], f"transaction.{field}")
old_pointer = (
None
if journal["old_pointer"] is None
else validate_pointer(journal["old_pointer"])
)
new_pointer = validate_pointer(journal["new_pointer"])
if journal["old_ledger_sha256"] is not None:
_sha256(journal["old_ledger_sha256"], "transaction.old_ledger_sha256")
_sha256(journal["new_ledger_sha256"], "transaction.new_ledger_sha256")
_sha256(journal["old_passport_sha256"], "transaction.old_passport_sha256")
_sha256(journal["new_passport_sha256"], "transaction.new_passport_sha256")
if new_pointer["ledger_path"] != journal["ledger_path"]:
_fail("transaction.new_pointer", "ledger_path does not equal transaction ledger_path")
if new_pointer["content_sha256"] != journal["new_ledger_sha256"]:
_fail("transaction.new_pointer", "digest does not equal new_ledger_sha256")
if old_pointer is None:
if journal["old_ledger_sha256"] is not None:
_fail(
"transaction.old_ledger_sha256",
"must be null when old_pointer is null",
)
else:
if old_pointer["ledger_path"] != journal["ledger_path"]:
_fail(
"transaction.old_pointer",
"ledger_path does not equal transaction ledger_path",
)
if journal["old_ledger_sha256"] != old_pointer["content_sha256"]:
_fail(
"transaction.old_ledger_sha256",
"must equal old_pointer.content_sha256",
)
return journal
def _path_digest(path: Path, *, field: str = "transaction_target") -> str | None:
try:
mode = path.lstat().st_mode
except FileNotFoundError:
return None
except OSError as exc:
raise TransactionRecoveryError(
f"cannot inspect {field} {path}: {exc}"
) from exc
if stat.S_ISLNK(mode) or not stat.S_ISREG(mode):
raise TransactionRecoveryError(
f"{field} must be a regular non-symlink file: {path}"
)
try:
raw = path.read_bytes()
except OSError as exc:
raise TransactionRecoveryError(f"cannot read {field} {path}: {exc}") from exc
return hashlib.sha256(raw).hexdigest()
def _cleanup_unjournaled_temps(passport: Path, ledger: Path | None = None) -> None:
paths = [_sidecars(passport)["journal_temp"], _sidecars(passport)["passport_temp"]]
if ledger is not None:
paths.append(_ledger_temp(ledger, passport))
for path in paths:
if not path.exists() and not path.is_symlink():
continue
_require_regular_nonsymlink(path, "transaction_temp")
try:
path.unlink()
except OSError as exc:
raise TransactionRecoveryError(f"cannot clear unjournaled temp {path}: {exc}") from exc
_fsync_directory(path.parent)
def _write_journal(path: Path, temp: Path, journal: Mapping[str, Any]) -> None:
if path.exists() or path.is_symlink():
raise TransactionRecoveryError(f"existing transaction journal: {path}")
if temp.exists() or temp.is_symlink():
raise TransactionRecoveryError(f"stale transaction journal temp: {temp}")
journal_bytes = canonical_bytes(journal)
_write_exclusive_durable(temp, journal_bytes)
_require_regular_nonsymlink(temp, "transaction_journal_temp")
try:
if temp.read_bytes() != journal_bytes:
raise TransactionRecoveryError(
"transaction journal temp changed before publication"
)
except OSError as exc:
raise TransactionRecoveryError(
f"cannot verify transaction journal temp {temp}: {exc}"
) from exc
if path.exists() or path.is_symlink():
raise TransactionRecoveryError(f"transaction journal appeared concurrently: {path}")
os.replace(temp, path)
_fsync_directory(path.parent)
def _recovery_passport_snapshot(
path: Path, field: str
) -> tuple[bytes, Mapping[str, Any]]:
try:
raw, document = _load_passport_snapshot(path)
except ContractError as exc:
raise TransactionRecoveryError(f"invalid {field} {path}: {exc}") from exc
return raw, document
def _recovery_passport_pointer(
document: Mapping[str, Any], field: str
) -> dict[str, str] | None:
try:
return _passport_pointer(document)
except ContractError as exc:
raise TransactionRecoveryError(f"invalid {field} pointer: {exc}") from exc
def _recover_transaction_locked(passport: Path, root: Path) -> bool:
sidecars = _sidecars(passport)
journal_path = sidecars["journal"]
if not journal_path.exists():
if journal_path.is_symlink():
raise TransactionRecoveryError("journal path is a symlink")
return False
try:
_require_regular_nonsymlink(journal_path, "transaction_journal")
except ContractError as exc:
raise TransactionRecoveryError(f"invalid journal {journal_path}: {exc}") from exc
try:
raw = journal_path.read_bytes()
value = _parse_json_bytes(raw, source=str(journal_path))
if raw != canonical_bytes(value):
raise ContractError("journal is not exact canonical JSON")
journal = _journal_shape(value)
except (OSError, ContractError) as exc:
raise TransactionRecoveryError(f"invalid journal {journal_path}: {exc}") from exc
expected_passport_rel = _relative_to_workspace(passport, root, "passport_path")
if journal["passport_path"] != expected_passport_rel:
raise TransactionRecoveryError("journal passport_path does not match lock owner")
ledger = _contained_path(root, journal["ledger_path"], "transaction.ledger_path")
passport_temp = _contained_path(
root, journal["passport_temp_path"], "transaction.passport_temp_path"
)
ledger_temp = _contained_path(
root, journal["ledger_temp_path"], "transaction.ledger_temp_path"
)
try:
_assert_safe_ledger_target(passport, ledger)
except ContractError as exc:
raise TransactionRecoveryError(f"invalid journal ledger target: {exc}") from exc
expected_passport_temp = sidecars["passport_temp"]
expected_ledger_temp = _ledger_temp(ledger, passport)
if passport_temp != expected_passport_temp:
raise TransactionRecoveryError(
"journal passport_temp_path is not the deterministic reserved path"
)
if ledger_temp != expected_ledger_temp:
raise TransactionRecoveryError(
"journal ledger_temp_path is not the deterministic reserved path"
)
new_digest = journal["new_ledger_sha256"]
old_digest = journal["old_ledger_sha256"]
ledger_digest = _path_digest(ledger, field="ledger target")
ledger_temp_present = ledger_temp.exists() or ledger_temp.is_symlink()
if ledger_temp_present:
temp_digest = _path_digest(ledger_temp, field="staged ledger")
if temp_digest != new_digest:
raise TransactionRecoveryError(
"staged ledger is not the journal's complete new byte image"
)
if ledger_digest == new_digest:
publish_ledger = False
elif ledger_digest == old_digest:
if not ledger_temp_present:
raise TransactionRecoveryError("new ledger temp is absent")
publish_ledger = True
else:
raise TransactionRecoveryError(
"ledger target is neither the recorded old nor new generation"
)
old_pointer = (
None if journal["old_pointer"] is None else validate_pointer(journal["old_pointer"])
)
new_pointer = validate_pointer(journal["new_pointer"])
old_passport_digest = journal["old_passport_sha256"]
new_passport_digest = journal["new_passport_sha256"]
staged_passport_present = passport_temp.exists() or passport_temp.is_symlink()
if staged_passport_present:
staged_raw, staged_document = _recovery_passport_snapshot(
passport_temp, "staged passport"
)
if hashlib.sha256(staged_raw).hexdigest() != new_passport_digest:
raise TransactionRecoveryError(
"staged passport is not the journal's complete new byte image"
)
if _recovery_passport_pointer(staged_document, "staged passport") != new_pointer:
raise TransactionRecoveryError("staged passport does not carry new pointer")
live_raw, live_document = _recovery_passport_snapshot(passport, "live passport")
live_passport_digest = hashlib.sha256(live_raw).hexdigest()
current_pointer = _recovery_passport_pointer(live_document, "live passport")
if live_passport_digest == new_passport_digest:
publish_passport = False
if current_pointer != new_pointer:
raise TransactionRecoveryError(
"new passport byte image does not carry the recorded new pointer"
)
elif live_passport_digest == old_passport_digest:
publish_passport = True
if current_pointer != old_pointer:
raise TransactionRecoveryError(
"old passport byte image does not carry the recorded old pointer"
)
if not staged_passport_present:
raise TransactionRecoveryError("new passport temp is absent")
else:
raise TransactionRecoveryError(
"live passport is neither the recorded old nor new byte generation"
)
# All journal relations, sources, and live generations are valid before
# recovery advances either member of the pair.
if publish_ledger:
if _path_digest(ledger, field="ledger target CAS") != old_digest:
raise TransactionRecoveryError("ledger target changed during recovery")
if _path_digest(ledger_temp, field="staged ledger CAS") != new_digest:
raise TransactionRecoveryError("staged ledger changed during recovery")
os.replace(ledger_temp, ledger)
_fsync_directory(ledger.parent)
if publish_passport:
check_raw, check_document = _recovery_passport_snapshot(
passport, "live passport CAS"
)
if hashlib.sha256(check_raw).hexdigest() != old_passport_digest:
raise TransactionRecoveryError("live passport changed during recovery")
if _recovery_passport_pointer(check_document, "live passport CAS") != old_pointer:
raise TransactionRecoveryError("live passport pointer changed during recovery")
check_staged_raw, check_staged_document = _recovery_passport_snapshot(
passport_temp, "staged passport CAS"
)
if hashlib.sha256(check_staged_raw).hexdigest() != new_passport_digest:
raise TransactionRecoveryError("staged passport changed during recovery")
if (
_recovery_passport_pointer(
check_staged_document, "staged passport CAS"
)
!= new_pointer
):
raise TransactionRecoveryError("staged passport pointer changed during recovery")
os.replace(passport_temp, passport)
_fsync_directory(passport.parent)
final_raw, final_document = _recovery_passport_snapshot(
passport, "recovered passport"
)
if hashlib.sha256(final_raw).hexdigest() != new_passport_digest:
raise TransactionRecoveryError(
"recovered passport digest is not the new passport digest"
)
if _recovery_passport_pointer(final_document, "recovered passport") != new_pointer:
raise TransactionRecoveryError("recovered passport pointer is not the new pointer")
if _path_digest(ledger, field="recovered ledger") != new_digest:
raise TransactionRecoveryError("recovered ledger digest is not the new digest")
for temp in (passport_temp, ledger_temp):
if temp.exists() or temp.is_symlink():
try:
_require_regular_nonsymlink(temp, "transaction_temp")
except ContractError as exc:
raise TransactionRecoveryError(
f"invalid transaction temp during cleanup: {exc}"
) from exc
temp.unlink()
try:
_require_regular_nonsymlink(journal_path, "transaction_journal")
except ContractError as exc:
raise TransactionRecoveryError(
f"transaction journal changed during recovery: {exc}"
) from exc
journal_path.unlink()
_fsync_directory(passport.parent)
return True
def recover_ledger_transaction(
passport_path: Path | str,
workspace_root: Path | str,
*,
lock_timeout: float = 30.0,
) -> bool:
"""Recover one interrupted two-file transaction, if a journal exists."""
root = _workspace_root(workspace_root)
_, passport = _authoritative_passport_path(passport_path, root)
with _transaction_lock(passport, timeout_seconds=lock_timeout):
return _recover_transaction_locked(passport, root)
def _load_bound_locked(
passport: Path,
root: Path,
catalog: Mapping[tuple[str, str, str], Mapping[str, Any]],
*,
expected_project_ref: str,
orphan_relative: str,
) -> dict[str, Any]:
document = _load_passport_doc(passport)
pointer = _passport_pointer(document)
if pointer is None:
orphan = _contained_path(root, orphan_relative, "orphan_candidate")
if orphan.exists() or orphan.is_symlink():
return {
"state": "orphan_ignored",
"notice": (
"ledger file exists without inquiry_ledger_ref; passport is "
"authoritative, so the file is ignored"
),
"ledger": None,
"projection": None,
"pointer": None,
"ledger_path": str(orphan),
}
return {
"state": "absent",
"notice": None,
"ledger": None,
"projection": None,
"pointer": None,
"ledger_path": None,
}
ledger_path = _contained_path(
root, pointer["ledger_path"], "inquiry_ledger_ref.ledger_path"
)
try:
_assert_safe_ledger_target(passport, ledger_path)
except ContractError as exc:
raise LedgerBindingError(str(exc)) from exc
try:
_require_regular_nonsymlink(ledger_path, "ledger_path")
raw = ledger_path.read_bytes()
ledger = _parse_json_bytes(raw, source=str(ledger_path))
if raw != canonical_bytes(ledger):
raise ContractError("ledger storage is not exact canonical JSON")
digest = hashlib.sha256(raw).hexdigest()
if digest != pointer["content_sha256"]:
raise ContractError(
f"pointer digest {pointer['content_sha256']} does not match ledger {digest}"
)
if ledger.get("schema_version") != pointer["ledger_version"]:
raise ContractError("pointer ledger_version does not match ledger schema_version")
projection = replay_ledger(
ledger,
catalog,
expected_project_ref=expected_project_ref,
require_materialized=True,
)
except (OSError, ContractError) as exc:
if isinstance(exc, LedgerBindingError):
raise
raise LedgerBindingError(str(exc)) from exc
return {
"state": "bound",
"notice": None,
"ledger": ledger,
"projection": projection,
"pointer": pointer,
"ledger_path": str(ledger_path),
}
def load_bound_ledger(
passport_path: Path | str,
workspace_root: Path | str,
profiles: Iterable[Mapping[str, Any]] | Mapping[Any, Mapping[str, Any]],
*,
expected_project_ref: str | None = None,
orphan_candidate: Path | str | None = None,
recover: bool = True,
lock_timeout: float = 30.0,
) -> dict[str, Any]:
"""Load the passport-authoritative ledger binding under the stable lock."""
root = _workspace_root(workspace_root)
expected = _text(expected_project_ref, "expected_project_ref")
_, passport = _authoritative_passport_path(passport_path, root)
catalog = build_profile_catalog(profiles)
orphan_relative = (
default_ledger_relative_path(passport, root)
if orphan_candidate is None
else _relative_to_workspace(orphan_candidate, root, "orphan_candidate")
)
with _transaction_lock(passport, timeout_seconds=lock_timeout):
journal = _sidecars(passport)["journal"]
if journal.exists() or journal.is_symlink():
if not recover:
raise TransactionRecoveryError("pending journal; recovery disabled")
_recover_transaction_locked(passport, root)
return _load_bound_locked(
passport,
root,
catalog,
expected_project_ref=expected,
orphan_relative=orphan_relative,
)
def _assert_append_only(old: Mapping[str, Any], new: Mapping[str, Any]) -> None:
for field in ("schema_version", "project_ref", "initial_profile_binding"):
if new[field] != old[field]:
_fail(f"ledger.{field}", "is immutable after materialization")
old_events = old["events"]
new_events = new["events"]
if len(new_events) <= len(old_events):
_fail("ledger.events", "publication must append at least one new event")
if new_events[: len(old_events)] != old_events:
_fail("ledger.events", "prior event history was rewritten or reordered")
def commit_ledger_transaction(
passport_path: Path | str,
ledger_path: Path | str,
workspace_root: Path | str,
ledger: Mapping[str, Any],
profiles: Iterable[Mapping[str, Any]] | Mapping[Any, Mapping[str, Any]],
project_ref: str | None = None,
*,
lock_timeout: float = 30.0,
crash_hook: Callable[[str], None] | None = None,
) -> dict[str, str]:
"""Durably publish ledger bytes and their passport pointer as one transaction.
``crash_hook`` is a deterministic test seam called after these durable
boundaries: ``temps_durable``, ``journal_durable``, ``ledger_published``,
``passport_published``, and ``journal_cleared``. Raising at or after
``journal_durable`` deliberately leaves recovery state intact.
"""
root = _workspace_root(workspace_root)
expected_project_ref = _text(project_ref, "expected_project_ref")
passport_relative, passport = _authoritative_passport_path(passport_path, root)
ledger_relative = _relative_to_workspace(ledger_path, root, "ledger_path")
ledger_target = _contained_path(root, ledger_relative, "ledger_path")
_assert_safe_ledger_target(passport, ledger_target)
_require_regular_nonsymlink(ledger_target, "ledger_path", may_absent=True)
catalog = build_profile_catalog(profiles)
replay_ledger(
ledger,
catalog,
expected_project_ref=expected_project_ref,
require_materialized=True,
)
ledger_bytes = canonical_bytes(ledger)
new_digest = hashlib.sha256(ledger_bytes).hexdigest()
new_pointer = {
"ledger_path": ledger_relative,
"ledger_version": POINTER_LEDGER_VERSION,
"content_sha256": new_digest,
}
validate_pointer(new_pointer)
hook = crash_hook or (lambda _phase: None)
sidecars = _sidecars(passport)
ledger_temp = _ledger_temp(ledger_target, passport)
journal_durable = False
with _transaction_lock(passport, timeout_seconds=lock_timeout):
if sidecars["journal"].exists() or sidecars["journal"].is_symlink():
_recover_transaction_locked(passport, root)
_cleanup_unjournaled_temps(passport, ledger_target)
old_passport_raw, document = _load_passport_snapshot(passport)
old_passport_digest = hashlib.sha256(old_passport_raw).hexdigest()
old_pointer = _passport_pointer(document)
old_digest: str | None = None
if old_pointer is None:
if ledger_target.exists() or ledger_target.is_symlink():
raise ContractError(
"ledger file exists without a passport pointer; it is an ignored "
"orphan and will not be overwritten or silently adopted"
)
else:
if old_pointer["ledger_path"] != ledger_relative:
_fail(
"ledger_path",
"must equal the existing passport pointer's stable ledger_path",
)
try:
_require_regular_nonsymlink(ledger_target, "ledger_path")
old_raw = ledger_target.read_bytes()
old_ledger = _parse_json_bytes(old_raw, source=str(ledger_target))
if old_raw != canonical_bytes(old_ledger):
raise ContractError("existing ledger is not exact canonical JSON")
old_digest = hashlib.sha256(old_raw).hexdigest()
if old_digest != old_pointer["content_sha256"]:
raise ContractError("existing pointer digest does not match ledger")
replay_ledger(
old_ledger,
catalog,
expected_project_ref=expected_project_ref,
require_materialized=True,
)
_assert_append_only(old_ledger, ledger)
except (OSError, ContractError) as exc:
raise LedgerBindingError(str(exc)) from exc
document["inquiry_ledger_ref"] = copy.deepcopy(new_pointer)
passport_bytes = _passport_bytes(document)
new_passport_digest = hashlib.sha256(passport_bytes).hexdigest()
_write_exclusive_durable(ledger_temp, ledger_bytes)
try:
passport_mode = stat.S_IMODE(passport.stat().st_mode)
_write_exclusive_durable(
sidecars["passport_temp"], passport_bytes, mode=passport_mode
)
except BaseException:
ledger_temp.unlink(missing_ok=True)
raise
hook("temps_durable")
journal = {
"schema_version": TRANSACTION_SCHEMA_VERSION,
"transaction_id": uuid.uuid4().hex,
"passport_path": passport_relative,
"ledger_path": ledger_relative,
"passport_temp_path": _relative_to_workspace(
sidecars["passport_temp"], root, "passport_temp_path"
),
"ledger_temp_path": _relative_to_workspace(
ledger_temp, root, "ledger_temp_path"
),
"old_pointer": _plain_pointer(old_pointer),
"new_pointer": _plain_pointer(new_pointer),
"old_ledger_sha256": old_digest,
"new_ledger_sha256": new_digest,
"old_passport_sha256": old_passport_digest,
"new_passport_sha256": new_passport_digest,
}
_journal_shape(journal)
try:
_write_journal(sidecars["journal"], sidecars["journal_temp"], journal)
journal_durable = True
hook("journal_durable")
if _path_digest(ledger_temp, field="staged ledger") != new_digest:
raise TransactionRecoveryError("staged ledger changed before publication")
if _path_digest(ledger_target, field="live ledger") != old_digest:
raise TransactionRecoveryError("live ledger changed before publication")
preflight_passport_raw, preflight_passport = _recovery_passport_snapshot(
sidecars["passport_temp"], "staged passport preflight"
)
if (
hashlib.sha256(preflight_passport_raw).hexdigest()
!= new_passport_digest
):
raise TransactionRecoveryError(
"staged passport changed before publication"
)
if (
_recovery_passport_pointer(
preflight_passport, "staged passport preflight"
)
!= new_pointer
):
raise TransactionRecoveryError(
"staged passport pointer changed before publication"
)
preflight_live_raw, preflight_live = _recovery_passport_snapshot(
passport, "live passport preflight"
)
if (
hashlib.sha256(preflight_live_raw).hexdigest()
!= old_passport_digest
):
raise TransactionRecoveryError("live passport changed before publication")
if (
_recovery_passport_pointer(preflight_live, "live passport preflight")
!= old_pointer
):
raise TransactionRecoveryError(
"live passport pointer changed before publication"
)
os.replace(ledger_temp, ledger_target)
_fsync_directory(ledger_target.parent)
hook("ledger_published")
staged_passport_raw, staged_passport = _recovery_passport_snapshot(
sidecars["passport_temp"], "staged passport"
)
if hashlib.sha256(staged_passport_raw).hexdigest() != new_passport_digest:
raise TransactionRecoveryError(
"staged passport changed before publication"
)
if (
_recovery_passport_pointer(staged_passport, "staged passport")
!= new_pointer
):
raise TransactionRecoveryError(
"staged passport pointer changed before publication"
)
live_passport_raw, live_passport = _recovery_passport_snapshot(
passport, "live passport CAS"
)
if hashlib.sha256(live_passport_raw).hexdigest() != old_passport_digest:
raise TransactionRecoveryError("live passport changed before publication")
if (
_recovery_passport_pointer(live_passport, "live passport CAS")
!= old_pointer
):
raise TransactionRecoveryError(
"live passport pointer changed before publication"
)
os.replace(sidecars["passport_temp"], passport)
_fsync_directory(passport.parent)
hook("passport_published")
# Verify the committed pair before deleting its recovery witness.
committed_raw, committed_document = _recovery_passport_snapshot(
passport, "committed passport"
)
if hashlib.sha256(committed_raw).hexdigest() != new_passport_digest:
raise TransactionRecoveryError("committed passport digest mismatch")
if (
_recovery_passport_pointer(committed_document, "committed passport")
!= new_pointer
):
raise TransactionRecoveryError("committed passport pointer mismatch")
if _path_digest(ledger_target, field="committed ledger") != new_digest:
raise TransactionRecoveryError("committed ledger digest mismatch")
_require_regular_nonsymlink(sidecars["journal"], "transaction_journal")
sidecars["journal"].unlink()
_fsync_directory(passport.parent)
hook("journal_cleared")
except BaseException:
# os.replace(journal_temp, journal) may have succeeded before a
# directory-fsync error was reported. If the journal name is
# visible, preserve both staged generations for recovery instead
# of converting a recoverable transaction into a broken one.
journal_durable = journal_durable or sidecars["journal"].exists()
if not journal_durable:
for temp in (
ledger_temp,
sidecars["passport_temp"],
sidecars["journal_temp"],
):
try:
temp.unlink(missing_ok=True)
except OSError:
pass
raise
return new_pointer
def _emit_json(value: Any) -> None:
print(json.dumps(value, ensure_ascii=False, indent=2, sort_keys=True))
def _event_input(path: Path) -> Mapping[str, Any]:
try:
raw = path.read_bytes()
except OSError as exc:
raise ContractError(f"cannot read event input {path}: {exc}") from exc
value = _parse_json_bytes(raw, source=str(path))
_object(value, "event", _EVENT_INPUT_FIELDS)
return value
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
subparsers = parser.add_subparsers(dest="command", required=True)
def add_ledger_inputs(
command_parser: argparse.ArgumentParser, *, authoritative: bool = False
) -> None:
command_parser.add_argument("ledger", type=Path)
command_parser.add_argument(
"--profile", action="append", required=True, type=Path
)
command_parser.add_argument(
"--project-ref",
required=authoritative,
help=(
"required project identity for authoritative mutation/display; "
"optional only for diagnostic validate/replay"
),
)
validate_parser = subparsers.add_parser("validate")
add_ledger_inputs(validate_parser)
validate_parser.add_argument("--allow-in-memory", action="store_true")
replay_parser = subparsers.add_parser("replay")
add_ledger_inputs(replay_parser)
replay_parser.add_argument("--allow-in-memory", action="store_true")
append_parser = subparsers.add_parser("append")
add_ledger_inputs(append_parser, authoritative=True)
append_parser.add_argument("--event", required=True, type=Path)
summary_parser = subparsers.add_parser("summary")
add_ledger_inputs(summary_parser, authoritative=True)
summary_parser.add_argument("--moment", choices=SUMMARY_MOMENTS, required=True)
summary_parser.add_argument("--signal-event-id", type=int)
summary_parser.add_argument(
"--force",
action="store_true",
help="render for diagnostics even when ARS_INQUIRY_LEDGER is not 1",
)
load_parser = subparsers.add_parser("load-bound")
load_parser.add_argument("--passport", required=True, type=Path)
load_parser.add_argument("--workspace-root", required=True, type=Path)
load_parser.add_argument("--profile", action="append", required=True, type=Path)
load_parser.add_argument("--project-ref", required=True)
load_parser.add_argument("--orphan-candidate", type=Path)
load_parser.add_argument("--no-recover", action="store_true")
commit_parser = subparsers.add_parser("commit")
commit_parser.add_argument("--passport", required=True, type=Path)
commit_parser.add_argument("--workspace-root", required=True, type=Path)
commit_parser.add_argument("--ledger-path", required=True, type=Path)
commit_parser.add_argument("--ledger", required=True, type=Path)
commit_parser.add_argument("--profile", action="append", required=True, type=Path)
commit_parser.add_argument("--project-ref", required=True)
args = parser.parse_args(argv)
try:
if args.command in {"validate", "replay", "append", "summary"}:
catalog = load_profile_catalog(args.profile)
ledger = load_ledger(args.ledger)
if args.command == "validate":
replay_ledger(
ledger,
catalog,
expected_project_ref=args.project_ref,
require_materialized=not args.allow_in_memory,
)
_emit_json({"status": "valid", "schema_version": LEDGER_SCHEMA_VERSION})
elif args.command == "replay":
_emit_json(
replay_ledger(
ledger,
catalog,
expected_project_ref=args.project_ref,
require_materialized=not args.allow_in_memory,
)
)
elif args.command == "append":
updated = append_event(
ledger,
_event_input(args.event),
catalog,
expected_project_ref=args.project_ref,
)
sys.stdout.buffer.write(canonical_bytes(updated))
elif args.command == "summary":
projection = replay_ledger(
ledger, catalog, expected_project_ref=args.project_ref
)
env = {ENV_FLAG: "1"} if args.force else None
rendered = checkpoint_summary(
projection,
moment=args.moment,
env=env,
signal_event_id=args.signal_event_id,
)
if rendered is not None:
print(rendered)
elif args.command == "load-bound":
catalog = load_profile_catalog(args.profile)
result = load_bound_ledger(
args.passport,
args.workspace_root,
catalog,
expected_project_ref=args.project_ref,
orphan_candidate=args.orphan_candidate,
recover=not args.no_recover,
)
_emit_json(result)
elif args.command == "commit":
catalog = load_profile_catalog(args.profile)
ledger = load_ledger(args.ledger)
pointer = commit_ledger_transaction(
args.passport,
args.ledger_path,
args.workspace_root,
ledger,
catalog,
args.project_ref,
)
_emit_json(pointer)
return 0
except (ContractError, ProfileContractError) as exc:
print(f"[inquiry_branch_ledger ERROR] {exc}", file=sys.stderr)
return 2
if __name__ == "__main__":
sys.exit(main())