mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
* fix(evals): #637 Markdown-decoration tolerance for conformance machine lines Every synthesis first attempt of the 2026-08-03 v0.2 baseline cohort (6/6) emitted the four mechanical audit lines inside a CommonMark code fence and failed [SYNTHESIS-PARSE: found 0]; one a2 retry re-emitted them as inline code spans and aborted the panel; one domain seat self-superseded its Severity declaration and shrank the panel. All three shapes render the content verbatim and visibly to a reader, so rejecting them is a false abort, not a leak guard. - check_panel_synthesis: new audit_candidate_lines keeps fenced content (fence-marker lines dropped, same open/close state machine) and unwraps whole-line inline code spans; the four audit-line grammars read from it. _one_body collapses byte-identical duplicates; distinct values stay a loud abort. Rationale and DA-marker grammars keep the plain-line source (declared boundary, pinned by accepted-miss tests). - check_phase_conformance: a finding's Severity declarations take the LAST value when all parse and all values are distinct (cross-line explicit supersession, #637 ms01_quant r1), with a [SEVERITY-SUPERSEDED] advisory in the gate log; zero, unparseable, and repeated-identical declarations keep the loud abort (anti-bundling guard unchanged). All three blocked artifacts of the cohort now pass; the four previously passing panels are unchanged. 596 checker tests green; full local suite 4984 passed / 3 skipped / 1 xfailed. Closes #637 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XEvPjWxkQjVDb9FuPCRACd * fix(evals): review-round hardening — escalation-only supersession + rendering-parity candidates (#637) Dual-track review round (security opus + codex gpt-5.6-sol xhigh, 8 P2s, all first-party verified) on the decoration-tolerance patch: - Severity supersession is now ESCALATION-ONLY (Minor < Major < Critical, strictly increasing chain): last-wins de-escalation could waive the Critical/Major Evidence-Anchor hard gate with one appended weaker line, and non-monotone chains are bundling, not self-correction. Two parseable declarations on ONE line (table-pipe form) are excluded from the supersession path entirely. - audit_candidate_lines now normalizes candidates to their CommonMark DISPLAYED form: padded inline-span bodies get the one-space strip, indented-fence content is dedented by the opener's indent. - _one_body docstring corrected to the implemented value-level collapse (not byte-identical); the fenced forbidden-marker asymmetry is now an explicitly documented accepted boundary with a pinning assertion. - Test strength: fence-STATE pinning via wrapped decoys after malformed / short / mismatched / Unicode-separator closers, CRLF, unterminated fence, operative-value anchor pin (Minor->Major without anchor aborts), de-escalation and bundle aborts, padded-span conflict abort. 612 pass in the two checker test files; full local suite 5234 passed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QerPGR5oNvpjqy3Sur39h * docs(changelog): #637 decoration-tolerance entry under [Unreleased] Fixed Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013QerPGR5oNvpjqy3Sur39h --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Edward Cheng-I Wu
committed by
GitHub
GitHub
parent
49e79a7c99
commit
7aaa635a10
File diff suppressed because one or more lines are too long
@@ -164,6 +164,82 @@ def strip_fences(
|
||||
return out
|
||||
|
||||
|
||||
_INLINE_CODE_LINE_RE = re.compile(
|
||||
r"^ {0,3}(?P<ticks>`+)(?P<body>[^`]+?)(?P=ticks)[ \t]*$"
|
||||
)
|
||||
|
||||
|
||||
def audit_candidate_lines(text: str) -> list[str]:
|
||||
"""Candidate lines for the synthesis mechanical audit-line grammars.
|
||||
|
||||
The four audit lines (``dimension_verdicts`` / ``fired_conditions`` /
|
||||
``da_critical_adjudications`` / bare ``editorial_decision=``) are
|
||||
routinely emitted inside a CommonMark code fence, or wrapped whole in an
|
||||
inline code span — both render the line verbatim and visibly to a
|
||||
reader, so rejecting them is a false abort, not a leak guard (#637:
|
||||
every synthesis first attempt of the 2026-08-03 baseline cohort, 6/6,
|
||||
fenced the audit block; one retry re-emitted it as inline code spans).
|
||||
This walker therefore keeps fenced content (dropping only the
|
||||
fence-marker lines themselves, with the same open/close state machine as
|
||||
``strip_fences``) and additionally yields the unwrapped body of any
|
||||
non-fenced line that is entirely one inline code span. Candidates are
|
||||
normalized to what a CommonMark renderer would DISPLAY, since rendered
|
||||
visibility is the acceptance criterion: an inline span whose body has
|
||||
both a leading and a trailing space renders with one space stripped
|
||||
from each side, and lines inside an indented fence render dedented by
|
||||
up to the opener's indent — both normalizations are applied here so
|
||||
the start-anchored grammars see the displayed text.
|
||||
|
||||
Declared boundaries (ACCEPTED misses — change deliberately, with
|
||||
tests): blockquoted or emphasis-wrapped audit lines never match the
|
||||
line-anchored grammars and stay non-candidates; an inline code span
|
||||
inside a fence renders its backticks literally and is not unwrapped;
|
||||
the rejection-rationale and DA-consistency-marker grammars keep the
|
||||
plain-line source in ``parse_synthesis`` because both are quotable
|
||||
diagnostics whose fenced or wrapped occurrences read as quotation.
|
||||
One asymmetry of that boundary is accepted deliberately: a fenced
|
||||
marker in a state where the marker is forbidden stays invisible to
|
||||
``check_da_terminal_gate`` and no longer aborts upstream (pre-#637 the
|
||||
whole fenced block aborted at ``fired_conditions``); the editorial
|
||||
decision itself is independently recomputed from the reviewer cards by
|
||||
the layer-2 check, so the slip is an accounting cosmetic, not a
|
||||
decision channel.
|
||||
"""
|
||||
out: list[str] = []
|
||||
fence_char: str | None = None
|
||||
fence_len = 0
|
||||
fence_indent = 0
|
||||
for line in _COMMONMARK_LINE_END_RE.split(text):
|
||||
if fence_char is not None:
|
||||
match = _FENCE_CLOSE_RE.fullmatch(line)
|
||||
if (match and match.group("fence")[0] == fence_char
|
||||
and len(match.group("fence")) >= fence_len):
|
||||
fence_char, fence_len, fence_indent = None, 0, 0
|
||||
else:
|
||||
dedent = 0
|
||||
while (dedent < fence_indent and dedent < len(line)
|
||||
and line[dedent] == " "):
|
||||
dedent += 1
|
||||
out.append(line[dedent:])
|
||||
continue
|
||||
match = _FENCE_OPEN_RE.fullmatch(line)
|
||||
if match:
|
||||
token = match.group("fence")
|
||||
info = match.group("info")
|
||||
if token[0] != "`" or "`" not in info:
|
||||
fence_char, fence_len = token[0], len(token)
|
||||
fence_indent = match.start("fence")
|
||||
continue
|
||||
out.append(line)
|
||||
if span := _INLINE_CODE_LINE_RE.fullmatch(line):
|
||||
body = span.group("body")
|
||||
if (len(body) >= 2 and body[0] == " " and body[-1] == " "
|
||||
and body.strip(" ")):
|
||||
body = body[1:-1]
|
||||
out.append(body)
|
||||
return out
|
||||
|
||||
|
||||
def _split_by(lines: list[str], heading_re: re.Pattern[str]):
|
||||
sections: dict[str, list[str]] = {}
|
||||
dupes: set[str] = set()
|
||||
@@ -960,14 +1036,25 @@ _MARKER_RE = re.compile(
|
||||
def _one_body(
|
||||
lines: list[str], pattern: re.Pattern[str], label: str, path: str
|
||||
) -> str:
|
||||
bodies = [match.group("body") for line in lines
|
||||
"""Exactly one DISTINCT body for an audit-line grammar.
|
||||
|
||||
Re-statements of an audit line that parse to the SAME stripped body (a
|
||||
plain line plus its fenced copy, or a padded inline-span variant of it)
|
||||
collapse to one value; the collapse is value-level, not byte-level, by
|
||||
design — the candidate list already mixes raw lines with their
|
||||
rendering-normalized forms. Two candidates that parse to different
|
||||
bodies remain a loud abort, so a decoy that disagrees with the
|
||||
operative line can never be silently absorbed (#637).
|
||||
"""
|
||||
bodies = [match.group("body").strip() for line in lines
|
||||
if (match := pattern.fullmatch(line))]
|
||||
if len(bodies) != 1:
|
||||
distinct = list(dict.fromkeys(bodies))
|
||||
if len(distinct) != 1:
|
||||
raise SynthesisError(
|
||||
f"[SYNTHESIS-PARSE: {path}: expected exactly one {label} line, "
|
||||
f"found {len(bodies)}]"
|
||||
f"found {len(distinct)}]"
|
||||
)
|
||||
return bodies[0].strip()
|
||||
return distinct[0]
|
||||
|
||||
|
||||
def _comma_tokens(body: str) -> list[str]:
|
||||
@@ -975,9 +1062,13 @@ def _comma_tokens(body: str) -> list[str]:
|
||||
|
||||
|
||||
def parse_synthesis(path: str, text: str, contract: dict) -> Synthesis:
|
||||
# The four mechanical audit lines read from the decoration-tolerant
|
||||
# candidate source; rationales and the DA marker stay plain-line-only
|
||||
# (see audit_candidate_lines for the declared boundary).
|
||||
lines = strip_fences(text)
|
||||
audit_lines = audit_candidate_lines(text)
|
||||
fired = _comma_tokens(_one_body(
|
||||
lines, _FIRED_LIST_RE, "fired_conditions", path
|
||||
audit_lines, _FIRED_LIST_RE, "fired_conditions", path
|
||||
))
|
||||
condition_ids = {c["condition_id"] for c in contract["failure_conditions"]}
|
||||
if len(fired) != len(set(fired)) or set(fired) - condition_ids:
|
||||
@@ -986,7 +1077,7 @@ def parse_synthesis(path: str, text: str, contract: dict) -> Synthesis:
|
||||
)
|
||||
|
||||
verdict_tokens = _comma_tokens(_one_body(
|
||||
lines, _VERDICTS_RE, "dimension_verdicts", path
|
||||
audit_lines, _VERDICTS_RE, "dimension_verdicts", path
|
||||
))
|
||||
verdicts: dict[str, str] = {}
|
||||
for token in verdict_tokens:
|
||||
@@ -1001,7 +1092,7 @@ def parse_synthesis(path: str, text: str, contract: dict) -> Synthesis:
|
||||
verdicts[match.group("dim")] = match.group("value")
|
||||
|
||||
adjudication_tokens = _comma_tokens(_one_body(
|
||||
lines, _ADJUDICATIONS_RE, "da_critical_adjudications", path
|
||||
audit_lines, _ADJUDICATIONS_RE, "da_critical_adjudications", path
|
||||
))
|
||||
adjudications: dict[str, str] = {}
|
||||
for token in adjudication_tokens:
|
||||
@@ -1016,8 +1107,10 @@ def parse_synthesis(path: str, text: str, contract: dict) -> Synthesis:
|
||||
)
|
||||
adjudications[match.group("id")] = match.group("value")
|
||||
|
||||
decisions = [match.group("action") for line in lines
|
||||
if (match := _DECISION_RE.fullmatch(line))]
|
||||
decisions = list(dict.fromkeys(
|
||||
match.group("action") for line in audit_lines
|
||||
if (match := _DECISION_RE.fullmatch(line))
|
||||
))
|
||||
if len(decisions) != 1 or decisions[0] not in ACTION_ENUM:
|
||||
raise SynthesisError(
|
||||
f"[SYNTHESIS-PARSE: {path}: expected exactly one valid decision]"
|
||||
|
||||
@@ -853,11 +853,46 @@ def check_scoring_seat_anchors(report: panel.ReviewerReport) -> None:
|
||||
)
|
||||
if not is_finding:
|
||||
continue
|
||||
if len(severities) != 1 or severity_declarations != 1:
|
||||
# A finding needs at least one parseable Severity declaration, and
|
||||
# every declaration must parse. When a card declares more than one
|
||||
# ACROSS lines and the chain strictly ESCALATES (Minor < Major <
|
||||
# Critical), the LAST in reading order is operative — the current
|
||||
# model generation self-corrects mid-card with explicit supersession
|
||||
# prose ("See the Severity line below, which supersedes the line
|
||||
# above", #637 ms01_quant r1: Major -> Critical), and Phase 2 permits
|
||||
# no retry for this class, so a strict exactly-one rule turns a
|
||||
# visible, reader-unambiguous correction into a whole-panel abort.
|
||||
# Every other multi-declaration shape keeps the loud abort:
|
||||
# de-escalation could waive the Critical/Major Evidence-Anchor
|
||||
# requirement by appending one weaker line; a non-monotone or
|
||||
# repeated-value chain signals several findings bundled under one W
|
||||
# heading (the one-finding-per-heading accounting feeds the severity
|
||||
# ladder); and two parseable declarations on ONE line are not a
|
||||
# reading-order correction at all. The advisory line below keeps the
|
||||
# full declaration trail in the gate log for adjudication.
|
||||
parseable_per_line = [
|
||||
sum(1 for _ in _SEVERITY_RE.finditer(line)) for line in block
|
||||
]
|
||||
if (not severities or len(severities) != severity_declarations
|
||||
or any(count > 1 for count in parseable_per_line)):
|
||||
raise ConformanceError(
|
||||
f"[FINDING-GRAMMAR: {report.path}: {title} must contain "
|
||||
"exactly one parseable Severity declaration]"
|
||||
)
|
||||
severity_rank = {"Minor": 0, "Major": 1, "Critical": 2}
|
||||
if any(severity_rank[later] <= severity_rank[earlier]
|
||||
for earlier, later in zip(severities, severities[1:])):
|
||||
raise ConformanceError(
|
||||
f"[FINDING-GRAMMAR: {report.path}: {title}: multiple "
|
||||
"Severity declarations must form a strictly escalating "
|
||||
"self-correction chain]"
|
||||
)
|
||||
if len(severities) > 1:
|
||||
print(
|
||||
f"[SEVERITY-SUPERSEDED: {report.path}: {title}: "
|
||||
+ " -> ".join(severities) + "]"
|
||||
)
|
||||
operative_severity = severities[-1]
|
||||
anchor_declarations = sum(
|
||||
len(_ANCHOR_DECL_RE.findall(line)) for line in block
|
||||
)
|
||||
@@ -865,7 +900,7 @@ def check_scoring_seat_anchors(report: panel.ReviewerReport) -> None:
|
||||
match.group("value") for line in block
|
||||
for match in _ANCHOR_RE.finditer(line)
|
||||
]
|
||||
if severities[0] not in {"Critical", "Major"}:
|
||||
if operative_severity not in {"Critical", "Major"}:
|
||||
if anchor_declarations > 1 or len(anchors) != anchor_declarations:
|
||||
raise ConformanceError(
|
||||
f"[FINDING-GRAMMAR: {report.path}: {title} may contain "
|
||||
@@ -877,7 +912,8 @@ def check_scoring_seat_anchors(report: panel.ReviewerReport) -> None:
|
||||
if len(anchors) != 1 or anchor_declarations != 1:
|
||||
raise ConformanceError(
|
||||
f"[ANCHOR-MISSING: {report.path}: {title} "
|
||||
f"{severities[0]} finding needs exactly one Evidence Anchor]"
|
||||
f"{operative_severity} finding needs exactly one "
|
||||
"Evidence Anchor]"
|
||||
)
|
||||
_validate_anchor(anchors[0], f"{report.path}:{title}")
|
||||
|
||||
|
||||
@@ -327,11 +327,15 @@ def test_malformed_fence_closer_keeps_reviewer_report_hidden():
|
||||
cps.parse_report("eic.md", text, FULL)
|
||||
|
||||
|
||||
def test_malformed_fence_closer_keeps_synthesis_hidden():
|
||||
def test_malformed_fence_closer_synthesis_parses_as_fenced_content():
|
||||
"""The fence never closes (`~~~not-a-close` is content, not a closer),
|
||||
so the whole synthesis stays fenced — and fenced audit lines are
|
||||
candidates since #637. Fence-STATE correctness stays pinned by the
|
||||
parse_report twins above, whose grammars remain fence-blind."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = "~~~text\n~~~not-a-close\n" + synthesis + "\n~~~\n"
|
||||
with pytest.raises(cps.SynthesisError, match="fired_conditions"):
|
||||
cps.parse_synthesis("s.md", text, FULL)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("separator", ("\x85", "\u2028", "\u2029"))
|
||||
@@ -347,11 +351,225 @@ def test_unicode_separator_cannot_close_commonmark_fence(separator):
|
||||
|
||||
|
||||
@pytest.mark.parametrize("separator", ("\x85", "\u2028", "\u2029"))
|
||||
def test_unicode_separator_keeps_synthesis_fenced(separator):
|
||||
def test_unicode_separator_synthesis_parses_as_fenced_content(separator):
|
||||
"""A Unicode separator is not a CommonMark line end: it can neither
|
||||
close the fence nor split the line, so the glued `~~~<sep>note` stays
|
||||
one content line. The rest of the synthesis is ordinary fenced content \u2014
|
||||
a candidate since #637 \u2014 and parses."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = "~~~text\n~~~" + separator + synthesis + "\n~~~\n"
|
||||
text = "~~~text\n~~~" + separator + "note\n" + synthesis + "\n~~~\n"
|
||||
parsed = cps.parse_synthesis("fenced-synthesis.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_fenced_audit_block_parses_as_observed_in_637_cohort():
|
||||
"""Every synthesis first attempt of the 2026-08-03 baseline cohort (6/6)
|
||||
emitted the four audit lines inside a plain code fence."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
parsed = cps.parse_synthesis(
|
||||
"s.md", "```\n" + synthesis + "\n```\n", FULL
|
||||
)
|
||||
assert parsed.fired == ["F0"]
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_inline_code_wrapped_audit_lines_parse():
|
||||
"""ms01_quant baseline r2's a2 retry wrapped each audit line whole in an
|
||||
inline code span, defeating the line-anchored grammar (#637)."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = "\n".join(f"`{line}`" for line in synthesis.split("\n"))
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_identical_duplicate_audit_lines_collapse():
|
||||
"""A byte-identical re-statement (plain line plus its fenced copy) is
|
||||
one candidate, not a duplicate abort."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
fired_line = next(line for line in synthesis.split("\n")
|
||||
if line.startswith("fired_conditions:"))
|
||||
text = synthesis + "\n```\n" + fired_line + "\n```\n"
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
def test_conflicting_fenced_audit_line_still_aborts():
|
||||
"""A fenced decoy that disagrees with the operative line stays a loud
|
||||
abort \u2014 tolerance collapses identical values only."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = synthesis + "\n```\nfired_conditions: [F1]\n```\n"
|
||||
with pytest.raises(cps.SynthesisError, match="found 2"):
|
||||
cps.parse_synthesis("s.md", text, FULL)
|
||||
|
||||
|
||||
def test_prose_quoted_checker_diagnostic_is_not_a_candidate():
|
||||
"""The a2 remediation notes of the #637 cohort quote the checker error
|
||||
inside longer prose lines; anchored grammars must not see those."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
note = (
|
||||
"**Checker remediation:** the prior emission failed "
|
||||
"`[SYNTHESIS-PARSE: expected exactly one fired_conditions line, "
|
||||
"found 0]` and the canonical block is re-emitted plainly above."
|
||||
)
|
||||
parsed = cps.parse_synthesis(
|
||||
"s.md", synthesis + "\n" + note + "\n", FULL
|
||||
)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
def test_blockquoted_audit_line_documents_an_accepted_miss():
|
||||
"""Documents an ACCEPTED miss: a blockquoted audit line renders as
|
||||
quotation and is deliberately not a candidate. Change deliberately."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = synthesis.replace("fired_conditions:", "> fired_conditions:", 1)
|
||||
with pytest.raises(cps.SynthesisError, match="fired_conditions"):
|
||||
cps.parse_synthesis("hidden-synthesis.md", text, FULL)
|
||||
cps.parse_synthesis("s.md", text, FULL)
|
||||
|
||||
|
||||
def test_inline_span_inside_fence_documents_an_accepted_miss():
|
||||
"""Documents an ACCEPTED miss: backticks inside a fence are literal
|
||||
rendered content, so a wrapped line there is not unwrapped. Change
|
||||
deliberately."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
fenced = "\n".join(f"`{line}`" for line in synthesis.split("\n"))
|
||||
with pytest.raises(cps.SynthesisError, match="fired_conditions"):
|
||||
cps.parse_synthesis("s.md", "```\n" + fenced + "\n```\n", FULL)
|
||||
|
||||
|
||||
def test_fenced_rationale_and_marker_document_an_accepted_miss():
|
||||
"""Documents an ACCEPTED miss: rationales and the DA marker keep the
|
||||
plain-line source \u2014 their fenced occurrences read as quotation. Change
|
||||
deliberately."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = (
|
||||
synthesis
|
||||
+ "\n```\nC9 rejection rationale: quoted example\n"
|
||||
"[DA-CRITICAL-VS-ACCEPT: 2 validated/unresolved]\n```\n"
|
||||
)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert "C9" not in parsed.rejection_rationales
|
||||
assert parsed.marker_count is None
|
||||
# The forbidden-marker asymmetry is part of the same accepted boundary:
|
||||
# with no active DA-critical adjudications a stated marker is forbidden,
|
||||
# and this fenced one stays invisible rather than aborting — the
|
||||
# decision channel is pinned independently by the layer-2 recompute.
|
||||
assert not any(
|
||||
"marker forbidden" in diag
|
||||
for diag in cps.check_da_terminal_gate(reports(), parsed)
|
||||
)
|
||||
|
||||
|
||||
def test_padded_inline_code_wrapped_audit_lines_parse():
|
||||
"""CommonMark renders `` ` body ` `` with one padding space stripped
|
||||
from each side, so a padded span displays the audit line verbatim and
|
||||
must be a candidate in its DISPLAYED form."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = "\n".join(f"` {line} `" for line in synthesis.split("\n"))
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_padded_span_conflicting_value_still_aborts():
|
||||
"""Rendering normalization must not weaken the distinct-value guard: a
|
||||
padded-span decoy that disagrees with the operative line still aborts."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = synthesis + "\n` fired_conditions: [F1] `\n"
|
||||
with pytest.raises(cps.SynthesisError, match="found 2"):
|
||||
cps.parse_synthesis("s.md", text, FULL)
|
||||
|
||||
|
||||
def test_all_space_span_body_is_not_stripped():
|
||||
"""CommonMark strips padding only when the body is not all spaces; an
|
||||
all-space span stays as-is and is simply a non-candidate."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
parsed = cps.parse_synthesis(
|
||||
"s.md", synthesis + "\n` `\n", FULL
|
||||
)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_indented_fence_content_parses_dedented():
|
||||
"""A fence opened with up to 3 leading spaces renders its content
|
||||
dedented by the opener's indent, so indented fenced audit lines are
|
||||
candidates in their displayed (dedented) form."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
indented = "\n".join(" " + line for line in synthesis.split("\n"))
|
||||
text = " ```\n" + indented + "\n ```\n"
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_crlf_fenced_audit_block_parses():
|
||||
"""CRLF line endings are CommonMark line ends; the fenced audit block
|
||||
parses identically under them."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = ("```\n" + synthesis + "\n```\n").replace("\n", "\r\n")
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
def test_malformed_closer_fence_state_pins_wrapped_decoy_hidden():
|
||||
"""Pins fence STATE through the candidate walker: `~~~not-a-close` does
|
||||
not close the fence, so a wrapped decoy after it is fenced — backticks
|
||||
render literally, the decoy is never unwrapped, and the plain synthesis
|
||||
before the fence parses. A mutation that accepts the malformed closer
|
||||
unwraps the decoy and flips this to a found-2 abort."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = (
|
||||
synthesis
|
||||
+ "\n~~~text\n~~~not-a-close\n`fired_conditions: [F1]`\n~~~\n"
|
||||
)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
def test_short_closer_fence_state_pins_wrapped_decoy_hidden():
|
||||
"""A prospective closer shorter than the opener does not close the
|
||||
fence (CommonMark length rule), so the wrapped decoy after it stays
|
||||
fenced and hidden."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = (
|
||||
synthesis
|
||||
+ "\n````\n```\n`fired_conditions: [F1]`\n````\n"
|
||||
)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
def test_mismatched_closer_fence_state_pins_wrapped_decoy_hidden():
|
||||
"""A backtick closer cannot close a tilde fence; the wrapped decoy
|
||||
after it stays fenced and hidden."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = (
|
||||
synthesis
|
||||
+ "\n~~~\n```\n`fired_conditions: [F1]`\n~~~\n"
|
||||
)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("separator", ("\x85", "\u2028", "\u2029"))
|
||||
def test_unicode_separator_fence_state_pins_wrapped_decoy_hidden(separator):
|
||||
"""A Unicode separator is not a CommonMark line end: `~~~<sep>note`
|
||||
stays one content line and cannot close the fence, so the wrapped
|
||||
decoy after it stays fenced and hidden. A mutation that splits on the
|
||||
separator closes the fence and flips this to a found-2 abort."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
text = (
|
||||
synthesis
|
||||
+ "\n~~~text\n~~~" + separator + "note\n"
|
||||
"`fired_conditions: [F1]`\n~~~\n"
|
||||
)
|
||||
parsed = cps.parse_synthesis("s.md", text, FULL)
|
||||
assert parsed.fired == ["F0"]
|
||||
|
||||
|
||||
def test_unterminated_fence_content_stays_candidate():
|
||||
"""An unterminated fence retains (not grows) its content, and fenced
|
||||
audit lines remain candidates to end-of-input."""
|
||||
synthesis, _ = synthesis_for(reports())
|
||||
parsed = cps.parse_synthesis("s.md", "```\n" + synthesis, FULL)
|
||||
assert parsed.decision == "editorial_decision=accept"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("score", ("warn", "block"))
|
||||
|
||||
@@ -1592,6 +1592,10 @@ second"""
|
||||
|
||||
|
||||
def test_same_line_duplicate_severity_declarations_fail():
|
||||
"""Unchanged by #637: a mid-line second declaration is a declaration
|
||||
(`_SEVERITY_DECL_RE`) whose value cannot parse mid-prose, so the
|
||||
declared-but-unparseable guard still aborts. Only cross-line
|
||||
supersession takes last-wins."""
|
||||
body = (
|
||||
"### W1: hidden critical\n"
|
||||
"**Severity**: Minor and **Severity**: Critical"
|
||||
@@ -1601,6 +1605,126 @@ def test_same_line_duplicate_severity_declarations_fail():
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_same_line_pipe_separated_severity_pair_fails():
|
||||
"""`_SEVERITY_RE` also parses after a table-cell pipe, so both halves
|
||||
of `Minor | Critical` on ONE line are parseable — but two parseable
|
||||
declarations on one line are not a reading-order self-correction and
|
||||
must not enter the supersession path (which could otherwise waive the
|
||||
anchor requirement via `Critical | Minor`)."""
|
||||
for pair in ("Minor | **Severity**: Critical",
|
||||
"Critical | **Severity**: Minor"):
|
||||
body = (
|
||||
"### W1: table smuggle\n"
|
||||
f"**Severity**: {pair}"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(
|
||||
phase.ConformanceError, match="exactly one parseable Severity"
|
||||
):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_cross_line_severity_supersession_takes_last(capsys):
|
||||
"""#637 ms01_quant baseline r1: the domain seat declared Major, then
|
||||
self-corrected to Critical with explicit supersession prose. The card
|
||||
passes with the last value operative and the trail in the gate log."""
|
||||
body = (
|
||||
"### W1: construct mismatch\n"
|
||||
"**Severity**: Major\n"
|
||||
"Correction: recording this as Critical; the Severity line below "
|
||||
"supersedes the line above.\n"
|
||||
"**Severity**: Critical\n"
|
||||
'**Evidence Anchor**: text: "quote" p. 1'
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
assert (
|
||||
"[SEVERITY-SUPERSEDED: p2.md: W1: construct mismatch: "
|
||||
"Major -> Critical]"
|
||||
) in capsys.readouterr().out
|
||||
|
||||
|
||||
def test_unparseable_severity_declaration_still_fails():
|
||||
body = (
|
||||
"### W1: bad value\n"
|
||||
"**Severity**: High"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(
|
||||
phase.ConformanceError, match="exactly one parseable Severity"
|
||||
):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_revisited_severity_value_still_fails():
|
||||
"""A chain that revisits a value (Minor -> Major -> Minor) is not a
|
||||
supersession — a non-escalating chain keeps the anti-bundling abort."""
|
||||
body = (
|
||||
"### W1: bundled pair\n"
|
||||
"**Severity**: Minor\n"
|
||||
"first\n"
|
||||
"**Severity**: Major\n"
|
||||
"second\n"
|
||||
"**Severity**: Minor"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(
|
||||
phase.ConformanceError, match="strictly escalating"
|
||||
):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_deescalating_severity_pair_still_fails():
|
||||
"""Critical -> Minor must abort: last-wins de-escalation would waive
|
||||
the Critical Evidence-Anchor hard gate with one appended line. Only
|
||||
the observed self-correction direction (escalation) is tolerated."""
|
||||
body = (
|
||||
"### W1: fabricated denominators\n"
|
||||
"**Severity**: Critical\n"
|
||||
"The paper invents denominators.\n"
|
||||
"**Severity**: Minor"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(
|
||||
phase.ConformanceError, match="strictly escalating"
|
||||
):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_distinct_severity_bundle_still_fails():
|
||||
"""Three findings bundled under one W heading with distinct descending
|
||||
severities are not a supersession chain and keep the loud abort."""
|
||||
body = (
|
||||
"### W1: bundled triple\n"
|
||||
"**Severity**: Critical\n"
|
||||
"first\n"
|
||||
"**Severity**: Major\n"
|
||||
"second\n"
|
||||
"**Severity**: Minor"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(
|
||||
phase.ConformanceError, match="strictly escalating"
|
||||
):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_escalating_supersession_operative_value_needs_anchor():
|
||||
"""Minor -> Major with NO anchor must abort at ANCHOR-MISSING: the
|
||||
LAST value (Major) is operative. Pins last-wins — if the first value
|
||||
(Minor) were operative the card would take the no-anchor branch and
|
||||
pass."""
|
||||
body = (
|
||||
"### W1: upgraded finding\n"
|
||||
"**Severity**: Minor\n"
|
||||
"on reflection this forecloses the design claim\n"
|
||||
"**Severity**: Major"
|
||||
)
|
||||
report, _ = parse_report("eic", body=body)
|
||||
with pytest.raises(phase.ConformanceError, match="ANCHOR-MISSING"):
|
||||
phase.check_scoring_seat_anchors(report)
|
||||
|
||||
|
||||
def test_noncanonical_heading_and_severity_label_fail_together():
|
||||
body = (
|
||||
"### Weakness 1: fabricated denominators\n"
|
||||
|
||||
Reference in New Issue
Block a user