mirror of
https://github.com/Imbad0202/academic-research-skills.git
synced 2026-09-14 13:51:17 +08:00
* fix: strip CJK outer wrapper marks only when they enclose one balanced unit (#800) The wrapper strip inherited from #431 was positional: it removed the first and last characters whenever they matched as a wrapper pair TYPE, without checking they belonged to the same bracket pair. 《红楼梦》与《金瓶梅》 — two titles joined in one string — normalized to 红楼梦》与《金瓶梅, leaving an orphaned 》 mid-key. Matching correctness was unaffected (both sides mangle identically; no exploitable asymmetry found in the #799 security pass), but the mangled key is a semantic anomaly for any future single-sided consumer. Fix adds _outer_pair_encloses: the outer marks are stripped only when the interior between them is itself balanced under all six wrapper pairs. 《围城》 still strips to 围城, nested balanced interiors still unwrap (《基于「ProEXC」的研究》 → 基于「ProEXC」的研究), while 《红楼梦》与《金瓶梅》, “研究”与“实践”, and “研究与“实践” keep their marks. Both consumers change together — the CJK client re-imports the shared function (#799), pinned behaviorally as well as by identity. The two new discriminating tests are mutation-verified to fail against the pre-fix module. Full suite: 9262 passed, 3 skipped. * fix: scope the CJK interior balance scan to the outer pair's own family (#804 review) Addresses the P1 and three advisories on PR #804. P1: `’` is the closer of `‘` AND the English apostrophe; `”` likewise appears unpaired in mixed typesetting. The family-blind interior scan read the lone `’` in `《Alzheimer’s病中ProEXC表达》` as an unbalanced quote and refused to strip a genuine `《…》` wrap. Verified against main: that pair went from exact=True / ratio 1.0 to exact=False / ratio 0.6818 — below the 0.70 floor, so the DOI-keyed ratio gate and the title-fallback exact gate failed together and a correct DOI could be reported as a mismatch. That is the failure class #798 repaired, so it is not covered by #800's conservative-direction blessing: that blessing is for titles whose outer marks are not one pair, and here they are. Fix scopes the scan to the outer pair's own family — a `《…》` wrap tracks only `《`/`》` and is blind to quote marks. This costs the check nothing it was buying: any mark that can orphan the OUTER pair is by definition of that pair's own family. The whole #800 behavior table survives (`《红楼梦》与《金瓶梅》` still trips on its stray `》`, `“研究”与“实践”` on its stray `”`, nested `《基于「ProEXC」的研究》` still unwraps, `《》` → ""), and the flagged interaction `《「研究』》` → `「研究』` now keeps its mismatched inner quotes as content, which is what distinguishes it from `《研究》`. Regressions added for both apostrophe shapes (`’s`, `’98`). The two lookup maps the family-blind scan needed are now dead and removed. Advisory 1: the depth-0 stray-closer branch is pinned. The first attempt did not discriminate — clamping absorbs one closer, so any title with equal opener/closer counts (every natural case, including `《红楼梦》与《金瓶梅》`) is refused by the trailing depth check anyway. Discriminating requires interior closers to outnumber openers by exactly the clamp count, which no natural title shape produces, so the test uses a documented synthetic asserted against the helper directly. All three branches are now mutation-verified: clamp-instead-of- refuse, drop-the-trailing-check, and family-blind (the P1 regression itself). Advisory 2: the module docstring's "behaviorally equivalent to the implementation this was promoted from" is qualified as historical — true of the #798/#799 promotion, false since #800 deliberately changed this behavior. Advisory 3: the changelog's non-CJK invariance claim is narrowed to what the code supports. The claim holds on the two `has_cjk`-gated paths; the client's `_cn_titles_match` is ungated, so a mark-carrying Han-free title can change verdict there (`《Hamlet》and《Macbeth》` vs its pre-mangled form — verified to flip). The empty-wrapper guarantee is likewise a `_cjk_titles_match` property: `exact_normalized_title("《》", "《》")` is still True through the base branch. Both narrowings are now pinned by tests so the prose cannot drift from the code. Full suite: 9270 passed, 4 skipped, 0 failed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -8,6 +8,8 @@ All notable changes to this project will be documented in this file.
|
||||
|
||||
- **`check_surface_form_parity` names the broken environment instead of blaming the manifest (#801 follow-up).** With the manifest file present but pyyaml unimportable, `_load_manifest` returned `None`, so the lint failed with "manifest … is present but empty / null / non-mapping" — a misdiagnosis pointing the reader at the wrong file (pyyaml is declared in `requirements-dev.txt`, so only a broken local environment can hit this). The missing-import case is now a distinct `_YamlUnavailableError` and the lint fails naming pyyaml and the `pip install -r requirements-dev.txt` remedy; regression test pins that the message names the environment, not the manifest shape. Also de-enumerated the hand-maintained dependency parenthetical in `docs/SETUP.md` / `docs/SETUP.zh-TW.md` Method line ("PyYAML + jsonschema" was already stale against the manifest it mirrors two lines above); both language files move together.
|
||||
|
||||
- **`normalize_cn_title` strips outer wrapper marks only when they enclose the whole title as one balanced unit (#800).** The wrapper strip inherited from #431 (and promoted unchanged by #798/#799) was positional: it removed the first and last characters whenever they matched as a wrapper pair *type*, without checking they belonged to the same bracket pair. `《红楼梦》与《金瓶梅》` — two titles joined in one string — therefore normalized to `红楼梦》与《金瓶梅`, leaving an orphaned `》` mid-key. Matching correctness was never affected (both sides of every comparison pass through the same normalization, and no exploitable asymmetry was found in the #799 security pass), but the mangled key is a semantic anomaly for any future single-sided consumer (display, logging, cache keys, or comparison against an externally-normalized key). Fix adds `_outer_pair_encloses`: the outer marks are stripped only when the interior between them is itself balanced under all six wrapper pairs, so `《围城》` still strips to `围城` and nested balanced interiors (`《基于「ProEXC」的研究》` → `基于「ProEXC」的研究`) still unwrap, while `《红楼梦》与《金瓶梅》` and `“研究”与“实践”` keep their marks. The interior scan is scoped to the outer pair's **own family**: `’` is also the English apostrophe and `”` also appears unpaired, so a family-blind scan read the lone `’` in `《Alzheimer’s病中ProEXC表达》` as an unbalanced quote and refused to strip a genuine `《…》` wrap — dropping a pair that matched before to exact=False and ratio 0.6818, below the 0.70 floor, which fails the DOI-keyed ratio gate and the title-fallback exact gate at once and is the failure class #798 repaired. Scoping costs the check nothing, since any mark that can orphan the outer pair is by definition of that pair's own family. Both consumers change together — the CJK client re-imports the shared function (#799), pinned behaviorally as well as by identity. Scope of the invariance claim, stated precisely: on the two `has_cjk`-gated paths (`exact_normalized_title`'s third branch and `_similarity`'s CJK fold) every verdict and ratio for titles without Han ideographs is unchanged, pinned by the #799 pre-fix oracles. The client's `_cn_titles_match` is **not** `has_cjk`-gated — it calls `normalize_cn_title` directly — so a mark-carrying title with no Han ideograph can change verdict there (`《Hamlet》and《Macbeth》` no longer matches a pre-mangled `Hamlet》and《Macbeth`); that path is DOI-keyed and Chinese-corpus-only in practice, so the narrowing is accepted rather than gated. Likewise the empty-wrapper guard is a property of the CJK branch specifically (`_cjk_titles_match` requires a non-empty key): `exact_normalized_title("《》", "《》")` remains True through the ungated base-normalization branch, as it did before this change.
|
||||
|
||||
- **CJK titles no longer fail the shared exact-title gate in the four index resolvers (#798).** `chinese_literature_client.py` already carried a Chinese-aware `normalize_cn_title` / `has_cjk` (#431 §"Chinese title matching"), but the four index resolvers (Semantic Scholar / OpenAlex / Crossref / arXiv) never saw it — they read the ASCII-centric `_text_similarity` helpers, where `.lower()` folds case but never width (P U+FF30 never reaches P U+0050) and `string.punctuation` contains none of `。`, `《》`, or U+3000. A Chinese paper an index served in a different-but-legitimate typesetting therefore missed on **two** paths: the DOI-keyed cross-check, which gates on the fuzzy ratio *alone* and scored a fullwidth spelling of the identical title at **0.625 — under the 0.70 floor — reporting a correct DOI as `DOI_MISMATCH`**; and the title-fallback search, which requires ratio **and** exact-normalized equality and so fell to `unresolvable`. Both feed the `*_unmatched` contamination signals, so a genuine paper could accumulate *k* across indexes and be rendered `CONTAMINATED-TRIANGULATION-UNMATCHED` — the protocol doc's own "P0, next to the word 'fabricated'". The failure was invisible to the English test corpus for a measurable reason: a Han character is a whole word, so the same six-codepoint corruption is 37.5% of a 16-character Chinese title but 8% of its 68-character English equivalent (measured 0.625 vs 0.912). Fix promotes `has_cjk` / `normalize_cn_title` into `scripts/_text_similarity.py` (behaviorally equivalent — the promotion hoists locals to module constants, precompiles the regex, and rewrites comments; the CJK client now re-imports rather than keeping a private copy, per the #128 anti-drift goal), adds the Chinese-aware form to `exact_normalized_title` as an additive third branch, and folds it into `_similarity` through the existing `max`. Both are gated on **both** sides carrying a Han ideograph, so every non-CJK verdict and every non-CJK *ratio* is provably unchanged — pinned by two oracle tests that re-state the pre-fix `exact_normalized_title` and `_similarity` formulas in full (the latter including the dotted-acronym branch) and assert exact agreement in both directions, so a raised score is caught as loudly as a lowered one. Cross-script and romanized pairs still cannot match (no translation oracle), Simplified/Traditional is still not folded, and an empty normalized key still never matches. Under the Chinese-aware form the ratio also regains discriminative power on the motivating pair: 1.000 for the identical title against an unchanged 0.510 for a genuinely different paper (the base form separated 0.566 from 0.510 — almost nothing). Also measured and pinned: on the motivating pair the pre-fix ratio was actively *anti-correlated* — the identical title scored **0.606 while a genuinely different paper scored 0.645**, so the wrong paper ranked higher. The repair leaves the unrelated pair's score byte-identical (0.6452 before and after; it is not equal under the CJK form, so nothing is folded in) and lifts only the true match, restoring the ordering title ranking depends on. Not in scope, and unchanged: the base ASCII normalization still collapses `ER+`/`ER-` and `p53`/`P53` because it maps ASCII punctuation to whitespace and lowercases, and `exact_normalized_title` ORs that form in; the fuzzy floor also remains a weak separator for CJK generally (that 0.645 near-miss is pre-existing and untouched here). 32 new tests, including 5 integration tests through the real Crossref client covering both broken paths, each verified to fail against the pre-fix module, plus a monkeypatched detonator pinning that the Chinese DOI-path matcher never consults the fuzzy ratio in either direction.
|
||||
|
||||
- **Autolink round-trip test no longer rides on an undeclared, version-sensitive dependency (#801).** `test_gfm_bare_urls_emails_and_schemes_cannot_autolink` soft-imported markdown-it-py — not listed in `requirements-dev.txt` — and silently returned when it was absent, so its no-`link_open` tail had never actually run in CI, while an ambient markdown-it-py 2.x (whose linkify pass still tokenizes the escaped forms as links; reproduced under 2.2.0 + linkify-it-py 2.0.3, reported by @akshath-raj in #799) failed the test on clean `main`. Verified dividing line: 2.2.0 fails, 3.0.0 and 4.0.0 pass with linkify-it-py 2.0.3 held constant. The tail is split into its own test (`test_escaped_markdown_yields_no_linkify_tokens_on_round_trip`) gated by `pytest.importorskip("markdown_it", minversion="3.0.0")` + `pytest.importorskip("linkify_it", minversion="2.0.3")` — ambient-old environments now skip visibly instead of failing or passing vacuously — and `requirements-dev.txt` declares `markdown-it-py>=3.0` + `linkify-it-py>=2.0.3` with a reverse pointer at the consuming test, so CI exercises the round-trip for the first time. The same declaration also newly activates the identical soft-import tail in `test_renderer_neutralizes_markdown_active_inventory_path` (`scripts/test_build_submission_packet_manifest.py`), which moves to the same `importorskip` idiom (no version floor: it parses default CommonMark without linkify) and gains the inline-children flatten its top-level-only token scan was missing — `link_open` / `image` / `html_inline` nest under `inline` tokens, so the pre-existing assertion could only ever catch `html_block`; its strengthened assertions verified under markdown-it-py 2.2.0, 3.0.0, and 4.0.0. The triplicated hostile-row construction in `test_evidence_rows.py` is consolidated into one `_hostile_row` helper. Renderer behavior and every renderer-side assertion are unchanged.
|
||||
|
||||
@@ -80,6 +80,48 @@ _CN_WRAPPERS = {
|
||||
("《", "》"), ("「", "」"), ("『", "』"), ("【", "】"),
|
||||
("“", "”"), ("‘", "’"),
|
||||
}
|
||||
|
||||
|
||||
def _outer_pair_encloses(text: str) -> bool:
|
||||
"""#800: True iff `text` starts and ends with marks that form ONE matching
|
||||
wrapper pair AND the interior is balanced *within that pair's own family* —
|
||||
i.e. the outer pair encloses the whole title as a single unit.
|
||||
|
||||
A first/last-character pair check alone is not enough: in
|
||||
`《红楼梦》与《金瓶梅》` the leading `《` and trailing `》` match as
|
||||
*character types* but belong to two different bracket pairs, and stripping
|
||||
them positionally leaves an orphaned `》` mid-string. Scanning the interior
|
||||
for an unbalanced `》` catches exactly that case, so such titles keep their
|
||||
marks. An empty interior counts as balanced, preserving the `《》` → ""
|
||||
normalization.
|
||||
|
||||
The scan is deliberately scoped to the outer pair's OWN family (#804 review
|
||||
P1). A family-blind scan is unsound because two of these marks are not
|
||||
exclusively wrappers: `’` is also the English apostrophe and `”` also
|
||||
appears unpaired in mixed typesetting. Reading the lone `’` in
|
||||
`《Alzheimer’s病中ProEXC表达》` as an unbalanced quote refused to strip
|
||||
a genuine `《…》` wrap, dropping a pair that matched on main to exact=False
|
||||
and ratio 0.6818 — below the 0.70 floor, so both resolver paths failed at
|
||||
once and a correct DOI could be reported as a mismatch. That is the failure
|
||||
class #798 repaired, so the apostrophe must be invisible to a `《》` scan.
|
||||
|
||||
Scoping costs nothing the check was buying: every mark that can orphan the
|
||||
OUTER pair is by definition a mark of that pair's own family. Interior marks
|
||||
of other families are content and survive the strip — `《「研究』》` →
|
||||
`「研究』` keeps the mismatched inner quotes, which is right, since they are
|
||||
what distinguishes it from `《研究》`."""
|
||||
if len(text) < 2 or (text[0], text[-1]) not in _CN_WRAPPERS:
|
||||
return False
|
||||
opener, closer = text[0], text[-1]
|
||||
depth = 0
|
||||
for ch in text[1:-1]:
|
||||
if ch == opener:
|
||||
depth += 1
|
||||
elif ch == closer:
|
||||
if depth == 0: # closes the OUTER opener: not one unit
|
||||
return False
|
||||
depth -= 1
|
||||
return depth == 0
|
||||
# The fullwidth-ASCII fold maps `.` to `.` first. `?`/`?` are kept: they can
|
||||
# distinguish an interrogative title from an otherwise identical one.
|
||||
_CN_TERMINAL_MARKS = "。."
|
||||
@@ -112,14 +154,21 @@ def normalize_cn_title(title: str | None) -> str:
|
||||
proper nouns, and a wrong fold would manufacture a false match. The pair is
|
||||
surfaced to the human instead.
|
||||
|
||||
Behaviorally equivalent to the implementation this was promoted from in
|
||||
`chinese_literature_client.py`, which re-imports it from here rather than
|
||||
keeping a second copy (#128 anti-drift). Not byte-identical: the promotion
|
||||
hoists the wrapper/terminal-mark sets to module constants, precompiles the
|
||||
Han-adjacent-space regex, and rewrites the comments. Equivalence of
|
||||
*behavior* is what the tests pin, on both the CJK path and — through the
|
||||
pre-fix oracles in `test_text_similarity.py` — every non-CJK verdict and
|
||||
ratio.
|
||||
Outer wrappers are stripped only when they enclose the whole title as one
|
||||
balanced unit (#800): `《围城》` loses its marks, but
|
||||
`《红楼梦》与《金瓶梅》` keeps them, because its first and last marks belong
|
||||
to two different bracket pairs and stripping them positionally would leave
|
||||
an orphaned `》` mid-string.
|
||||
|
||||
Promoted here from `chinese_literature_client.py`, which re-imports it
|
||||
rather than keeping a second copy (#128 anti-drift). The promotion itself
|
||||
(#798/#799) was behaviorally equivalent — it only hoisted the wrapper and
|
||||
terminal-mark sets to module constants, precompiled the Han-adjacent-space
|
||||
regex, and rewrote the comments. That equivalence is now **historical**:
|
||||
#800 deliberately changed the wrapper-strip behavior described above, so
|
||||
this function no longer matches the pre-#800 implementation. What the tests
|
||||
pin today is the #800 behavior on the CJK path plus — through the pre-fix
|
||||
oracles in `test_text_similarity.py` — every non-CJK verdict and ratio.
|
||||
"""
|
||||
# Fold only the fullwidth ASCII compatibility block that was observed in the
|
||||
# motivating metadata. Whole-string NFKC/casefold is too broad for an exact
|
||||
@@ -137,7 +186,10 @@ def normalize_cn_title(title: str | None) -> str:
|
||||
while text:
|
||||
previous = text
|
||||
text = text.rstrip(_CN_TERMINAL_MARKS).strip()
|
||||
if len(text) >= 2 and (text[0], text[-1]) in _CN_WRAPPERS:
|
||||
# #800: strip the outer wrapper only when it genuinely encloses the
|
||||
# whole title as one balanced unit — not merely when the first and last
|
||||
# characters happen to match as pair types.
|
||||
if _outer_pair_encloses(text):
|
||||
text = text[1:-1].strip()
|
||||
if text == previous:
|
||||
break
|
||||
|
||||
@@ -199,6 +199,13 @@ _NON_CJK_PAIRS = [
|
||||
(" ", ""),
|
||||
("。", "《》"),
|
||||
("ProEXC assay", "ProEXC assay"),
|
||||
# #800: wrapper marks present but no Han ideograph — the pairedness check
|
||||
# must not leak through the has_cjk gate and accidentally strip wrappers
|
||||
# on non-CJK titles. (If normalize_cn_title were called on these, the
|
||||
# outer marks would strip and the pairs would match — exactly the
|
||||
# regression this entry is pinned against.)
|
||||
("《Attention Is All You Need》", "Attention Is All You Need"),
|
||||
("《A Study of Foo》", "A Study of Foo"),
|
||||
]
|
||||
|
||||
|
||||
@@ -288,6 +295,148 @@ class NormalizeCnTitleTest(unittest.TestCase):
|
||||
with self.subTest(wrapped=wrapped):
|
||||
self.assertEqual(ts.normalize_cn_title(wrapped), "研究")
|
||||
|
||||
def test_strips_outer_quotation_wrappers(self) -> None:
|
||||
for wrapped in ("“围城”", "‘围城’"):
|
||||
with self.subTest(wrapped=wrapped):
|
||||
self.assertEqual(ts.normalize_cn_title(wrapped), "围城")
|
||||
|
||||
def test_keeps_marks_when_outer_marks_belong_to_two_different_pairs(self) -> None:
|
||||
"""#800: the first/last marks of `《红楼梦》与《金瓶梅》` match as pair
|
||||
TYPES (`《`/`》`) but belong to two different brackets — one opening
|
||||
each title. Positional stripping left an orphaned `》` mid-string; the
|
||||
pairedness check keeps the title intact instead."""
|
||||
for title in ("《红楼梦》与《金瓶梅》", "“研究”与“实践”"):
|
||||
with self.subTest(title=title):
|
||||
self.assertEqual(ts.normalize_cn_title(title), title)
|
||||
|
||||
def test_nested_interior_wrapper_survives_outer_strip(self) -> None:
|
||||
"""A genuine outer pair enclosing a balanced interior still strips, and
|
||||
the balanced interior marks survive as content."""
|
||||
self.assertEqual(
|
||||
ts.normalize_cn_title("《基于「ProEXC」的研究》"),
|
||||
"基于「ProEXC」的研究",
|
||||
)
|
||||
|
||||
def test_unclosed_opener_in_interior_keeps_marks(self) -> None:
|
||||
"""An opener inside the interior that never closes proves the outer
|
||||
marks are not one unit — a distinct failure mode from the stray-closer
|
||||
case above (stack non-empty at the end vs. pop from empty), and also
|
||||
verified to fail against the pre-fix positional strip."""
|
||||
self.assertEqual(ts.normalize_cn_title("“研究与“实践”"), "“研究与“实践”")
|
||||
|
||||
def test_apostrophe_in_embedded_english_does_not_veto_the_outer_strip(self) -> None:
|
||||
"""#804 review P1: `’` is the closer of `‘`, but it is ALSO the
|
||||
apostrophe in embedded English — the same codepoint in a non-wrapper
|
||||
role. A family-blind interior scan read the lone `’` in
|
||||
`《Alzheimer’s病中…》` as an unbalanced quote and refused to strip a
|
||||
genuine `《…》` wrap, so a title that matched on main stopped matching:
|
||||
exact False and ratio 0.6818, below the 0.70 floor. That takes out the
|
||||
DOI-keyed ratio gate and the title-fallback exact gate at once, which
|
||||
is the exact failure class #798 repaired — not something #800's
|
||||
conservative-direction blessing covers, since here the outer pair DOES
|
||||
enclose the title.
|
||||
|
||||
Scoping the balance scan to the outer pair's own family fixes it: a
|
||||
`《…》` wrap tracks only `《`/`》` and is blind to quote marks."""
|
||||
for wrapped, bare in (
|
||||
("《Alzheimer’s病中ProEXC表达》", "Alzheimer’s病中ProEXC表达"),
|
||||
("《’98年香港回归研究》", "’98年香港回归研究"),
|
||||
):
|
||||
with self.subTest(wrapped=wrapped):
|
||||
self.assertEqual(ts.normalize_cn_title(wrapped), ts.normalize_cn_title(bare))
|
||||
self.assertTrue(ts.exact_normalized_title(wrapped, bare))
|
||||
self.assertEqual(ts._similarity(wrapped, bare), 1.0)
|
||||
|
||||
def test_interior_scan_ignores_other_families_but_not_its_own(self) -> None:
|
||||
"""The family scoping is a narrowing, not a blanket weakening: a stray
|
||||
closer of the OUTER pair's own family still blocks the strip, while a
|
||||
stray mark of any other family is content.
|
||||
|
||||
`《「研究』》` is the interaction the review flagged: the interior
|
||||
`「研究』` is mismatched, but not in the `《》` family, so the outer pair
|
||||
still encloses one unit and strips. The mismatched interior marks
|
||||
survive as content — which is correct, since they are exactly what
|
||||
distinguishes this title from `《研究》`."""
|
||||
self.assertEqual(ts.normalize_cn_title("《「研究』》"), "「研究』")
|
||||
# Same-family stray closer still blocks, at any interior depth.
|
||||
self.assertEqual(ts.normalize_cn_title("《红楼梦》与《金瓶梅》"), "《红楼梦》与《金瓶梅》")
|
||||
|
||||
def test_unclosed_interior_opener_blocks_the_strip(self) -> None:
|
||||
"""The trailing `depth == 0` check, pinned on its own: `《基于《研究」的分析》`
|
||||
has an interior `《` that nothing closes, so the outer `》` is closing the
|
||||
INNER opener and the outer pair is not one unit."""
|
||||
self.assertEqual(
|
||||
ts.normalize_cn_title("《基于《研究」的分析》"), "《基于《研究」的分析》"
|
||||
)
|
||||
|
||||
def test_stray_interior_closer_blocks_the_strip_independently(self) -> None:
|
||||
"""#804 review advisory: the depth-0 stray-closer branch had no
|
||||
discriminating test — a mutant clamping it (`depth = max(0, depth - 1)`)
|
||||
instead of refusing passed the whole suite, including the two natural
|
||||
titles above.
|
||||
|
||||
The reason is arithmetic, and worth recording so this test is not
|
||||
"simplified" back later: clamping absorbs one closer, so a title whose
|
||||
opener/closer counts are equal — every natural case here, including
|
||||
`《红楼梦》与《金瓶梅》` — ends at depth 1 under the mutant and is refused
|
||||
anyway, by the OTHER branch. Discriminating requires interior closers to
|
||||
outnumber openers by exactly the clamp count, which no natural title
|
||||
shape produces. Hence a deliberately synthetic string, asserted against
|
||||
the helper directly: `《》研究《实践》》` opens at depth 0 with a `》`,
|
||||
which is precisely what the branch exists to refuse."""
|
||||
self.assertFalse(ts._outer_pair_encloses("《》研究《实践》》"))
|
||||
# The natural stray-closer title stays refused too — belt and braces,
|
||||
# since this is the branch that keeps `《红楼梦》与《金瓶梅》` intact.
|
||||
self.assertFalse(ts._outer_pair_encloses("《红楼梦》与《金瓶梅》"))
|
||||
|
||||
def test_mangled_key_still_matches_its_identically_mangled_counterpart(self) -> None:
|
||||
"""Matching correctness was never broken by the positional strip (both
|
||||
sides mangled identically), and the pairedness fix must not change that
|
||||
either way: equal normalizations still match on both paths."""
|
||||
a = "《红楼梦》与《金瓶梅》"
|
||||
self.assertTrue(ts.exact_normalized_title(a, a))
|
||||
# The paired form now keeps its marks while an old-style bare variant
|
||||
# differs — conservative non-matching is acceptable per #800.
|
||||
self.assertFalse(ts.exact_normalized_title(a, "红楼梦》与《金瓶梅"))
|
||||
|
||||
def test_ungated_client_path_narrowing_is_explicit(self) -> None:
|
||||
"""#804 review advisory: the invariance claim must not be stated more
|
||||
broadly than the code supports.
|
||||
|
||||
`exact_normalized_title` and `_similarity` reach the CJK form only
|
||||
behind `has_cjk`, so a Han-free title is untouched there. The client's
|
||||
`_cn_titles_match` has NO such gate — it calls `normalize_cn_title`
|
||||
directly — so a mark-carrying Han-free title can change verdict on that
|
||||
path. Pinned here as a known, accepted narrowing rather than left to be
|
||||
rediscovered: the path is DOI-keyed and Chinese-corpus-only in
|
||||
practice."""
|
||||
from chinese_literature_client import _cn_titles_match
|
||||
|
||||
wrapped, mangled = "《Hamlet》and《Macbeth》", "Hamlet》and《Macbeth"
|
||||
self.assertFalse(_cn_titles_match(wrapped, mangled))
|
||||
# The gated shared helpers never saw this pair as CJK, before or after.
|
||||
self.assertFalse(ts.exact_normalized_title(wrapped, mangled))
|
||||
|
||||
def test_empty_wrapper_guard_is_a_cjk_branch_property(self) -> None:
|
||||
"""The "`《》` never matches" guarantee belongs to `_cjk_titles_match`,
|
||||
which requires a non-empty normalized key. `exact_normalized_title`
|
||||
still returns True for `《》` vs itself through the ungated BASE
|
||||
normalization branch — unchanged by #800, and pinned so the changelog's
|
||||
narrowed wording stays honest."""
|
||||
self.assertEqual(ts.normalize_cn_title("《》"), "")
|
||||
self.assertFalse(ts._cjk_titles_match("《》", "「」"))
|
||||
self.assertTrue(ts.exact_normalized_title("《》", "《》"))
|
||||
|
||||
def test_pairedness_behavior_is_shared_by_the_cjk_client(self) -> None:
|
||||
"""#800 acceptance: both consumers change together. The client
|
||||
re-imports the shared function (#799), so identity covers it — pinned
|
||||
behaviorally here so a future private copy cannot hide the drift."""
|
||||
import chinese_literature_client as cn
|
||||
|
||||
title = "《红楼梦》与《金瓶梅》"
|
||||
self.assertEqual(cn.normalize_cn_title(title), ts.normalize_cn_title(title))
|
||||
self.assertEqual(cn.normalize_cn_title(title), title)
|
||||
|
||||
def test_removes_whitespace_touching_han(self) -> None:
|
||||
self.assertEqual(ts.normalize_cn_title("基于 ProEXC 的研究"), "基于ProEXC的研究")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user