Files
imbad0202__academic-researc…/scripts
Akshath Rajkumar 5debcd2efb fix: strip CJK outer wrapper marks only when they enclose one balanced unit (#800) (#804)
* fix: strip CJK outer wrapper marks only when they enclose one balanced unit (#800)

The wrapper strip inherited from #431 was positional: it removed the first
and last characters whenever they matched as a wrapper pair TYPE, without
checking they belonged to the same bracket pair. 《红楼梦》与《金瓶梅》 —
two titles joined in one string — normalized to 红楼梦》与《金瓶梅, leaving
an orphaned 》 mid-key.

Matching correctness was unaffected (both sides mangle identically; no
exploitable asymmetry found in the #799 security pass), but the mangled key
is a semantic anomaly for any future single-sided consumer.

Fix adds _outer_pair_encloses: the outer marks are stripped only when the
interior between them is itself balanced under all six wrapper pairs.
《围城》 still strips to 围城, nested balanced interiors still unwrap
(《基于「ProEXC」的研究》 → 基于「ProEXC」的研究), while 《红楼梦》与《金瓶梅》,
“研究”与“实践”, and “研究与“实践” keep their marks.

Both consumers change together — the CJK client re-imports the shared
function (#799), pinned behaviorally as well as by identity. The two new
discriminating tests are mutation-verified to fail against the pre-fix
module. Full suite: 9262 passed, 3 skipped.

* fix: scope the CJK interior balance scan to the outer pair's own family (#804 review)

Addresses the P1 and three advisories on PR #804.

P1: `’` is the closer of `‘` AND the English apostrophe; `”` likewise appears
unpaired in mixed typesetting. The family-blind interior scan read the lone `’`
in `《Alzheimer’s病中ProEXC表达》` as an unbalanced quote and refused to
strip a genuine `《…》` wrap. Verified against main: that pair went from
exact=True / ratio 1.0 to exact=False / ratio 0.6818 — below the 0.70 floor, so
the DOI-keyed ratio gate and the title-fallback exact gate failed together and a
correct DOI could be reported as a mismatch. That is the failure class #798
repaired, so it is not covered by #800's conservative-direction blessing: that
blessing is for titles whose outer marks are not one pair, and here they are.

Fix scopes the scan to the outer pair's own family — a `《…》` wrap tracks only
`《`/`》` and is blind to quote marks. This costs the check nothing it was
buying: any mark that can orphan the OUTER pair is by definition of that pair's
own family. The whole #800 behavior table survives (`《红楼梦》与《金瓶梅》`
still trips on its stray `》`, `“研究”与“实践”` on its stray `”`, nested
`《基于「ProEXC」的研究》` still unwraps, `《》` → ""), and the flagged
interaction `《「研究』》` → `「研究』` now keeps its mismatched inner quotes as
content, which is what distinguishes it from `《研究》`. Regressions added for
both apostrophe shapes (`’s`, `’98`).

The two lookup maps the family-blind scan needed are now dead and removed.

Advisory 1: the depth-0 stray-closer branch is pinned. The first attempt did
not discriminate — clamping absorbs one closer, so any title with equal
opener/closer counts (every natural case, including `《红楼梦》与《金瓶梅》`)
is refused by the trailing depth check anyway. Discriminating requires interior
closers to outnumber openers by exactly the clamp count, which no natural title
shape produces, so the test uses a documented synthetic asserted against the
helper directly. All three branches are now mutation-verified: clamp-instead-of-
refuse, drop-the-trailing-check, and family-blind (the P1 regression itself).

Advisory 2: the module docstring's "behaviorally equivalent to the
implementation this was promoted from" is qualified as historical — true of the
#798/#799 promotion, false since #800 deliberately changed this behavior.

Advisory 3: the changelog's non-CJK invariance claim is narrowed to what the
code supports. The claim holds on the two `has_cjk`-gated paths; the client's
`_cn_titles_match` is ungated, so a mark-carrying Han-free title can change
verdict there (`《Hamlet》and《Macbeth》` vs its pre-mangled form — verified to
flip). The empty-wrapper guarantee is likewise a `_cjk_titles_match` property:
`exact_normalized_title("《》", "《》")` is still True through the base branch.
Both narrowings are now pinned by tests so the prose cannot drift from the code.

Full suite: 9270 passed, 4 skipped, 0 failed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 17:55:24 +08:00
..