diff --git a/CHANGELOG.md b/CHANGELOG.md index 9558ce31..58ba947b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,7 +8,7 @@ All notable changes to this project will be documented in this file. - **`claude plugin eval` suite for the `academic-paper` revision-coach flow (`plugin-evals/`; toward #851, not closed).** Seven cases (five that should fire, two that should not) with twenty graders, run as a with/without-plugin ablation so the headline is the uplift Δ, not a pass rate. The author-defined quality axis is no unauthorised rewriting: the response must not draft manuscript prose, change anything no reviewer asked for, or assert results or changes that have not happened; secondary axes are no comment dropped, push-back allowed on a factually wrong reviewer point, and a committee letter yields a tracker with no peer-review grading. Inputs are synthetic and cover four shapes (mixed-format journal comments, an unnumbered decision-letter email, OpenReview-style scores with a "should we push back" question, and a formal ethics-committee letter), in zh-TW and English. Calibration against the 2026-09-12 pilots is recorded in `plugin-evals/README.md` together with the run command, the `timeout_seconds` / `max_turns` ceilings, the observed pilot cost, and three caveats — including that the ICLR case did not invoke the skill in either pilot, which is filed as #851 and uses that case's display-only `skill-fired` grader as its acceptance check. `plugin-evals/results/` is gitignored. This adds a regression guard only; it claims no measured uplift figure (a single-run pilot is not a measurement). -- **es-ES (Spanish, Spain) README (#855; toward #850, not closed).** `README.es-ES.md` is a full translation of `README.md` at v3.21.2 by a native speaker, structure kept byte-comparable with the English file (same heading, table, and fence order; code blocks, identifiers, and links untouched), contributed by @didacrios. The five existing READMEs gain the `[Español](README.es-ES.md)` nav link. Drift is policed the same way as the Korean README: `check_readme_es_sections()` in `scripts/check_spec_consistency.py` pins the version badge, release-note headings, mode headings, and forbidden retired phrases, and `scripts/check_reviewer_role_label.py` gains the es-ES REQUIRED entry. Translation only; no skill, mode, or trigger changes. The companion es-ES trigger-keyword change is #856 and is recorded here when it merges. +- **es-ES (Spanish, Spain) README and trigger phrases (#855, #856; toward #850, not closed).** `README.es-ES.md` is a full translation of `README.md` at v3.21.2 by a native speaker, structure kept byte-comparable with the English file (same heading, table, and fence order; code blocks, identifiers, and links untouched), contributed by @didacrios. The five existing READMEs gain the `[Español](README.es-ES.md)` nav link. Drift is policed the same way as the Korean README: `check_readme_es_sections()` in `scripts/check_spec_consistency.py` pins the version badge, release-note headings, mode headings, and forbidden retired phrases, and `scripts/check_reviewer_role_label.py` gains the es-ES REQUIRED entry. Translation only; no skill, mode, or trigger changes. The companion trigger change (#856, merged as a366e39, also by @didacrios) follows the #509 Korean precedent: each of the four `SKILL.md` files gains one body `**Español**:` line of intent-specific compounds (no broad standalone words) and a conservative Spanish subset in its frontmatter `description`, which is the surface the router reads; `revisar artículo` stays exclusive to the reviewer and `enmendar mi artículo` marks revision intent, pinned by two es-ES boundary fixtures under `tests/fixtures/issue_133_routing/` (11 and 12). `academic-pipeline/SKILL.md` is content-locked, so its `CONTENT_LOCKS` hash in `scripts/check_pipeline_boundary_semantics.py` is re-pinned in the same change. All four descriptions stay under the Agent Skills 1,024-character ceiling counted in code points of the parsed value (deep-research 1,021, academic-paper 867, academic-paper-reviewer 878, academic-pipeline 783); the lint that enforces that ceiling is filed as #864. Evidence: the `05-apa-es-locale` citation-check eval case fires the skill 2 of 2 on the branch against 0 of 2 on the previous `main`; a five-phrase Claude Code routing smoke (sonnet, two runs each) fired 2/2 on the lit-review, full-pipeline, and Socratic phrases, 1/2 on `revisa este artículo` (0/2 on the previous `main`), and 0/2 on `redacta un artículo` (also 0/2 on the previous `main`; the model writes the article in one turn without calling any skill, the bypass tracked in #851 and #857). Two small follow-ups (three broad terms left in the deep-research body line; fixture 11 asking for reviewer comments it does not attach) are #865. Activation layer only; no mode, IRON RULE, schema, or workflow text changes. - **`claude plugin eval` suite for the `academic-paper` citation-check flow (`plugin-evals-citation-check/`; surfaces #857 and #858, neither closed).** Eight cases (six that should fire, two that should not), run as a with/without-plugin ablation so the headline is the uplift Δ, not a pass rate. The author-defined quality axis is the four citation failures that matter: a reference the user has no source for, the right paper with the wrong authors, a hedged or minor finding cited as an established result, and a retracted or concern-flagged paper cited as live evidence. Every fire case ships a complete synthetic source pack so all four are detectable offline; the sandbox has no network, so "no source" is graded as flagging the entry as unverifiable, never as a real lookup. Secondary axes: mechanical errors, no false positives on clean entries, no claim of online verification, no rewriting of the manuscript. Cases cover APA 7 (en, zh-TW mixed, es), IEEE, Vancouver with the style unnamed, and Chicago notes-bibliography; the negatives are an APA-to-IEEE conversion request and a Python unused-import question. Cases pin `model: sonnet` and the run command sets `--judge-model opus` so the judge is never the agent model. Calibration against the 2026-09-13 pilots is recorded in `plugin-evals-citation-check/README.md` with the run command, the `timeout_seconds` / `max_turns` ceilings, the observed pilot cost, and the caveats: the with-plugin arm cannot load the mode's own prompt in the eval sandbox because the command stub references plugin files by relative path (#857), so Δ ≈ 0 is the honest current reading; plain-language prompts fired the skill in 3 of 6 cases (#858, with the Spanish case as one data point for #850). `plugin-evals-citation-check/results/` is gitignored. This adds a regression guard only; it claims no measured uplift figure.