mirror of
https://github.com/daymade/claude-code-skills.git
synced 2026-09-14 16:15:21 +08:00
feat(claude-md-progressive-disclosurer): hotspot-first profiling + scripted section sink + compression-fidelity audit (Case 19) (#277)
* feat(claude-md-progressive-disclosurer): Step 2.0 热点测量 + 消费方截断哨兵 + 整节下沉脚本化 + 压缩保真审计（案例 19） 真实 168KB 全局 CLAUDE.md 优化 session 反哺：热点排序缺位被用户当场打断、 Codex project_doc_max_bytes 静默截断 41% 数周、压缩重述丢限定词（/forks） 被独立审计抓出。新增 scripts/profile_claude_md.py（分节字节表=工作顺序、 fence 假节守卫、CJK token 比率）与 scripts/sink_sections.py（verbatim 下沉 事务：提取→追加→自底向上替换→整串验证→回滚，拒写 symlink 目标）。 载体表补官方 ~/.claude/rules/ paths 机制与 <200 行目标。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvNKng24noMLREHpRmpTWN * fix(claude-md-progressive-disclosurer): 独立审阅 round-1 全部落实——symlink guard 补目录级盲区(HIGH) + 6 LOW HIGH-1: 文件级 islink 被父目录 symlink 静默穿透（审阅实测打穿），改 realpath≠abspath 判定 + --allow-symlinked-target 显式放行；四条路径双向标定 （dir-symlink ABORT / flag 放行 / EOF 末节 / 重名标题 abort）。LOW: EOF 末节 支持、重名唯一性检查、snippet 保留标题行入文档、父节含子节提示、案例 19 size 数字加"非通过条件"注记、164/168KB 时点澄清、分隔线。 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NvNKng24noMLREHpRmpTWN --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
daymade
@@ -103,7 +103,7 @@
|
||||
"description": "Claude Code operations suite that bundles archive-aware local conversation discovery across Claude Code and Codex, internal-timestamp full-event session search and recovery, interrupted-work continuation, plugin/skill troubleshooting, CLAUDE.md progressive disclosure optimization, statusline configuration, exported .txt repair, full claude.ai conversation extraction with tool-call rendering and file download, plugin marketplace development, writing/testing/debugging Claude Code hooks, multi-provider profile isolation for running Kimi/MiniMax/GLM/DeepSeek/StepFun/Anthropic in separate windows, local source sync for Claude/Codex skill installs, personal-memory migration into tool-agnostic AGENTS.md reference docs, and terminal-output-to-PNG rendering for visual CLI verification under one shared namespace. Install once to get the full Claude Code power-user toolkit.",
|
||||
"source": "./daymade-claude-code",
|
||||
"strict": false,
|
||||
"version": "1.41.0",
|
||||
"version": "1.42.0",
|
||||
"category": "suite",
|
||||
"keywords": [
|
||||
"suite",
|
||||
|
||||
@@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- **claude-md-progressive-disclosurer** (`daymade-claude-code` v1.42.0): fold a real 168KB-global-CLAUDE.md optimization session back into the methodology — the failure modes were caught live, each by the user or an independent audit, and none was encoded in the skill yet. (1) New **Step 2.0 hotspot profiling**: measure before proposing — whole-context share (via `/context`), a per-section byte table whose descending order IS the work order, line-length distribution (lines >1KB are the "rule + war story fused into one bullet" signature: 4.4% of lines carried 35.6% of bytes), and consumer truncation caps. Added after the session's executor proposed a ~3% extensions cleanup while the 70k-token hotspot file sat waiting for a go — user's words: "你没有先去管热点，而是先找了一堆很小很小的东西". Encodes the two instrument pitfalls hit the same day: fence-unaware heading regexes fabricate phantom sections (a fake 45.9KB section distorted the first ranking), and chars/4 underestimates CJK-dense files by >2x (measured ~0.42 tokens/byte). (2) **Consumer-cap check + sentinel**: the same file read through an `~/.codex/AGENTS.md` symlink had outgrown its configured 96KiB `project_doc_max_bytes` — 41% of the body silently invisible to Codex for weeks, zero errors anywhere; the fix is raise-cap + shrink-file, plus a SessionStart size-vs-cap sentinel so the next overrun reports itself. (3) Step 3 gains the **scripted whole-section sink procedure** with two bundled stdlib-only scripts (`scripts/profile_claude_md.py`, `scripts/sink_sections.py`): fence-aware extraction → verbatim append under dated provenance headers → bottom-up splice → whole-string substring verification (grep ORs per line and passes lossy moves) → automatic rollback; refuses symlinked targets, which would silently edit another repo (prior art searched and cited in-script: mdsplit splits whole files, has no transactional sink contract). Verified live: 10 sections / 119KB sunk, 10/10 whole-string checks, zero information loss confirmed by independent audit. (4) Step 5b gains the **compressed-restatement fidelity audit**: qualifier-level word diff — the audit caught "public + 0 stars/forks" compressed to "0 stars", six characters that halved a push gate's condition — plus expected-hunks-only diffing (every non-equal hunk must map to a declared change). Carrier table updated with the official `~/.claude/rules/` + `paths:` frontmatter mechanism (boundary stated: triggers on file reads, not Bash moments), the official <200-line target, `/doctor`'s trim check (v2.1.206+), and zero-cost HTML-comment maintainer notes. Full war story: references Case 19.
|
||||
- **claude-code-history-files-finder** (`daymade-claude-code` v1.40.0): new **Core Operation 6 — Triage Session Endings** (`analyze_sessions.py triage`), for "which sessions did a reboot/crash cut off" and "which older sessions are still waiting on a reply, not actually done" — a capability two real tasks in one conversation needed and the existing `list`/`search`/`stats` didn't support, so both were done by hand-rolled one-off scripts first. Classifies each in-scope session into one of five structural `kind`s (`interrupted_explicit`, `net_error`, `done`, `empty`, `stuck_no_result`) and always prints the full session ID plus the complete last-assistant text, not a truncated title — the two axes ("what kind of record ends it" vs "does it still need a reply") are independent, and only reading the full text answers the second one. An independent review then found and this session fixed two blocker-severity bugs before ship: (1) the whole-file tool_use/tool_result pending tracker was a single-pass discard-then-add, which is **not** actually order-independent — `discard()` on an id not yet seen is a silent no-op, so a `tool_result` written before its `tool_use` (a real race on fast round-trips, per `references/session_file_format.md`'s "Tool Use / Tool Result Ordering") left the id wrongly "pending"; measured on a 500-file real sample, 14 hit the ordering and 11 (79%) had their classification flip. Fixed by accumulating two never-mutated sets and diffing them once at the end. (2) a turn with text *and* a tool_use block took its kind from the text alone regardless of block position, so a narrate-then-call-tool turn ending mid-tool-call (a real, non-exotic shape: 16 occurrences in a 401-file/36,829-record sample) reported `done` with the tool call invisible — the reference doc's own "trailing tool_use block" wording implied position-awareness the code didn't have. Fixed by deriving the kind from the raw content of the last assistant record only, and folding "tool call resolved but no further reply" and "thinking-only final turn" into the same `stuck_no_result` bucket as "still-pending tool call" — all three mean the same thing for triage purposes: the final turn produced no textual reply. Also fixed: an empty-result exit code/message that differed depending on whether `--kind` or the scope itself zeroed the result (now one check, one message, exit 1 either way); an unguarded `--all-projects` with no date bound could dump tens of thousands of lines with no warning (`--limit` default changed from unlimited to 200, with `--limit 0` as the explicit opt-in); and the `excluded ... automated` counter conflating two different exclusion mechanisms (generic smoke-test regex vs. the new `--exclude-title-prefix`) is now two counters.
|
||||
- **claude-code-history-files-finder** (`daymade-claude-code` v1.39.0): new **Core Operation 5 — Extract Verbatim User Messages** (`scripts/extract_user_messages.py` + 10 unit tests), producing a reading page (HTML + Markdown) of what the user actually typed across every home and archive. The extractor operationalizes v1.38.0's contamination taxonomy: command envelopes (XML wrapper and bare `/cmd`, args preserved) and hook/loop-injected boilerplate route to appendices — the boilerplate detector is generic (identical long text at >= `--min-dup` occurrences, no hardcoded patterns) and covers both the standalone and the tail-appended shapes; `[Image #N]` placeholders strip; whole-document pastes split off by the >=2000-chars-AND->=60%-ASCII rule; agent-voiced re-injection subtracts only against assistant texts *earlier* than the record (a later agent echo never eats the user's original). Mid-work input is recovered from `attachment.queued_command` (string and list payload variants) with a 120 s de-dupe window against delivered user records. A real-corpus run over a heavy multi-profile history reproduced the hand-built reference extraction from the incident session, with every remaining delta accounted for (sliding 7-day window between runs; the single user-confirmed agent-voiced entry a generic tool cannot know about).
|
||||
- **claude-code-history-files-finder** (`daymade-claude-code` v1.38.0): `session_file_format.md` gains two sections, both grounded in a real extraction incident where a "what the user actually said" archive shipped contaminated output through three rounds of user correction. (1) **`attachment` records: queued mid-work user input** — text typed while the assistant is still working never lands as a `type == "user"` record; it lives in `attachment.queued_command.prompt` (a string, with an observed list-of-blocks variant), and `attachment.origin.kind` separates `human` (the user) from `peer` (another agent/session) from absent (harness notifications). An extractor reading only user records dropped 153 messages over a 7-day window — exactly the interruption corrections. (2) **A user-role record is not necessarily user-authored text** — `promptSource: "typed"` / `origin.kind: "human"` prove only that text entered through the input box, not who wrote it. Five contamination classes with the splitters that worked: command envelopes (XML wrapper and bare `/cmd` — keep args, they carry real words); hook/loop-injected boilerplate (standalone records AND appended to the tail of the user's own sentence — a prefix-only filter misses the second shape); `[Image #N]` placeholders inside `text` blocks; whole-document pastes (normalized length ≥ 2000 chars AND ≥ 60% ASCII held up without misfiring on long Chinese voice dictations); and agent-voiced re-injection — undetectable from record fields, catchable only by content matching against assistant texts *earlier* than the record, and only in verbatim form (a partial rewrite with a verbatim title defeats exact-match and prefix matching alike). Also records the structurally-safe drop list: `system`/`sdk` promptSource, `isMeta`, `tool_result`, interrupt markers, compact-summary continuations.
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
Security scan passed
|
||||
Scanned at: 2026-07-27T08:18:01.901333+00:00
|
||||
Scanned at: 2026-08-07T01:31:58.579761+00:00
|
||||
Tool: gitleaks + pattern-based validation
|
||||
Content hash: b091903228ca133b66f5e704d70293441ee09dd71125e49a788e6d2113106eb7
|
||||
Content hash: a7b1399baa9de62de8ee718f2ba6a5c09df1e68b3c1ea78189abc820ec59e605
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
---
|
||||
name: claude-md-progressive-disclosurer
|
||||
description: |
|
||||
Optimize / slim down / restructure a CLAUDE.md (or AGENTS.md) using progressive disclosure — move low-frequency detail to Level 2 references while keeping Level 1 lean, WITHOUT losing information. Use this whenever the user asks to optimize / 精简 / 瘦身 / 重构 CLAUDE.md, asks "CLAUDE.md 是不是太大了 / 太长了" (is my CLAUDE.md too big / too long / bloated), wants to 把内容拆到 reference / 外部 / Level 2, do 整段外移 / 渐进式披露 / progressive disclosure, or whenever a CLAUDE.md duplicates info across files or the LLM keeps failing to follow its rules. ALSO trigger the moment an optimization turns into moving / cutting / compressing sections of a CLAUDE.md — even mid-task while another claude-md skill is already running. Distinct from claude-md quality auditors / scorers: this is the restructuring-and-offloading methodology that guarantees zero information loss (grep-verified pointers, verbatim moves, 5b content-integrity audit).
|
||||
Optimize / slim down / restructure a CLAUDE.md (or AGENTS.md) using progressive disclosure — move low-frequency detail to Level 2 references while keeping Level 1 lean, WITHOUT losing information. Use this whenever the user asks to optimize / 精简 / 瘦身 / 重构 CLAUDE.md, asks "CLAUDE.md 是不是太大了 / 太长了" (is my CLAUDE.md too big / too long / bloated), wants to 把内容拆到 reference / 外部 / Level 2, do 整段外移 / 渐进式披露 / progressive disclosure, or whenever a CLAUDE.md duplicates info across files or the LLM keeps failing to follow its rules. ALSO trigger the moment an optimization turns into moving / cutting / compressing sections of a CLAUDE.md — even mid-task while another claude-md skill is already running. Distinct from claude-md quality auditors / scorers: this is the restructuring-and-offloading methodology that guarantees zero information loss (grep-verified pointers, verbatim moves, 5b content-integrity audit). Covers hotspot-first profiling (先打热点) and cross-tool truncation (Codex project_doc_max_bytes).
|
||||
---
|
||||
|
||||
# CLAUDE.md 渐进式披露优化器
|
||||
@@ -89,6 +89,8 @@ Skills 描述常驻正文按需、以及 **Claude Code 的动态工具选择（
|
||||
它出现在最该出现的位置）。同一条规则挂两个载体不算违反 SSOT，前提是
|
||||
**只有一处写规则正文，另一处指向它**。
|
||||
|
||||
**官方载体新增（2026-08 对照 code.claude.com 官方文档核过）**：`~/.claude/rules/`（用户级）与项目级 `.claude/rules/` 可把规则拆成多主题文件；带 `paths:` frontmatter 的 rule **只在 Claude 操作匹配文件时加载**——官方版的按需载体。⚠️ 边界：paths 按**文件读写**触发、不按 Bash 命令触发——代理 / git 类"Bash 时刻"规则用不了它，文件类规则（`*.py` / `*.docx` / 测试文件）适用。另外三条官方事实随手可用：单文件目标 **<200 行**（官方原文 "Longer files consume more context and reduce adherence"）；`/doctor` 自带 CLAUDE.md trim 检查（v2.1.206+，可当独立第二意见）；**块级 HTML 注释在注入前被剥掉**——维护者备注零 token 成本。auto memory 的 MEMORY.md 只加载前 200 行 / 25KB，塞过期条目挤占的是活配额。
|
||||
|
||||
**🚫 缩正文前必须实证「真有别的机制在强制吗」，禁止推断。**
|
||||
把 L1 正文缩成「由 X 强制 + 指针」之前，构造一条真实的违规命令喂给 X，看它到底拦不拦：
|
||||
|
||||
@@ -181,7 +183,20 @@ cp CLAUDE.md CLAUDE.md.bak.$(date +%Y%m%d_%H%M%S)
|
||||
|
||||
### Step 2: 内容分类
|
||||
|
||||
分两阶段。**先分诊，再分层**——跳过分诊会把噪音忠实搬进 Level 2，把 reference 变垃圾场。
|
||||
分三阶段。**先测量，再分诊，再分层**——跳过测量会把力气花在小头上，跳过分诊会把噪音忠实搬进 Level 2，把 reference 变垃圾场。
|
||||
|
||||
#### 2.0 热点测量（先于一切提案——性能优化的第一课）
|
||||
|
||||
**先量化，后动手；按贡献度排序，先打最大的。** 优化提案落在 3% 的小头上、而 70% 的热点在旁边没人动，是本 skill 实战里被用户当场打断的真实失败（案例 19：一份 168KB 的全局 CLAUDE.md 占每 session 启动上下文 69%，执行者却先端出一盘扩展清理——用户原话「你没有先去管热点，而是先找了一堆很小很小的东西」）。测量四步（`scripts/profile_claude_md.py` 产出第 2/3 步；第 1 步是会话内命令、第 4 步手查各消费方配置）：
|
||||
|
||||
1. **全局占比**：`/context` 看这份文件在整个启动上下文里占多大——确认它是不是热点，还是别的类别才是
|
||||
2. **分节字节表**：按 heading 统计每节 bytes/lines 并降序——**工作顺序 = 这张表的降序**（⚠️ 父节字节含全部子节：降序在**同层之间**比较，容器节跳过看它最大的子节）；提案端出去前自问：这是当前最大贡献者吗？不是的话，最大的那个为什么不在最前面？
|
||||
3. **行长分布**：>1KB 的巨型行是「规则+战例焊死在一个 bullet」的签名（实战：4.4% 的行承载 35.6% 的字节）
|
||||
4. **消费方上限**：这份文件若被其他工具消费（如 Codex 经 `~/.codex/AGENTS.md` symlink 读同一文件），逐个查它们的截断上限——Codex `project_doc_max_bytes` 默认 32 KiB，超出的**尾部静默不可见、无任何报错**（实战：上限配过 96 KiB 当时够用，文件长到 164KB 后 41% 的正文对 Codex 隐形数周，整个协作规则 cluster 都在盲区）。修法 = 提上限 + 瘦文件双管齐下；再把「文件 size vs 上限」装进 SessionStart 体检哨兵——没有机械力量看着这个数字，它必然再次静默失效
|
||||
|
||||
⚠️ 测量仪器自身的两个坑（都实测踩过，脚本已内建规避；先在已知答案的样本上校准，见案例 17/19）：
|
||||
- **heading 正则必须感知 code fence**——fence 里的 `# 注释` 会被当成标题，凭空造出不存在的大节（实测造出过一个假的 45.9KB 节，热点排序整个失真）
|
||||
- **CJK 文件禁用 chars/4 估 token**——中文密集文本实测 ~0.42 token/byte（≈2.4 bytes/token），chars/4 低估一倍以上；有 `/context` 实测值就按实测比率折算，并一律标「est.」
|
||||
|
||||
#### 2.1 信号分诊（必要性闸门，先决）
|
||||
|
||||
@@ -233,6 +248,13 @@ cp CLAUDE.md CLAUDE.md.bak.$(date +%Y%m%d_%H%M%S)
|
||||
3. 可以在 Level 2 中添加结构（标题、分隔线），但**不要删减、改写、合并**原始内容
|
||||
4. 如果确实有冗余（同一段话在原文中出现了多次），在 Level 2 中保留一份完整的，注释说明去重
|
||||
|
||||
#### 整节批量下沉的机械流程（≥3 节时脚本化，禁手搬）
|
||||
|
||||
手工复制粘贴 10 个节必出错。用 `scripts/sink_sections.py`（spec 驱动；实战一次通过 10 节 / 119KB，整串验证 10/10 零丢失）：**按精确标题行定界提取原文（fence 感知）→ verbatim 追加到目标 reference（带日期 provenance header，新文件配 intro）→ 自底向上替换 L1 压缩版（行号不失效）→ 每节整串子串验证（grep 对多行原文按行 OR、会放过丢半段的搬运，必须 python `in` 整串判断）→ 任一验证失败自动回滚源文件**。两条硬规则：
|
||||
|
||||
- **拒写 symlink 目标（含父目录）**：目标路径任一环节是 symlink（文件本身、或**父目录**——文件级 `islink` 检查会被目录级 symlink 静默穿透，独立审阅实测打穿过），"本地追加"实际在改 link 指向的那个仓（触发它的版本 bump / commit 义务，且那个仓可能 public）。脚本按 `realpath ≠ abspath` 判定并 abort，特意跨 link（如 macOS `/tmp`）用 `--allow-symlinked-target` 显式放行；正确动作是落一个本地兄弟文件 + provenance 注明「与 symlink 源后续合并」
|
||||
- **先全部提取、后统一替换**：提取按原始行号一次做完，替换自底向上——两步交错会让未处理节的行号漂移。压缩版 snippet **必须保留原 start_heading 行**（验证器逐个检查标题存活，改名即 FAIL 回滚）；用作定界的标题在源文件里必须唯一（重名 abort）
|
||||
|
||||
### Step 4: 更新 Level 1
|
||||
|
||||
1. **在开头添加「信息记录原则」**（项目概述之后，Reference 索引之前）
|
||||
@@ -395,6 +417,8 @@ done < /tmp/pointers.txt
|
||||
|
||||
**禁止将"故意删除"作为分类来掩盖信息丢失。** 每一项"故意删除"都必须说明 canonical source 在哪里。如果说不出来，就不是"故意删除"，而是"遗漏"。
|
||||
|
||||
**压缩重述的保真审计（L1 留了压缩版时必查）**：压缩最容易丢的不是整段——是**限定词**。实战（案例 19）：原句「public + 0 stars/forks 且用户明确授权」被压成「0 stars 且明确授权」，6 个字符消失，一道闸门的条件字面上放宽了一半；同场审计还抓到「自称只省略战例、实际连 4 条可执行判据也省了」的申报口径不符。两个审计动作：① 对每条压缩重述，把**操作性子句**（条件 / 数值 / 枚举 / hook 名 / 否定词）与原句逐词 diff——整段丢失 5b 能抓，一个 "/forks" 只有子句级 diff 能抓；② 全文跑 expected-hunks-only 检查——difflib 比对基线，每个非 equal hunk 必须指认到一条已声明的改动，指认不了的就是计划外差异。
|
||||
|
||||
**独立 agent 做 5b 是默认动作，不是「大量压缩时才用」**：执行者自审有「乐观偏差」——倾向相信自己砍掉的内容都有归属。启动一个**独立 sub-agent**（**普通 subagent，禁 fork**——fork 继承你的盲区，只会盖个「已审」的章）做完整逐节 5b（读原始文件 + 当前文件 + 所有 reference，逐个信息点验证归属，只返回「真丢失 / 指针失准」清单）。它没有你的 sunk-cost，能抓到你抽查会放过的。prompt 模板 + 批量内容点 grep 脚本见 `references/progressive_disclosure_principles.md` 附录 D。
|
||||
|
||||
**为什么从「强烈推荐」升级为默认**：本 skill 的真实使用中，执行者抽查 5 点「自我感觉良好」，独立 agent 逐节查 55 点才暴露真问题。更硬的一次（2026-07-26）：**反模式 6 的规则完整、连「grep 原句逐字节仍命中」的判据都写在文件里，执行者读过、认同、自认在遵守，重写时仍删掉了一条约束**——而且它只存在于被删的那处，reference 里没有副本。**这不是缺规则，是散文规则由缺陷源本人执行时对该缺陷源天然失效**（poka-yoke 的 control vs warning：警告依赖警觉，而警觉正是完成驱动碾过的东西）。唯一逮住它的是换人的第二轮独立审阅。
|
||||
|
||||
+37
@@ -617,6 +617,43 @@ proxy、缩略图渲染…），**没有任何一个在拦 bypass 参数**。
|
||||
|
||||
---
|
||||
|
||||
## 案例 19：热点排序缺位 + 消费方静默截断 + 压缩丢限定词（真实使用事故，2026-08-07）
|
||||
|
||||
### 背景
|
||||
|
||||
一份用户级全局 `~/.claude/CLAUDE.md` 长到 168KB / 895 行（实测 ~70k tokens，占每个 session 启动上下文的 69%）。用户问「怎么优化」，本 skill 正确触发并完成 reframe（没有跑向砍行数）；诊断也正确定位了结构问题——39 个 >1KB 的巨型 bullet 承载全文 35.6% 的字节，形态全是「规则句 + 完整战例焊死在一个 bullet」。
|
||||
|
||||
### 错误做法
|
||||
|
||||
执行者接着跑了一轮环境体检，把产出的**扩展清理提案**（禁用若干插件与 skill，估算收益 2.5–3.5k tokens ≈ 全部启动底噪的 3%）端给用户确认——而 70k 的热点文件在旁边等一句「go」。用户当场打断：
|
||||
|
||||
> 「你没有遵守性能优化的方法论，你没有先去管热点，而是先找了一堆很小很小的东西。」
|
||||
|
||||
### 问题
|
||||
|
||||
1. **测量做了，排序没做。** 所有数字都在手上（70k vs 3k），但提案顺序没按贡献度排——把「能立即做的小活」排在了「等确认的大活」前面。性能优化第一课：先打最大贡献者，热点清完再看小头还剩多少。
|
||||
2. **消费方截断静默失效数周。** 同一文件经 `~/.codex/AGENTS.md` symlink 被 Codex 读取，`project_doc_max_bytes` 配过 96 KiB——配置时够用；文件长到 164KB 时（发现截断的时点，优化启动时已 168KB），没有任何机制看着这个数字，41% 的正文（整个协作规则 cluster、事故响应节、全部查表层）对 Codex 隐形，零报错。
|
||||
3. **压缩重述丢限定词。** 热点下沉执行后，独立审计在 L1 压缩版里抓到：原句「public + 0 stars/forks 且用户明确授权」被压成「0 stars 且明确授权」——6 个字符消失，一道推送闸门的条件字面上放宽了一半。同场还抓到「自称只省略战例，实际连 4 条可执行判据行也省了」的申报口径不符。
|
||||
4. **测量仪器自身埋雷。** 第一版分节统计把 code fence 里的 `# 注释` 当成 heading，凭空造出一个 45.9KB 的假节；CJK 密集文本用 chars/4 估 token 低估一倍以上（该文件实测 ~0.42 token/byte）。
|
||||
|
||||
### 正确做法（当场修正后的完整序）
|
||||
|
||||
1. **Step 2.0 热点测量先行**：`/context` 全局占比 → 分节字节表降序 → 行长分布 → 消费方上限，四个数字摆齐再谈方案；**工作顺序 = 分节表降序**。
|
||||
2. **消费方上限当场修**（一行 config），并把「文件 size vs 上限」装进 SessionStart 体检哨兵——修一次不够，要让下次超限自己报警。
|
||||
3. **热点下沉走脚本化整节流程**（SKILL.md Step 3 的机械流程）：10 节 / 119KB verbatim 下沉到 6 个 reference，整串验证 10/10，L1 从 168KB 降到 92KB，信息丢失 0（独立审计确认：46 个 diff hunk 中 45 个落在声明的替换区间、1 个是声明过的插入）。（size 数字是下沉的副产物记录——**通过条件是整串验证与独立审计，不是这个数**；别把它当汇报成果，那是铁律与案例 15 判违规的形状。）
|
||||
4. **审计维度含压缩保真**：操作性子句逐词 diff（抓到 /forks）+ expected-hunks-only 全文差异指认。
|
||||
5. **防复长写进目标文件的治理层**：「战例出生在主题 reference，L1 只收规则句 + 一行 Why + 指针」——否则一次性清理会在几个月内长回来（该文件近三天就新增了 27 处带日期战例条目）。
|
||||
|
||||
### 教训
|
||||
|
||||
1. **测量的产出是排序，不是数字。** 量完不排序、排完不按序做，测量就白做了——提案端出去前自问：这是当前最大贡献者吗？不是的话，最大的那个为什么不在最前面？
|
||||
2. **文件有第二个读者时，优化对象是「文件 × 每个消费方的截断窗口」。** 查所有消费方的上限是测量步骤的一部分，不是收尾彩蛋。
|
||||
3. **静默失效的配置需要哨兵。** 「当时配够了」的上限会被增长追上；没有机械检查盯着的数字等于没有数字。
|
||||
4. **压缩的单位风险在限定词。** 整段丢失 5b 能抓，一个 "/forks" 只有子句级逐词 diff 能抓——压缩版审计必须下到子句级。
|
||||
5. **先校准仪器再测量**（案例 17 的延伸）：fence 感知、CJK token 比率——测量脚本自己就是仪器，先在已知答案的样本上跑一遍再信它。
|
||||
|
||||
---
|
||||
|
||||
## 附录 A：信息记录原则模板（供注入用户 CLAUDE.md）
|
||||
|
||||
> SKILL.md 原则 0 的完整模板。触发场景：执行 Step 4 更新 Level 1 时，把下面整块原样复制进目标 CLAUDE.md 开头（项目概述之后）。
|
||||
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Profile a CLAUDE.md before proposing any optimization (SKILL.md Step 2.0).
|
||||
|
||||
Outputs, in one pass:
|
||||
1. Per-section byte/line table (code-fence-aware), sorted by size — the work
|
||||
order for hotspot-first optimization IS this table's descending order.
|
||||
2. Line-length distribution — lines >1KB are the signature of "rule + war
|
||||
story fused into one bullet".
|
||||
3. The longest N lines with previews, to locate the giant bullets.
|
||||
|
||||
Instrument calibration notes (both pitfalls were hit in real use — Case 19):
|
||||
- Headings are parsed fence-aware: a `# comment` inside a code fence is NOT
|
||||
a heading. A naive regex once fabricated a phantom 45.9KB section and
|
||||
distorted the entire hotspot ranking.
|
||||
- Token estimates: chars/4 UNDERESTIMATES CJK-dense files by >2x. Measured
|
||||
ratio on a real CJK-heavy CLAUDE.md was ~0.42 tokens/byte. Prefer scaling
|
||||
from a live `/context` measurement; always label estimates "est.".
|
||||
|
||||
Prior art (searched 2026-08, none fits): mrkdwn-analysis / markdown word-count
|
||||
tools do element counts and word/char stats, not per-section BYTE attribution
|
||||
with hotspot ordering — which is this skill's Step 2.0 artifact. Stdlib-only
|
||||
by design (skill scripts must run with no installs).
|
||||
|
||||
Usage:
|
||||
python3 profile_claude_md.py <path/to/CLAUDE.md> [--top N] [--tokens-per-byte R]
|
||||
"""
|
||||
import argparse
|
||||
import re
|
||||
import sys
|
||||
|
||||
HEADING_RE = re.compile(r'^(#{1,6}) (.+)$')
|
||||
|
||||
|
||||
def parse_headings(lines):
|
||||
"""Return [(line_idx, level, title)], skipping fenced code blocks."""
|
||||
in_fence = False
|
||||
heads = []
|
||||
for i, line in enumerate(lines):
|
||||
if line.lstrip().startswith('```'):
|
||||
in_fence = not in_fence
|
||||
continue
|
||||
if in_fence:
|
||||
continue
|
||||
m = HEADING_RE.match(line)
|
||||
if m:
|
||||
heads.append((i, len(m.group(1)), m.group(2).strip()))
|
||||
if in_fence:
|
||||
print('WARN: unbalanced code fence — fence-aware parsing may be off', file=sys.stderr)
|
||||
return heads
|
||||
|
||||
|
||||
def section_table(lines, heads, max_level=3):
|
||||
"""Each heading owns [its line, next heading of same-or-higher level)."""
|
||||
rows = []
|
||||
for idx, (ln, lvl, title) in enumerate(heads):
|
||||
if lvl > max_level:
|
||||
continue
|
||||
end = len(lines)
|
||||
for j in range(idx + 1, len(heads)):
|
||||
if heads[j][1] <= lvl:
|
||||
end = heads[j][0]
|
||||
break
|
||||
body = '\n'.join(lines[ln:end])
|
||||
rows.append((lvl, title, len(body.encode('utf-8')), end - ln))
|
||||
return rows
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('path')
|
||||
ap.add_argument('--top', type=int, default=20, help='longest lines to list (default 20)')
|
||||
ap.add_argument('--tokens-per-byte', type=float, default=None,
|
||||
help='ratio from a live /context measurement; omit to skip token estimates')
|
||||
args = ap.parse_args()
|
||||
|
||||
src = open(args.path, encoding='utf-8').read()
|
||||
lines = src.split('\n')
|
||||
total = len(src.encode('utf-8'))
|
||||
heads = parse_headings(lines)
|
||||
|
||||
print(f'TOTAL {total} bytes / {len(lines)} lines / {len(heads)} headings')
|
||||
if args.tokens_per_byte:
|
||||
print(f'est. tokens: ~{int(total * args.tokens_per_byte):,} '
|
||||
f'(ratio {args.tokens_per_byte} from live measurement)')
|
||||
else:
|
||||
print('token estimate skipped — pass --tokens-per-byte from a live /context '
|
||||
'measurement (chars/4 underestimates CJK by >2x)')
|
||||
|
||||
print(f'\n== 分节字节表（降序 = 热点工作顺序） ==')
|
||||
print(' （父节字节含全部子节——降序在**同层之间**比较；容器节跳过、看它最大的子节）')
|
||||
print(f"{'lvl':<5}{'bytes':>9}{'lines':>7}{'%':>7} title")
|
||||
for lvl, title, b, nl in sorted(section_table(lines, heads), key=lambda r: -r[2]):
|
||||
print(f"{'#'*lvl:<5}{b:>9}{nl:>7}{100*b/total:>6.1f}% {title[:64]}")
|
||||
|
||||
print('\n== 行长分布 ==')
|
||||
buckets = [('<200B', 0, 200), ('200-500B', 200, 500), ('500B-1KB', 500, 1000),
|
||||
('1-2KB', 1000, 2000), ('>2KB', 2000, 10**9)]
|
||||
for name, lo, hi in buckets:
|
||||
sel = [len(l.encode()) for l in lines if lo <= len(l.encode()) < hi]
|
||||
by = sum(sel)
|
||||
print(f"{name:<10} {len(sel):>6} 行 {by:>9} B {100*by/total:>6.1f}%")
|
||||
|
||||
print(f'\n== 最长 {args.top} 行（巨型 bullet 定位） ==')
|
||||
ranked = sorted(((len(l.encode()), i + 1, l) for i, l in enumerate(lines)), reverse=True)
|
||||
for b, i, l in ranked[:args.top]:
|
||||
print(f'{b:>7}B L{i:<6} {l[:90]}')
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+183
@@ -0,0 +1,183 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Verbatim whole-section sink for CLAUDE.md progressive disclosure (SKILL.md Step 3).
|
||||
|
||||
For each section in the spec: extract [start_heading, end_heading) from the source
|
||||
(exact-line match, code-fence-aware) -> append verbatim to the target reference under
|
||||
a dated provenance header (new files get an intro) -> replace the L1 range with the
|
||||
compressed snippet, bottom-up so earlier line numbers stay valid -> verify every
|
||||
original is a WHOLE-STRING substring of its target (grep ORs per line and would pass
|
||||
a lossy move) -> roll the source back if any check fails.
|
||||
|
||||
Hard rules encoded here (each learned in real use — Case 19):
|
||||
- REFUSES targets whose path crosses a symlink at ANY component — the file
|
||||
itself OR any parent directory (checked as realpath != abspath; a file-only
|
||||
islink() check was bypassed in review by a symlinked parent dir). A symlinked
|
||||
target means your "local" append actually edits whatever repo it points into
|
||||
(its version-bump/commit obligations, possibly public). Sink to a local
|
||||
sibling file instead and note the merge-later — or pass
|
||||
--allow-symlinked-target when the hop is deliberate (e.g. macOS /tmp).
|
||||
- Extract everything first (original line numbers), then splice bottom-up.
|
||||
- Fail-fast on missing/suspiciously-small snippets BEFORE writing anything.
|
||||
- Timestamped backups of the source and every touched existing target.
|
||||
- Headings used as boundaries must be UNIQUE in the source (ambiguity aborts).
|
||||
|
||||
Spec (JSON):
|
||||
{
|
||||
"source": "path/to/CLAUDE.md",
|
||||
"sections": [
|
||||
{
|
||||
"key": "short-id",
|
||||
"start_heading": "### Exact Heading Line",
|
||||
"end_heading": "### Next Heading Line (exclusive; OMIT for a last-section-to-EOF sink)",
|
||||
"snippet_file": "compressed replacement. MUST retain the section's original start_heading line verbatim — the post-splice verifier checks every heading survived; a renamed heading fails and rolls back",
|
||||
"target_ref": "path/to/reference.md",
|
||||
"provenance_title": "Section name for the archive header",
|
||||
"new_file_intro": "# Title\\n\\n> optional; written only when target_ref is new\\n"
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Prior art (searched 2026-08): mdsplit (github.com/markusstraub/mdsplit) splits a
|
||||
whole file into per-heading files with fidelity guarantees, but has no notion of
|
||||
sinking SELECTED sections into EXISTING topical references, in-place L1
|
||||
replacement, whole-string verification, rollback, or the symlink guard — the
|
||||
transactional contract this script exists for. Stdlib-only by design.
|
||||
|
||||
Usage:
|
||||
python3 sink_sections.py <spec.json> [--min-snippet-bytes 200]
|
||||
"""
|
||||
import argparse
|
||||
import datetime
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
|
||||
def find_heading(lines, heading):
|
||||
"""All exact full-line matches, skipping fenced code blocks."""
|
||||
in_fence = False
|
||||
hits = []
|
||||
for i, line in enumerate(lines):
|
||||
if line.lstrip().startswith('```'):
|
||||
in_fence = not in_fence
|
||||
continue
|
||||
if in_fence:
|
||||
continue
|
||||
if line.rstrip() == heading:
|
||||
hits.append(i)
|
||||
return hits
|
||||
|
||||
|
||||
def find_unique_heading(lines, heading, key, role):
|
||||
hits = find_heading(lines, heading)
|
||||
if not hits:
|
||||
sys.exit(f"ABORT: {role} heading not found [{key}]: {heading[:60]}")
|
||||
if len(hits) > 1:
|
||||
sys.exit(f"ABORT: {role} heading ambiguous [{key}] — appears {len(hits)}x "
|
||||
f"(lines {', '.join(str(h+1) for h in hits)}): {heading[:60]}")
|
||||
return hits[0]
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument('spec')
|
||||
ap.add_argument('--min-snippet-bytes', type=int, default=200,
|
||||
help='abort if a compressed snippet is smaller than this (default 200)')
|
||||
ap.add_argument('--allow-symlinked-target', action='store_true',
|
||||
help='permit a target whose path crosses a symlink (deliberate hops only, '
|
||||
'e.g. macOS /tmp -> /private/tmp); default aborts')
|
||||
args = ap.parse_args()
|
||||
|
||||
spec = json.load(open(args.spec, encoding='utf-8'))
|
||||
src_path = spec['source']
|
||||
sections = spec['sections']
|
||||
|
||||
# ---- fail-fast preflight: snippets exist & are not stubs -------------------
|
||||
snippets = {}
|
||||
for s in sections:
|
||||
p = s['snippet_file']
|
||||
if not os.path.isfile(p):
|
||||
sys.exit(f"ABORT: snippet missing: {p}")
|
||||
text = open(p, encoding='utf-8').read()
|
||||
if len(text.strip().encode()) < args.min_snippet_bytes:
|
||||
sys.exit(f"ABORT: snippet suspiciously small (<{args.min_snippet_bytes}B): {p}")
|
||||
snippets[s['key']] = text.rstrip() + '\n\n'
|
||||
|
||||
src = open(src_path, encoding='utf-8').read()
|
||||
lines = src.split('\n')
|
||||
|
||||
# ---- locate all boundaries in ORIGINAL coordinates -------------------------
|
||||
bounds = []
|
||||
for s in sections:
|
||||
a = find_unique_heading(lines, s['start_heading'], s['key'], 'start')
|
||||
if s.get('end_heading'):
|
||||
b = find_unique_heading(lines, s['end_heading'], s['key'], 'end')
|
||||
else:
|
||||
b = len(lines) # omitted end_heading = sink through EOF
|
||||
if b <= a:
|
||||
sys.exit(f"ABORT: boundary fail [{s['key']}]: start={a} end={b}")
|
||||
bounds.append((s, a, b))
|
||||
for (s1, a1, b1), (s2, a2, b2) in zip(sorted(bounds, key=lambda x: x[1]),
|
||||
sorted(bounds, key=lambda x: x[1])[1:]):
|
||||
if b1 > a2:
|
||||
sys.exit(f"ABORT: overlap [{s1['key']}] and [{s2['key']}]")
|
||||
|
||||
# ---- symlink guard (file OR any parent dir) + backups -----------------------
|
||||
ts = datetime.datetime.now().strftime('%Y%m%d_%H%M%S')
|
||||
for s, _, _ in bounds:
|
||||
tgt = s['target_ref']
|
||||
literal, resolved = os.path.abspath(tgt), os.path.realpath(tgt)
|
||||
if resolved != literal and not args.allow_symlinked_target:
|
||||
sys.exit(f"ABORT: target path crosses a symlink: {tgt}\n"
|
||||
f" literal : {literal}\n"
|
||||
f" resolves: {resolved}\n"
|
||||
f" A symlinked component (file OR parent dir) silently edits whatever repo it\n"
|
||||
f" points into. Sink to a local sibling file and note the merge-later task —\n"
|
||||
f" or re-run with --allow-symlinked-target if this hop is deliberate.")
|
||||
shutil.copy2(src_path, f'{src_path}.bak.presink.{ts}')
|
||||
for tgt in {s['target_ref'] for s, _, _ in bounds}:
|
||||
if os.path.isfile(tgt):
|
||||
shutil.copy2(tgt, f'{tgt}.bak.{ts}')
|
||||
|
||||
# ---- 1) append originals verbatim ------------------------------------------
|
||||
today = datetime.date.today().isoformat()
|
||||
extracted = {}
|
||||
for s, a, b in bounds:
|
||||
orig = '\n'.join(lines[a:b])
|
||||
extracted[s['key']] = (orig, s['target_ref'])
|
||||
header = (f"\n\n---\n\n# [{today} 下沉] {s['provenance_title']} —— 原文 verbatim 存档\n\n")
|
||||
if not os.path.isfile(s['target_ref']):
|
||||
intro = s.get('new_file_intro', f"# {s['provenance_title']}\n")
|
||||
open(s['target_ref'], 'w', encoding='utf-8').write(intro)
|
||||
with open(s['target_ref'], 'a', encoding='utf-8') as f:
|
||||
f.write(header + orig.rstrip() + '\n')
|
||||
print(f"appended [{s['key']}] {len(orig.encode()):>8}B -> {os.path.basename(s['target_ref'])}")
|
||||
|
||||
# ---- 2) splice compressed snippets, bottom-up ------------------------------
|
||||
for s, a, b in sorted(bounds, key=lambda x: -x[1]):
|
||||
lines[a:b] = snippets[s['key']].split('\n')
|
||||
new = '\n'.join(lines)
|
||||
open(src_path, 'w', encoding='utf-8').write(new)
|
||||
|
||||
# ---- 3) verify: whole-string substring, headings survive -------------------
|
||||
fails = []
|
||||
for key, (orig, tgt) in extracted.items():
|
||||
if orig.rstrip() not in open(tgt, encoding='utf-8').read():
|
||||
fails.append(f"[{key}] original not whole-string-present in {tgt}")
|
||||
cur = open(src_path, encoding='utf-8').read()
|
||||
for s, _, _ in bounds:
|
||||
if s['start_heading'] not in cur:
|
||||
fails.append(f"[{s['key']}] L1 heading lost: {s['start_heading'][:40]}")
|
||||
if fails:
|
||||
print('\nVERIFY FAIL:', *(' ' + f for f in fails), sep='\n')
|
||||
shutil.copy2(f'{src_path}.bak.presink.{ts}', src_path)
|
||||
sys.exit('source rolled back (reference appends kept for inspection)')
|
||||
|
||||
ob, nb = len(src.encode()), len(cur.encode())
|
||||
print(f"\nOK {len(bounds)}/{len(bounds)} sections verified (whole-string) | "
|
||||
f"source {ob} -> {nb} bytes | backup: {src_path}.bak.presink.{ts}")
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
Reference in New Issue
Block a user