From 882d24ff3f67a1aad14d8fc11335bca084f60dbf Mon Sep 17 00:00:00 2001 From: Lingzhi Yang Date: Fri, 20 Feb 2026 19:53:15 -0500 Subject: [PATCH] Add skills design spec and implementation checklist Co-Authored-By: Claude Opus 4.6 --- SKILLS_DESIGN.md | 768 ++++++++++++++++++++++++++++++++++++++++++ SKILLS_IMPLEMENTED.md | 585 ++++++++++++++++++++++++++++++++ 2 files changed, 1353 insertions(+) create mode 100644 SKILLS_DESIGN.md create mode 100644 SKILLS_IMPLEMENTED.md diff --git a/SKILLS_DESIGN.md b/SKILLS_DESIGN.md new file mode 100644 index 0000000..171bd78 --- /dev/null +++ b/SKILLS_DESIGN.md @@ -0,0 +1,768 @@ +# Research Paper Writing: Claude Skill Modules + +> Extracted from 17 GitHub repos studying LLM-agent-driven research automation. +> Each skill is a self-contained, reusable Claude Code skill for the research paper lifecycle. + +--- + +## Skill Map Overview + +``` +Phase 0: Research Planning + [S01] research-planning — 研究规划与论文架构设计 + [S02] idea-generation — 研究想法生成与新颖性评估 + +Phase 1: Literature & Knowledge + [S03] literature-search — 文献检索（Semantic Scholar / arXiv / Google Scholar） + [S04] literature-review — 文献综述生成（多视角对话模拟） + [S05] related-work-writing — Related Work 段落撰写 + +Phase 2: Formalization + [S06] math-reasoning — 数学推理、公式推导、定理证明 + [S07] algorithm-design — 算法设计与伪代码生成 + [S08] atomic-decomposition — 原子概念分解（数学公式 ↔ 代码双向映射） + +Phase 3: Implementation + [S09] experiment-code — 实验代码编写（ML training/eval pipeline） + [S10] code-debugging — 代码调试与自动修复 + [S11] experiment-design — 实验设计（baselines、ablation、hyperparameter） + +Phase 4: Results & Visualization + [S12] data-analysis — 数据分析与统计检验 + [S13] figure-generation — 科研图表生成（matplotlib + VLM反馈闭环） + [S14] table-generation — LaTeX 表格生成（实验结果 → 发表级表格） + +Phase 5: Writing + [S15] paper-writing-section — 逐节论文撰写（Abstract → Conclusion） + [S16] latex-formatting — LaTeX 格式、模板、编译 + [S17] citation-management — BibTeX 引用管理与引文插入 + +Phase 6: Review & Polish + [S18] self-review — 自动审稿（NeurIPS/ICLR review form） + [S19] rebuttal-writing — Rebuttal 撰写（逐条回复审稿意见） + [S20] paper-revision — 基于审稿意见的论文修改 + +Phase 7: Assembly & Delivery + [S21] paper-assembly — 论文全流程整合（数学+图表+数据+引用 → 完整论文） + [S22] paper-compilation — LaTeX 编译与错误修复 + [S23] backward-traceability — 数据可追溯（PDF中数字 → 生成代码行的超链接） + +Phase 8: Extended + [S24] survey-generation — 综述论文自动生成 + [S25] paper-to-code — 论文 → 可运行代码仓库 + [S26] slide-generation — 论文 → 演示幻灯片/Poster + [S27] novelty-assessment — 研究新颖性评估与文献对比 + [S28] symbolic-equation — 科学方程发现（符号回归 + LLM引导） +``` + +--- + +## Detailed Skill Specifications + +--- + +### S01: research-planning — 研究规划与论文架构设计 + +**Source repos:** AI-Researcher (plan_agent), AgentLaboratory (plan_formulation), Paper2Code (1_planning.py) + +**What it does:** +- Takes a research topic/idea as input +- Outputs: research questions, methodology outline, paper structure, section-by-section plan +- Generates Mermaid class/sequence diagrams for system architecture (from Paper2Code) +- Creates a dependency graph of what needs to be done first + +**Key techniques extracted:** +1. **4-turn planning conversation** (Paper2Code): Overall plan → Architecture design (UML) → Logic analysis (task list) → Config extraction +2. **Diverge-converge framework** (AI-Researcher): Generate 5 orthogonal research directions, score by novelty/reliability/potential, deepen the best +3. **Plan formulation dialogue** (AgentLaboratory): Postdoc-PhD role-play to refine plan iteratively + +**Prompt template core:** +``` +You are an experienced research advisor. Given a research topic and optional reference papers: + +1. Identify the core research question and its significance +2. Propose a methodology with clear steps +3. Design the paper structure (8 sections: Abstract, Introduction, Background, Related Work, Methods, Experiments, Results, Discussion) +4. Create a dependency-ordered task list for implementation +5. Identify key baselines, datasets, and evaluation metrics +6. Flag risks and potential failure modes + +Output format: structured JSON with sections, tasks, dependencies, and timeline. +``` + +--- + +### S02: idea-generation — 研究想法生成与新颖性评估 + +**Source repos:** AI-Scientist (generate_ideas.py), SciMON, AI-Researcher (idea_agent) + +**What it does:** +- Given a research area + existing codebase, generates novel research ideas +- Iterative reflection (up to 5 rounds) to refine each idea +- Novelty check against Semantic Scholar / arXiv +- Scores ideas on: Interestingness, Feasibility, Novelty (1-10) + +**Key techniques:** +1. **Seed idea bootstrapping** (AI-Scientist): Start with 1-2 hand-written ideas, generate more conditioned on archive +2. **Reflection convergence** (AI-Scientist): Refine until LLM says "I am done" +3. **Literature-grounded novelty check** (AI-Scientist): Up to 10 rounds of Semantic Scholar queries +4. **Novelty optimization loop** (SciMON): Retrieve inspirations → iteratively optimize for novelty + +**Output per idea:** +```json +{ + "Name": "adaptive_attention_pruning", + "Title": "Adaptive Attention Head Pruning via Gradient-Guided Importance Scoring", + "Experiment": "detailed implementation plan...", + "Interestingness": 8, + "Feasibility": 7, + "Novelty": 9 +} +``` + +--- + +### S03: literature-search — 文献检索 + +**Source repos:** STORM (rm.py), AI-Scientist (generate_ideas.py), data-to-paper (literature_search.py), OpenResearcher + +**What it does:** +- Multi-source academic search: Semantic Scholar, arXiv, OpenAlex, CrossRef, Google Scholar +- Returns structured results: title, authors, year, venue, abstract, citation count, BibTeX +- Supports embedding-based similarity sorting +- Filters by citation count, recency, relevance + +**API integrations:** +- Semantic Scholar: `api.semanticscholar.org/graph/v1/paper/search` (fields: title, authors, venue, year, abstract, citationCount, citationStyles) +- arXiv: `arxiv` Python package +- OpenAlex: `pyalex` library +- CrossRef: REST API for DOI-based lookup + +--- + +### S04: literature-review — 文献综述生成 + +**Source repos:** STORM (knowledge_curation.py), AutoSurvey, AgentLaboratory (literature_review phase) + +**What it does:** +- Multi-perspective dialogue simulation: generates N expert personas, each conducts multi-turn Q&A +- Each persona asks questions from their unique angle, grounded by live search results +- Synthesizes all conversations into a structured knowledge base +- Citation precision ~85%, recall ~85% (STORM benchmark) + +**Key techniques:** +1. **Persona generation** (STORM): Find related Wikipedia topics → extract TOC → LLM generates diverse editor personas +2. **Grounded expert answers** (STORM): TopicExpert converts questions to search queries, retrieves top-k results, synthesizes answer with inline citations +3. **Parallel persona execution** (STORM): ThreadPoolExecutor runs all persona conversations concurrently +4. **Semantic retrieval** (STORM): SentenceTransformer embeddings + cosine similarity for per-section information retrieval + +--- + +### S05: related-work-writing — Related Work 段落撰写 + +**Source repos:** STORM, LitLLM, AI-Scientist (perform_writeup.py) + +**What it does:** +- Takes collected literature + paper draft as input +- Generates a Related Work section with proper citations +- "Compare and contrast" style, not just descriptions +- Sentence-level planning (LitLLM): keyword extraction → retrieval → re-ranking → generation + +**Prompt core (from AI-Scientist per_section_tips):** +``` +Related Work should compare and contrast prior work with your approach. +Don't just describe what others did — explain how your work differs and why. +Organize by theme, not chronologically. +Cite broadly — not just the most popular papers. +``` + +--- + +### S06: math-reasoning — 数学推理、公式推导、定理证明 + +**Source repos:** AI-Researcher (survey_agent math extraction), LLM-SR, data-to-paper (analysis coding) + +**What it does:** +- Formal mathematical derivation with step-by-step reasoning +- Theorem statement and proof generation in LaTeX +- Equation numbering and cross-referencing +- Statistical test selection and interpretation +- Symbolic equation discovery (LLM-SR: evolutionary search for scientific equations) + +**Key techniques:** +1. **Math formula extraction from papers** (AI-Researcher Paper Survey Agent): Navigate paper → find formula → extract LaTeX +2. **Backward-traceable computations** (data-to-paper): `\num{formula}` → evaluated at compile time → hyperlinked to source +3. **LLM-guided symbolic regression** (LLM-SR): LLM proposes candidate equations → evolutionary search optimizes fit + +--- + +### S07: algorithm-design — 算法设计与伪代码生成 + +**Source repos:** Paper2Code (planning stage), AI-Researcher (plan_agent) + +**What it does:** +- Generates algorithm pseudocode in LaTeX `algorithmic` environment +- Creates UML class diagrams (Mermaid syntax) +- Designs sequence diagrams for system flow +- Ensures consistency between pseudocode and actual implementation + +**Prompt pattern:** +``` +Given a method description: +1. Formalize the algorithm with clear input/output +2. Write pseudocode using \begin{algorithmic} environment +3. Generate a Mermaid classDiagram showing data structures +4. Generate a Mermaid sequenceDiagram showing execution flow +5. Verify consistency: every pseudocode step must map to a code module +``` + +--- + +### S08: atomic-decomposition — 原子概念分解 + +**Source repos:** AI-Researcher (survey_agent.py) — this is its core innovation + +**What it does:** +- Decomposes a research idea into atomic, self-contained concepts +- For each atom: extracts math formula from papers + finds code implementation in repos +- Bidirectional mapping: `math_formula ↔ code_implementation` +- Creates a structured note for each concept with: definition, formula, code, references + +**Output format:** +```json +{ + "definition": "Kernelized Gumbel-Softmax Operator", + "math_formula": "Z = softmax((log π + g) / τ), g ~ Gumbel(0,1)", + "code_implementation": "def gumbel_softmax(logits, tau=1.0): ...", + "reference_papers": ["NodeFormer: A Scalable Graph Structure Learning Transformer"], + "reference_codebases": ["LarFii/nodeformer"] +} +``` + +**Why this matters:** Ensures every mathematical concept in the paper has a verified code implementation, and every code module traces back to a formal mathematical definition. + +--- + +### S09: experiment-code — 实验代码编写 + +**Source repos:** AI-Scientist (perform_experiments.py), AgentLaboratory (mlesolver.py), AI-Researcher (ml_agent.py) + +**What it does:** +- Generates complete ML training/evaluation pipelines +- Iterative code improvement via REPLACE/EDIT operations +- Self-debugging: runs code, captures errors, fixes automatically (up to 4 retries) +- Hill-climbing optimization: maintains pool of best-scoring code candidates + +**Key techniques:** +1. **Aider diff-mode editing** (AI-Scientist): LLM outputs unified diffs applied to experiment files +2. **REPLACE/EDIT operations** (AgentLaboratory): Two editing primitives with line-range targeting +3. **Mandatory project structure** (AI-Researcher): `data/`, `model/`, `training/`, `testing/`, `run_training_testing.py` +4. **Code reflection** (AgentLaboratory): After improvement, reflect on what worked → inject insights into next iteration + +**Constraints enforced:** +- No placeholder code (`pass`, `...`, `raise NotImplementedError`) +- Must use actual datasets (not toy data) +- Must generate figures (Figure_1.png, Figure_2.png minimum) +- PyTorch or scikit-learn only (no TensorFlow/Keras) + +--- + +### S10: code-debugging — 代码调试与自动修复 + +**Source repos:** AI-Scientist (perform_experiments.py), data-to-paper (debugger.py), AI-Scientist-v2 (parallel_agent.py _debug()) + +**What it does:** +- Captures runtime errors from experiment execution +- Feeds truncated stderr + code context back to LLM +- Structured error categorization: ImportError, FileNotFoundError, TimeoutError, etc. +- State machine for fix strategy: "repost" / "leave" / "regen" +- Up to 5 repair attempts with code reflection + +**Key techniques:** +1. **Truncated error feedback** (AI-Scientist): Last N lines of stderr fed back +2. **Structured RunIssue objects** (data-to-paper): category + description + fix instructions +3. **Monkey-patched sandbox** (data-to-paper): Override pandas, scipy, sklearn to track operations and enforce safety +4. **Automated code repair tool** (AgentLaboratory): Dedicated "repair" system prompt distinct from "generate" prompt + +--- + +### S11: experiment-design — 实验设计 + +**Source repos:** AI-Scientist-v2 (agent_manager.py 4-stage), AI-Researcher (exp_analyser.py) + +**What it does:** +- Designs complete experiment plans: baselines, ablations, hyperparameter sweeps +- 4-stage progressive experiment management: + - Stage 1: Initial implementation (simple dataset, working baseline) + - Stage 2: Baseline tuning (hyperparameters, multiple datasets) + - Stage 3: Creative research (novel improvements, 3+ datasets) + - Stage 4: Ablation studies (systematic component analysis) +- Multi-seed evaluation for statistical significance +- VLM-evaluated stage completion (training curves checked for convergence) + +**Key techniques:** +1. **Progressive staging** (AI-Scientist-v2): Each stage has explicit goals and completion criteria +2. **Ablation study design** (AI-Researcher): Analyzes results → proposes component removal experiments +3. **Sensitivity analysis** (AI-Researcher): Tests key hyperparameters across ranges +4. **Best-node selection** (AI-Scientist-v2): LLM holistically selects best experiment considering metrics + VLM plot analysis + +--- + +### S12: data-analysis — 数据分析与统计检验 + +**Source repos:** data-to-paper (analysis coding), AgentLaboratory (results_interpretation) + +**What it does:** +- Generates Python code for statistical analysis of experimental results +- Selects appropriate statistical tests (t-test, ANOVA, chi-square, etc.) +- Interprets p-values, effect sizes, confidence intervals +- Produces structured analysis reports with numerical evidence + +**Key techniques:** +1. **4-round code review** (data-to-paper): + - Round 1: Fundamental math/statistics flaws + - Round 2: Data handling issues + - Round 3: Per-table review + - Round 4: Cross-table completeness +2. **Allowed packages whitelist** (data-to-paper): pandas, numpy, scipy, statsmodels, sklearn only +3. **Results interpretation dialogue** (AgentLaboratory): Postdoc guides PhD to extract meaningful insights + +--- + +### S13: figure-generation — 科研图表生成 + +**Source repos:** MatPlotAgent, AI-Scientist-v2 (VLM feedback), AI-Scientist (plot.py) + +**What it does:** +- Generates publication-quality matplotlib/seaborn figures +- 3-module closed loop: Query expansion → Code generation → VLM visual feedback +- VLM (GPT-4V) evaluates figure quality: data representation, labels, colors, readability +- Automatic code retry on execution errors (up to 4 attempts) +- Professional styling with consistent color palettes + +**Key techniques:** +1. **Query expansion** (MatPlotAgent): Raw instruction → step-by-step coding instructions before code generation +2. **VLM feedback loop** (MatPlotAgent): Generated PNG + original query → GPT-4V → improvement instructions → regenerate code +3. **Plot aggregation** (AI-Scientist-v2): Multiple experiment `.npy` files → unified ~12 final figures +4. **Figure-caption alignment review** (AI-Scientist-v2): VLM checks each figure's caption accuracy and informativeness + +**Score improvement:** GPT-4 baseline 48.86 → with MatPlotAgent 61.16 (+12.3 points) + +--- + +### S14: table-generation — LaTeX 表格生成 + +**Source repos:** data-to-paper (displayitems stage), AI-Researcher (comparison tables) + +**What it does:** +- Converts experimental results (JSON/CSV/DataFrame) to publication-ready LaTeX tables +- Handles: `booktabs` styling, bold best results, multi-row/multi-column layouts +- Baseline comparison tables from prior work (scraped from papers) +- Ablation study tables with component-wise analysis + +**Prompt pattern:** +``` +Given experimental results in JSON format: +1. Design table layout (rows = methods, columns = metrics/datasets) +2. Generate LaTeX using booktabs package (\toprule, \midrule, \bottomrule) +3. Bold the best result in each column +4. Add proper captions and labels +5. Ensure all numbers match the actual experimental logs — do not hallucinate +``` + +--- + +### S15: paper-writing-section — 逐节论文撰写 + +**Source repos:** AI-Scientist (perform_writeup.py), AgentLaboratory (papersolver.py), data-to-paper (writing stages) + +**What it does:** +- Writes each paper section individually with section-specific guidance +- Two refinement passes per section (AI-Scientist): criticize → fix errors → compress +- Scaffold generation (AgentLaboratory): skeleton → fill section by section +- Word count enforcement (~4000 words for 8-page paper) + +**Per-section tips (from AI-Scientist + AgentLaboratory):** + +| Section | Key Guidance | +|---------|-------------| +| **Abstract** | TL;DR → why it's hard → contribution → how verified. Single paragraph only. | +| **Introduction** | Longer abstract; list contributions as bullet points. | +| **Background** | Problem setting with formal notation; define all symbols. | +| **Related Work** | Compare and contrast, not just describe. Organize by theme. | +| **Methods** | Precise mathematical equations; what we do AND why. | +| **Experimental Setup** | Datasets, metrics, hyperparameters. Don't hallucinate hardware details. | +| **Results** | Only report numbers from actual logs. Include ablations. Include all figures. | +| **Discussion/Conclusion** | Brief recap → limitations → future work as "academic offspring." | + +**Two-pass refinement (AI-Scientist):** +``` +Pass 1: Criticize the section. Fix: unenclosed math, broken refs, LaTeX errors, + hallucinated numbers, duplicate labels, verbosity. +Pass 2: Identify redundancies. Save space. Be more concise without weakening the message. +``` + +--- + +### S16: latex-formatting — LaTeX 格式、模板、编译 + +**Source repos:** AI-Scientist (template.tex), AgentLaboratory (papersolver.py auto-inject), data-to-paper (latex_to_pdf.py) + +**What it does:** +- Manages LaTeX templates for major venues (ICML, ICLR, NeurIPS, AAAI, ACL, ICBINB) +- Auto-injects required packages (amsmath, booktabs, hyperref, algorithm, etc.) +- Handles common formatting issues: overfull hboxes, missing labels, duplicate sections +- Conference-specific formatting rules + +**Auto-injected packages (from AgentLaboratory):** +``` +amsmath, amssymb, array, algorithm, algorithmicx, algpseudocode, +booktabs, colortbl, color, enumitem, fontawesome5, float, graphicx, +hyperref, listings, makecell, multicol, multirow, pgffor, pifont, +soul, sidecap, subcaption, titletoc, footmisc, url, wrapfig, xcolor, xspace +``` + +--- + +### S17: citation-management — BibTeX 引用管理 + +**Source repos:** AI-Scientist (perform_writeup.py citation harvesting), data-to-paper (literature_search.py) + +**What it does:** +- Iterative citation harvesting: LLM reads draft → identifies most needed citation → searches Semantic Scholar → selects from results → injects BibTeX +- Pre-compilation validation: every `\cite{key}` must exist in `.bib` +- Deduplication of citation entries +- Proper BibTeX formatting with all required fields + +**Citation loop (from AI-Scientist, 20 rounds):** +``` +Round N: +1. LLM reads current draft, identifies gap needing a citation +2. LLM generates a Semantic Scholar search query +3. API returns top-10 results with title/abstract/BibTeX +4. LLM selects the most relevant paper(s) +5. BibTeX appended to references.bib +6. LLM integrates \cite{key} into the paper text +``` + +--- + +### S18: self-review — 自动审稿 + +**Source repos:** AI-Scientist (perform_review.py), AgentLaboratory (ReviewersAgent), ChatReviewer + +**What it does:** +- Structured review using NeurIPS/ICLR review form +- Ensemble voting: 5 independent reviews → meta-review aggregation +- Three reviewer personas: harsh-fair, harsh-critical, open-minded +- Reflection loop: refine review up to 5 rounds +- Scores: Originality, Quality, Clarity, Significance, Soundness, Presentation, Contribution (1-4), Overall (1-10) + +**Review form fields:** +```json +{ + "Summary": "...", + "Strengths": ["..."], + "Weaknesses": ["..."], + "Originality": 3, + "Quality": 3, + "Clarity": 3, + "Significance": 3, + "Soundness": 3, + "Presentation": 3, + "Contribution": 3, + "Overall": 6, + "Confidence": 4, + "Decision": "Accept/Reject" +} +``` + +**Few-shot calibration (AI-Scientist):** Uses 3 real ICLR papers + reviews as reference: +- "Attention Is All You Need" (scored 8/10 Accept) +- Real ICLR submissions with known scores + +--- + +### S19: rebuttal-writing — Rebuttal 撰写 + +**Source repos:** ChatReviewer (chat_response.py) + +**What it does:** +- Takes reviewer comments as input +- Extracts concerns one by one +- Generates point-by-point responses +- Key instruction: "Reply with what we have done, not what we will do" +- Proper rebuttal formatting: Concern → Response per reviewer + +**Output format:** +``` +# Response to Reviewers + +## Reviewer #1 +**Concern #1:** [extracted concern] +**Author Response:** [detailed response with evidence] + +**Concern #2:** [extracted concern] +**Author Response:** [detailed response with evidence] + +## Reviewer #2 +... +``` + +--- + +### S20: paper-revision — 基于审稿意见修改论文 + +**Source repos:** AI-Scientist (perform_improvement), AgentLaboratory (report_refinement with second_round) + +**What it does:** +- Takes review feedback and current paper draft +- Maps each weakness/concern to specific sections needing changes +- Applies targeted edits preserving paper structure +- Re-compiles and re-reviews to verify improvement +- Supports iterative revision loops + +**Key technique (AgentLaboratory):** When PhD decides to revise: +1. Copy current state to `prev_*` fields +2. Reset all downstream phases (experiments, writing) +3. Re-run the pipeline with reviewer feedback injected as notes +4. Compare new scores vs previous scores + +--- + +### S21: paper-assembly — 论文全流程整合 + +**Source repos:** AI-Scientist (launch_scientist.py), AI-Researcher (main_ai_researcher.py), AgentLaboratory (ai_lab_repo.py) + +**What it does:** +- Orchestrates the entire paper pipeline end-to-end +- Manages state propagation between phases +- Handles checkpointing and resumption +- Integrates: literature → plan → code → experiments → figures → tables → writing → review + +**Orchestration patterns:** +1. **Sequential pipeline** (AI-Scientist): generate_ideas → experiments → writeup → review +2. **Multi-agent state broadcasting** (AgentLaboratory): `set_agent_attr()` propagates results to all agents +3. **FlowModule caching** (AI-Researcher): Cache each agent's output, support resume/replay +4. **Copilot mode checkpoints** (AgentLaboratory): Human can intervene at any phase boundary + +--- + +### S22: paper-compilation — LaTeX 编译与错误修复 + +**Source repos:** AI-Scientist (perform_writeup.py generate_latex), data-to-paper (latex_to_pdf.py) + +**What it does:** +- Full LaTeX compilation pipeline: `pdflatex → bibtex → pdflatex → pdflatex` +- Pre-compilation validation: + - All `\cite{key}` exist in `.bib` + - All `\includegraphics{file}` exist as `.png` + - No duplicate figure references + - No duplicate `\section{}` headers +- Error correction loop: `chktex` analysis → LLM fixes → recompile (up to 5 rounds) +- Handles common errors: unescaped underscores, HTML syntax in LaTeX, unclosed environments + +--- + +### S23: backward-traceability — 数据可追溯 + +**Source repos:** data-to-paper (the defining innovation) + +**What it does:** +- Every number in the final PDF hyperlinks back to the exact code line that produced it +- Uses `\hypertarget{label}{value}` and `\hyperlink{label}{value}` LaTeX commands +- `\num{formula, "explanation"}` evaluated at compile time via `eval()` +- Appendix contains full code listing with hyperlink anchors +- Calculation Notes section shows `formula = result` for each computed value + +**Implementation:** +1. Code output: Numbers tagged with `\hypertarget{R1a}{45.3}` +2. Paper text: Author writes `\num{\hyperlink{R1a}{45.3}, "mean age"}` +3. Compile time: `eval()` verifies formula, generates hyperlink +4. Appendix: Code listing with `\hypertarget` at relevant lines +5. Result: Click any number in PDF → jumps to code line that produced it + +--- + +### S24: survey-generation — 综述论文自动生成 + +**Source repos:** AutoSurvey, STORM + +**What it does:** +- Generates complete academic survey papers +- Parallel multi-LLM sub-section generation +- RAG-based real-time knowledge updates +- Multi-perspective coverage for comprehensive topic review + +--- + +### S25: paper-to-code — 论文 → 可运行代码仓库 + +**Source repos:** Paper2Code (PaperCoder) + +**What it does:** +- Takes an ML paper PDF and generates a complete, runnable code repository +- 3-stage pipeline: Planning (UML + dependency graph) → Analysis (per-file logic) → Coding (dependency-ordered generation) +- Dependency-ordered code generation: each file sees all previously generated files +- Mermaid diagrams as rigid interface contracts across all stages + +**Key metrics:** PaperBench 44.26% vs baseline 16.4%, 85% rated helpful, only 0.48% code lines need human modification + +--- + +### S26: slide-generation — 论文 → 演示幻灯片/Poster + +**Source repos:** (Gap identified in research — no dedicated tool exists) + +**What it does:** +- Converts a completed paper into presentation slides (Beamer/PowerPoint) +- Extracts key figures, tables, and equations +- Creates a narrative flow suitable for oral presentation +- Poster layout for conference poster sessions + +**Note:** This is an identified gap. This skill would be novel — to be designed from best practices. + +--- + +### S27: novelty-assessment — 研究新颖性评估 + +**Source repos:** AI-Scientist (check_idea_novelty), data-to-paper (assess_novelty stage), SciMON + +**What it does:** +- Takes a research idea and searches literature systematically +- Multi-round search-evaluate loop (up to 10 rounds) +- Harsh critic persona for novelty evaluation +- Binary decision: Novel / Not Novel with justification +- Tracks "most similar papers" for positioning + +**System prompt (from AI-Scientist):** +``` +Be a harsh critic for novelty. Ensure there is a sufficient contribution +for a new conference or workshop paper. You will be given access to the +Semantic Scholar API to survey the literature. +``` + +--- + +### S28: symbolic-equation — 科学方程发现 + +**Source repos:** LLM-SR + +**What it does:** +- Uses LLMs to discover scientific equations from data +- LLM-guided evolutionary search over symbolic expression space +- Combines neural reasoning with symbolic optimization +- Outputs interpretable mathematical relationships + +--- + +## Cross-Cutting Patterns Used Across Skills + +### Pattern 1: Reflection Convergence +Used in: S02, S15, S18 +``` +Loop until LLM says "I am done" or max_rounds: + 1. Generate output + 2. LLM critiques own output + 3. LLM refines output +``` + +### Pattern 2: Code-Execute-Fix Loop +Used in: S09, S10, S12, S13 +``` +Loop up to max_retries: + 1. LLM generates code + 2. Execute in subprocess + 3. If error: feed error back → LLM fixes + 4. If success: evaluate output quality +``` + +### Pattern 3: Multi-Source Search + Selection +Used in: S03, S17, S27 +``` +1. LLM generates search query +2. API returns top-k results +3. LLM selects most relevant +4. Repeat with refined queries +``` + +### Pattern 4: VLM Feedback Loop +Used in: S13, S11 +``` +1. Generate visual artifact (figure/plot) +2. Send image to VLM (GPT-4V) +3. VLM provides improvement instructions +4. Regenerate based on feedback +``` + +### Pattern 5: Ensemble + Meta-Aggregation +Used in: S18 +``` +1. Generate N independent outputs (e.g., 5 reviews) +2. Aggregate via meta-review prompt +3. Average numerical scores +``` + +--- + +## GitHub Repos Cloned → `/Users/lingzhi/Code/research-engine/github/` + +| Repo | Primary Skills | Stars | +|------|---------------|-------| +| AI-Scientist | S01, S02, S09, S15, S17, S18, S22 | 12.1k | +| AI-Scientist-v2 | S11, S13, S15, S21 | 2.1k | +| AI-Researcher | S01, S08, S09, S10, S11, S12 | 4.5k | +| AgentLaboratory | S01, S04, S09, S15, S18, S20, S21 | 5.3k | +| data-to-paper | S06, S10, S12, S14, S22, S23 | 756 | +| storm | S04, S05, S24 | 27.9k | +| AutoSurvey | S24 | 458 | +| gpt-researcher | S03, S04 | 25.4k | +| Paper2Code | S07, S25 | 4.2k | +| MatPlotAgent | S13 | 105 | +| ChatReviewer | S18, S19 | — | +| SciMON | S02, S27 | — | +| LitLLM | S05 | 21 | +| LLM-SR | S28 | — | +| MLR-Copilot | S01, S09, S11 | 67 | +| OpenResearcher | S03 | 492 | +| ReviewAdvisor | S18 | — | + +--- + +## Recommended Priority for Implementation + +### Tier 1 — Core (must-have, daily use) +1. **S15** paper-writing-section — 最常用，每篇论文都需要 +2. **S03** literature-search — 基础设施 +3. **S17** citation-management — 每篇论文都需要 +4. **S06** math-reasoning — 理论论文核心 +5. **S13** figure-generation — 每篇论文都需要 +6. **S16** latex-formatting — 基础设施 +7. **S22** paper-compilation — 基础设施 +8. **S18** self-review — 提交前必做 + +### Tier 2 — High Value (significant time savings) +9. **S01** research-planning — 大幅提升效率 +10. **S02** idea-generation — 创新起点 +11. **S09** experiment-code — 加速实现 +12. **S14** table-generation — 节省大量格式化时间 +13. **S05** related-work-writing — 耗时最多的部分之一 +14. **S04** literature-review — 深度文献综述 +15. **S19** rebuttal-writing — 审稿后必需 + +### Tier 3 — Advanced (specialized scenarios) +16. **S11** experiment-design — 完善实验设计 +17. **S12** data-analysis — 数据密集型研究 +18. **S10** code-debugging — 复杂实验调试 +19. **S20** paper-revision — 大修场景 +20. **S27** novelty-assessment — 提交前评估 +21. **S08** atomic-decomposition — 复杂系统论文 +22. **S07** algorithm-design — 算法论文 +23. **S21** paper-assembly — 端到端自动化 + +### Tier 4 — Extended (nice-to-have) +24. **S23** backward-traceability — 可重复性 +25. **S24** survey-generation — 综述论文 +26. **S25** paper-to-code — 复现工作 +27. **S26** slide-generation — 报告展示 +28. **S28** symbolic-equation — 特定领域 diff --git a/SKILLS_IMPLEMENTED.md b/SKILLS_IMPLEMENTED.md new file mode 100644 index 0000000..0ed2740 --- /dev/null +++ b/SKILLS_IMPLEMENTED.md @@ -0,0 +1,585 @@ +# Research Paper Writing: Implemented Claude Skills + +> Implementation record for 30 skills deployed at `~/.claude/skills/`. +> Companion to [SKILLS_DESIGN.md](./SKILLS_DESIGN.md) which contains the original design specifications. + +--- + +## Summary + +| Category | Count | With Scripts | Prompt-Only | +|----------|-------|-------------|-------------| +| Research Discovery & Planning | 6 | 4 | 2 | +| Method Design | 4 | 0 | 4 | +| Experiment Pipeline | 4 | 2 | 2 | +| Paper Writing | 4 | 1 | 3 | +| Figures, Tables & Citations | 4 | 4 | 0 | +| LaTeX & Compilation | 3 | 3 | 0 | +| Review & Polish | 5 | 3 | 2 | +| **Total** | **30** | **17** | **13** | + +Scripts total: 27 Python + 7 CJS (Excalidraw) across 15 skill directories. + +--- + +## Phase 0: Research Discovery & Planning + +### 1. `deep-research` — Systematic Literature Survey + +**Design ref:** Pre-dates SKILLS_DESIGN.md (not in S01-S28). Most script-heavy skill. + +**How it works:** +- **Prompt:** Orchestrates a 6-phase literature survey: Frontier (latest conferences) → Survey (35-80 papers) → Deep Dive (8-15 detailed reads) → Code & Tools (GitHub extraction) → Synthesis (cross-paper analysis) → Compilation (final report). +- **Scripts (7):** + - `search_semantic_scholar.py` — Semantic Scholar API search, returns JSONL with title/authors/year/venue/abstract/citations + - `search_arxiv.py` — arXiv API search via `arxiv` package + - `download_papers.py` — Download PDFs from URLs or arXiv IDs + - `extract_pdf.py` — Extract text from PDF using PyMuPDF + - `paper_db.py` — Merge, deduplicate, and manage paper databases (JSONL format) + - `bibtex_manager.py` — Convert JSONL paper records to BibTeX entries + - `compile_report.py` — Assemble final survey report from phase outputs + +**Usage pattern:** Scripts handle API calls, PDF processing, and data management. Prompt guides Claude through analysis, synthesis, and gap identification at each phase. + +--- + +### 2. `literature-search` — Academic Paper Search + +**Design ref:** S03 + +**How it works:** +- **Prompt:** Expands user query into 2-4 complementary searches, runs across 3+ APIs, merges and deduplicates results, ranks by citations (0.3) + recency (0.3) + venue quality (0.2) + relevance (0.2). +- **Scripts (4 own + shared from deep-research):** + - `search_crossref.py` — CrossRef API search with BibTeX generation, type mapping (article/inproceedings/book). Stdlib-only. *New.* + - `download_arxiv_source.py` — Search arXiv by title, download source tarball, extract .tex files. Stdlib-only (urllib + xml.etree). *New.* + - `search_openalex.py` — OpenAlex API with citation count and year filtering + - Also uses: `deep-research/scripts/search_semantic_scholar.py`, `search_arxiv.py`, `paper_db.py`, `bibtex_manager.py` + +**Usage pattern:** Scripts call search APIs and return structured JSONL. Prompt handles query expansion, result ranking, and relevance filtering. + +--- + +### 3. `literature-review` — Multi-Perspective Literature Review + +**Design ref:** S04 + +**How it works:** +- **Prompt:** Generates 3-5 expert personas from different research perspectives. Each persona conducts a multi-turn grounded Q&A conversation (3-5 turns). All conversations are synthesized into a unified knowledge base with inline citations. +- **Scripts:** Shares search scripts from `literature-search` and `deep-research`. + +**Usage pattern:** Prompt-driven role-play simulation. Scripts provide the search backbone for grounding expert answers in real literature. + +--- + +### 4. `idea-generation` — Research Idea Generation + +**Design ref:** S02 + +**How it works:** +- **Prompt:** Generates 3-5 diverse research ideas, each with Name/Title/Experiment plan. Iterative reflection (up to 5 rounds) to refine. Scores each idea on Interestingness, Feasibility, Novelty (1-10 scale). +- **Scripts (1):** + - `novelty_check.py` — Searches Semantic Scholar for similar work, evaluates overlap in multiple rounds + +**Usage pattern:** Prompt generates and refines ideas. Script validates novelty against existing literature. + +--- + +### 5. `novelty-assessment` — Research Novelty Evaluation + +**Design ref:** S27 + +**How it works:** +- **Prompt:** Adopts a harsh critic persona. Runs up to 10 rounds of search-evaluate loops. Final output is a binary Novel/Not Novel decision with justification. Identifies the most similar existing papers and explains differentiation. +- **Scripts:** Shares `idea-generation/scripts/novelty_check.py` and `deep-research/scripts/search_semantic_scholar.py`. + +**Usage pattern:** Prompt drives the adversarial evaluation. Scripts perform iterative literature searches. + +--- + +### 6. `research-planning` — Research Plan Design + +**Design ref:** S01 + +**How it works:** +- **Prompt:** 4-stage planning framework: Overall Plan → Architecture Design (UML) → Logic Design (task list) → Configuration. Outputs paper structure, section-by-section plan, dependency-ordered task graph, baselines, datasets, metrics, and risk flags. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude produces structured research plans as JSON or Markdown. + +--- + +## Phase 1: Method Design + +### 7. `atomic-decomposition` — Atomic Concept Decomposition + +**Design ref:** S08 + +**How it works:** +- **Prompt:** Decomposes a complex research idea into atomic, self-contained concepts. For each concept, performs a Paper Survey (extract math formula from papers) and Code Survey (find implementation in repos). Creates bidirectional mapping: `math_formula ↔ code_implementation`. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude reads papers and code to build a structured knowledge base of atomic definitions. + +--- + +### 8. `algorithm-design` — Algorithm Design & Pseudocode + +**Design ref:** S07 + +**How it works:** +- **Prompt:** Formalizes algorithms with clear input/output/parameters. Generates LaTeX pseudocode using `algorithm` + `algpseudocode` environments. Creates Mermaid UML class diagrams and sequence diagrams. Verifies consistency: every pseudocode step maps to a code module. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Outputs LaTeX algorithm environments and Mermaid diagram code. + +--- + +### 9. `math-reasoning` — Mathematical Reasoning + +**Design ref:** S06 + +**How it works:** +- **Prompt:** Six task modes: + - `derive` — Step-by-step equation derivation with justifications, boxed final result + - `prove` — Formal theorem proof using appropriate technique (induction, contradiction, construction) + - `formalize` — Problem setting formalization with variable definitions and assumptions + - `stats` — Statistical test selection and proper reporting + - `notation` — Generate notation table with all symbols used in the paper + - `verify` — Check mathematical correctness of existing derivations +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Outputs LaTeX math notation. + +--- + +### 10. `symbolic-equation` — Scientific Equation Discovery + +**Design ref:** S28 + +**How it works:** +- **Prompt:** Implements the LLM-SR framework: LLM-guided evolutionary search over symbolic expression space. Multi-island algorithm with softmax-based cluster sampling for diversity. Island reset mechanism prevents premature convergence. LLM proposes candidate equations, evaluates fitness against data, iteratively improves. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude generates and evaluates candidate equations in an evolutionary loop. + +--- + +## Phase 2: Experiment Pipeline + +### 11. `experiment-design` — Experiment Plan Design + +**Design ref:** S11 + +**How it works:** +- **Prompt:** 4-stage progressive framework (from AI-Scientist-v2): + - Stage 1: Initial Implementation — basic working baseline on simple dataset + - Stage 2: Baseline Tuning — hyperparameters on 2+ datasets, no architecture changes + - Stage 3: Creative Research — novel improvements on 3+ datasets + - Stage 4: Ablation Studies — systematic component analysis +- **Scripts (1):** + - `design_experiments.py` — Generates experiment design JSON/Markdown: baseline list, ablation matrix, hyperparameter grid, metric selection by task type (classification/regression/generation/detection/segmentation/retrieval), total run estimation. Stdlib-only. *New.* + +**Usage pattern:** Script generates the structural experiment plan. Prompt fills in domain-specific details and rationale. + +--- + +### 12. `experiment-code` — Experiment Code Writing + +**Design ref:** S09 + +**How it works:** +- **Prompt:** Three actions: + - `generate` — Create complete training/evaluation pipeline with logging and figure generation + - `improve` — Read results, reflect on what worked, apply targeted edits, re-run and compare + - `debug` — Identify root cause, apply minimal fix, up to 4 retries +- Enforces constraints: no placeholder code, must use real datasets, must generate figures, PyTorch/scikit-learn only. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude writes, executes, and iteratively improves experiment code. + +--- + +### 13. `code-debugging` — Code Debugging + +**Design ref:** S10 + +**How it works:** +- **Prompt:** Structured error analysis pipeline: + 1. Categorize error (SyntaxError, ImportError, RuntimeError, TimeoutError, OutputError, LogicError) + 2. Analyze root cause from traceback + 3. Apply targeted fix (up to 4 retries) + 4. Reflect: explain error, identify lines, describe fix, note patterns to avoid +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude follows the categorize→analyze→fix→reflect workflow. + +--- + +### 14. `data-analysis` — Statistical Data Analysis + +**Design ref:** S12 + +**How it works:** +- **Prompt:** Generates analysis code in 7 sections (IMPORT → LOAD DATA → DATASET PREPARATIONS → DESCRIPTIVE STATISTICS → PREPROCESSING → ANALYSIS → SAVE). Then runs a 4-round code review: Round 1 (code flaws) → Round 2 (data handling) → Round 3 (per-table) → Round 4 (cross-table completeness). Statistical test selection table guides appropriate test choice. +- **Scripts (2):** + - `stat_summary.py` — Loads CSV/JSON, detects data types, recommends statistical tests (t-test, Mann-Whitney, Wilcoxon, ANOVA, Kruskal-Wallis), computes effect sizes (Cohen's d), outputs significance stars. Requires numpy + scipy. *New.* + - `format_pvalue.py` — Formats p-values as text, significance stars (`*`/`**`/`***`/`ns`), LaTeX notation, or JSON. Supports batch processing from CLI values, CSV, or stdin. Stdlib-only. *New.* + +**Usage pattern:** Scripts handle statistical computation and formatting. Prompt performs the 4-round review and generates the full analysis code. + +--- + +## Phase 3: Paper Writing + +### 15. `paper-writing-section` — Section-by-Section Writing + +**Design ref:** S15 + +**How it works:** +- **Prompt:** Writes each paper section with section-specific guidance: + + | Section | Guidance | + |---------|----------| + | Abstract | TL;DR → why hard → contribution → how verified. Single paragraph. | + | Introduction | Longer abstract; list contributions as bullet points. | + | Background | Problem setting with formal notation; define all symbols. | + | Related Work | Compare and contrast, not just describe. Organize by theme. | + | Methods | Precise equations; what we do AND why. | + | Experiments | Datasets, metrics, hyperparameters. No hallucinated numbers. | + | Results | Only from actual logs. Include ablations and all figures. | + | Discussion | Brief recap → limitations → future work. | + + Two-pass refinement: Pass 1 (fix errors, unenclosed math, broken refs, hallucinated numbers) → Pass 2 (remove redundancies, compress, smooth transitions). +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude reads existing materials, writes the section, then self-refines in two passes. + +--- + +### 16. `related-work-writing` — Related Work Section + +**Design ref:** S05 + +**How it works:** +- **Prompt:** 4-step process: + 1. Analyze paper's key contributions and novelty claims + 2. Organize literature into thematic clusters + 3. Write each theme paragraph: topic sentence → describe key works → compare/contrast with this paper + 4. Refine: verify citation reasons are clear, novelty is explicit, all cite keys resolve +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Focused specifically on the Related Work section with compare-and-contrast emphasis. + +--- + +### 17. `survey-generation` — Survey Paper Generation + +**Design ref:** S24 + +**How it works:** +- **Prompt:** AutoSurvey pipeline: + 1. Collect 50-200 papers via Semantic Scholar/arXiv + 2. Generate N outlines in parallel, merge best elements + 3. RAG-based subsection writing: retrieve relevant papers per subsection, generate with inline citations + 4. Validate citations: check titles match, verify claims are supported + 5. Enhance local coherence: read adjacent sections, refine transitions + 6. Convert paper title citations to BibTeX `\cite{key}` format +- **Scripts:** Shares `deep-research/scripts/search_semantic_scholar.py`. + +**Usage pattern:** Prompt orchestrates the multi-step RAG pipeline. Script provides the search backend. + +--- + +### 18. `paper-to-code` — Paper to Runnable Code + +**Design ref:** S25 + +**How it works:** +- **Prompt:** Paper2Code 3-stage pipeline: + 1. Planning — Overall plan, architecture design (UML), task breakdown, configuration extraction + 2. Analysis — Per-file detailed logic analysis + 3. Coding — Dependency-ordered code generation, each file sees all previously generated files + 4. Debugging — If execution fails, identify root cause and apply fixes +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude reads the paper, designs the architecture, then generates code file by file in dependency order. + +--- + +## Phase 4: Figures, Tables & Citations + +### 19. `figure-generation` — Scientific Figure Generation + +**Design ref:** S13 + +**How it works:** +- **Prompt:** 3-phase pipeline: + 1. Query Expansion — expand user description into step-by-step coding specifications + 2. Code Generation with Execution Loop — generate matplotlib/seaborn script, execute, fix errors (up to 4 retries) + 3. Visual Refinement — read generated PNG, inspect using VLM feedback, generate corrections +- **Scripts (1):** + - `figure_template.py` — Generates starter matplotlib code for 10 figure types: `bar`, `line`, `heatmap`, `scatter`, `training_curve`, `ablation`, `radar`, `violin`, `tsne`, `attention`. The last 4 templates were added in the upgrade. *Updated.* + +**Usage pattern:** Script generates code scaffolds for common figure types. Prompt handles the iterative refinement loop with VLM feedback. + +--- + +### 20. `table-generation` — LaTeX Table Generation + +**Design ref:** S14 + +**How it works:** +- **Prompt:** Converts JSON/CSV experiment results to publication-ready LaTeX tables with `booktabs` styling, bold best results, proper captions and labels. Supports comparison, ablation, descriptive, and multi-dataset table types. +- **Scripts (1):** + - `results_to_table.py` — Converts JSON/CSV data to LaTeX. 4 table types: `comparison` (methods × metrics), `ablation` (variants × metrics), `descriptive` (dataset stats), `multi-dataset` (methods × datasets × metrics). Flags: `--bold-best`, `--significance` (p-value stars), `--underline-second` (second-best results). *Updated.* + +**Usage pattern:** Script handles the mechanical conversion. Prompt decides table layout and which results to include. + +--- + +### 21. `citation-management` — BibTeX Citation Management + +**Design ref:** S17 + +**How it works:** +- **Prompt:** Four actions: + - `harvest` — Iterative citation harvesting: scan draft for uncited claims, search Semantic Scholar, add candidate BibTeX entries (up to 20 rounds) + - `validate` — Pre-compilation check: every `\cite{key}` must exist in `.bib`, every `\includegraphics` must exist + - `add` — Add a specific paper by title or DOI + - `format` — Standardize and deduplicate `.bib` file +- **Scripts (2 own + shared):** + - `harvest_citations.py` — Scans .tex for sentences lacking `\cite`, searches Semantic Scholar API, outputs candidate BibTeX. Stdlib-only. *New.* + - `validate_citations.py` — Checks cite keys vs .bib, label vs ref consistency, figure file existence. `--fix` mode generates placeholder entries for missing keys. *Updated.* + - Also uses: `deep-research/scripts/bibtex_manager.py`, `search_semantic_scholar.py` + +**Usage pattern:** Scripts automate search and validation. Prompt handles citation selection and integration into paper text. + +--- + +### 22. `backward-traceability` — Numeric Value Traceability + +**Design ref:** S23 + +**How it works:** +- **Prompt:** Ensures every number in the final PDF traces to the exact code line that produced it. Workflow: + 1. Tag code outputs with `\hypertarget{label}{value}` + 2. Reference in paper with `\hyperlink{label}{value}` + 3. Use `\num{formula}` for derived values (compile-time evaluation) + 4. Generate appendix code listing with hypertarget anchors + 5. Verify all hyperlinks resolve correctly +- **Scripts (1):** + - `ref_numeric_values.py` — Two modes: `--scan` (report all hypertarget/hyperlink usage, orphan references, unreferenced numbers) and `--verify` (cross-reference integrity between .tex and code output, value mismatch detection). Stdlib-only. *New.* + +**Usage pattern:** Script scans and verifies traceability. Prompt guides how to add hypertarget/hyperlink tags. + +--- + +## Phase 5: LaTeX & Compilation + +### 23. `latex-formatting` — LaTeX Formatting & Templates + +**Design ref:** S16 + +**How it works:** +- **Prompt:** Three actions: + - `setup` — Create project directory with conference template (ICML, ICLR, NeurIPS, AAAI, ACL) + - `fix` — Fix common LaTeX issues: unescaped characters, math mode errors, float placement, cross-references + - `check` — Pre-submission validation: word count, section structure, anonymization, citation consistency +- **Scripts (2):** + - `latex_checker.py` — Checks word count, section completeness (flags missing expected sections), citation/figure/equation counts, venue-specific rules, anonymization. `--fix` mode calls `clean_latex.py` after checking. *Updated.* + - `clean_latex.py` — Replaces special characters with LaTeX equivalents (28 special chars + 20 non-UTF8 chars). Skips math environments, comments, command definitions, tabular environments, and LaTeX commands. Stdlib-only. *New.* + +**Usage pattern:** Scripts handle automated checking and cleaning. Prompt does venue-specific template setup and complex formatting fixes. + +--- + +### 24. `paper-compilation` — LaTeX Compilation + +**Design ref:** S22 + +**How it works:** +- **Prompt:** Full compilation pipeline: pdflatex → bibtex → pdflatex → pdflatex. Pre-compilation validation, up to 5 rounds of error correction, post-compilation report (page count, warnings, style issues). +- **Scripts (2):** + - `compile_paper.py` — Runs the full pdflatex+bibtex pipeline, optional `chktex` style checking. `--auto-fix` flag runs `fix_latex_errors.py` + recompile up to 3 rounds automatically. *Updated.* + - `fix_latex_errors.py` — Parses pdflatex `.log` files, classifies errors (undefined commands, missing math mode, mismatched environments, missing files), applies automated fixes: HTML tag conversion, environment balancing, missing figure commenting. `--dry-run` mode for preview. Stdlib-only. *New.* + +**Usage pattern:** Scripts automate the compile-fix-recompile cycle. Prompt handles complex errors that require understanding paper content. + +--- + +### 25. `excalidraw-skill` — Excalidraw Diagramming + +**Design ref:** Not in S01-S28. Uses MCP tools, not traditional scripts. + +**How it works:** +- **Prompt:** Programmatic canvas control via MCP Server tools. Mandatory quality gate after every diagram: check text truncation, element overlap, arrow crossing, spacing, and readability. Workflows include: Draw (plan grid → create elements → bind arrows → verify), Iterative Refinement (screenshot → evaluate → fix), File I/O (export/import .excalidraw), and Sharing (export to excalidraw.com URL). +- **Scripts (7 CJS):** MCP server implementation files (not called directly by users). + +**Usage pattern:** Claude calls MCP tools (`create_element`, `batch_create_elements`, `describe_scene`, `get_canvas_screenshot`) to manipulate a live Excalidraw canvas in the browser. + +--- + +## Phase 6: Review & Polish + +### 26. `self-review` — Automated Paper Review + +**Design ref:** S18 + +**How it works:** +- **Prompt:** Simulates peer review using the NeurIPS review form. Three independent reviewer personas (harsh-fair, harsh-critical, open-minded) each produce a full review. Reflection refinement loop (up to 3 rounds). Reviews are aggregated into a meta-review with averaged scores. Scores: Originality, Quality, Clarity, Significance, Soundness, Presentation, Contribution (1-4), Overall (1-10). +- **Scripts (2):** + - `extract_pdf_text.py` — Extracts raw text from PDF, outputs as plain text or markdown + - `parse_pdf_sections.py` — Parses PDF into structured sections using PyMuPDF font-size analysis. Detects title (largest font), headings (ALL CAPS or larger font), and body text. Outputs `{title, pages, sections: [{name, text, page}]}`. Requires pymupdf. *New.* + +**Usage pattern:** Scripts extract paper content from PDF. Prompt runs 3 independent reviews, refines, and aggregates. + +--- + +### 27. `paper-revision` — Paper Revision from Reviews + +**Design ref:** S20 + +**How it works:** +- **Prompt:** 5-step process: + 1. Parse reviewer concerns — extract, classify (major/minor), map to specific paper sections, prioritize + 2. Plan revisions — create mapping: Concern → Section → Action → New Content + 3. Execute revisions — read section, apply edits, run additional experiments if needed, mark changes with `\revised{}` + 4. Verify improvements — re-run self-review, check all concerns addressed, check page count + 5. Write revision summary — list all changes with cross-references to reviewer concerns +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Claude systematically addresses each reviewer concern. + +--- + +### 28. `rebuttal-writing` — Rebuttal Writing + +**Design ref:** S19 + +**How it works:** +- **Prompt:** Extracts reviewer concerns one by one. For each concern, generates a response following the pattern: Acknowledge → Respond with evidence → Describe what was done (not what will be done). Outputs formatted rebuttal with `## Reviewer #N` / `**Concern #N**` / `**Author Response**` structure. +- **Scripts:** None. + +**Usage pattern:** Purely prompt-driven. Emphasis on evidence-based responses to specific concerns. + +--- + +### 29. `slide-generation` — Presentation Slide Generation + +**Design ref:** S26 + +**How it works:** +- **Prompt:** Converts a completed paper into Beamer LaTeX slides or poster. Standard flow: extract key content → design 15-20 slide structure → generate Beamer code → simplify for presentation (max 1 key message per slide, max 6 bullets, large figures). Optional poster layout with 4-column structure. +- **Scripts (1):** + - `extract_paper_elements.py` — Parses .tex (resolves `\input{}` directives), extracts title, authors, abstract, sections (with hierarchy), figures (path + caption + label), equations, and tables. Generates complete Beamer skeleton or raw JSON. Supports theme selection. Stdlib-only. *New.* + +**Usage pattern:** Script extracts paper structure and generates slide skeleton. Prompt fills in content and optimizes the narrative flow. + +--- + +### 30. `paper-assembly` — End-to-End Paper Orchestrator + +**Design ref:** S21 + +**How it works:** +- **Prompt:** Manages the full 9-phase paper pipeline: literature → planning → code → results → figures → tables → bibliography → sections → compilation. Supports checkpointing after each phase and resumption. Quality gates verify outputs before proceeding. State propagation passes results to downstream phases. +- **Scripts (1):** + - `assembly_checker.py` — Scans a paper project directory, checks completeness of all 9 pipeline phases, analyzes .tex section coverage (abstract, introduction, method, experiment, conclusion), verifies citation cross-references, reports missing artifacts, suggests next steps with skill recommendations. Stdlib-only. *New.* + +**Usage pattern:** Script assesses current pipeline state. Prompt orchestrates calls to other skills in dependency order. + +--- + +## Skill Dependency Graph + +Every SKILL.md includes a `## Related Skills` section linking upstream, downstream, and complementary skills. The full graph: + +``` +deep-research ──→ literature-search ──→ citation-management ──→ paper-compilation + literature-review related-work-writing latex-formatting + novelty-assessment survey-generation + +idea-generation ──→ research-planning ──→ experiment-design ──→ experiment-code + atomic-decomposition code-debugging + +experiment-code ──→ data-analysis ──→ figure-generation ──→ paper-writing-section + table-generation related-work-writing + backward-traceability + +paper-writing-section ──→ latex-formatting ──→ paper-compilation ──→ self-review +citation-management paper-revision + rebuttal-writing + +paper-assembly (orchestrator) ──→ all skills above ──→ slide-generation +``` + +--- + +## Scripts Inventory + +### New Scripts (12) + +| Script | Skill | Lines | Dependencies | Source | +|--------|-------|-------|-------------|--------| +| `search_crossref.py` | literature-search | ~260 | stdlib | data-to-paper crossref.py | +| `download_arxiv_source.py` | literature-search | ~230 | stdlib | AI-Researcher arxiv.py | +| `harvest_citations.py` | citation-management | ~245 | stdlib | AI-Scientist citation loop pattern | +| `clean_latex.py` | latex-formatting | ~240 | stdlib | data-to-paper clean_latex.py | +| `fix_latex_errors.py` | paper-compilation | ~305 | stdlib | data-to-paper + AI-Scientist patterns | +| `parse_pdf_sections.py` | self-review | ~260 | pymupdf | ChatReviewer get_paper_from_pdf.py | +| `ref_numeric_values.py` | backward-traceability | ~265 | stdlib | data-to-paper ref_numeric_values.py | +| `stat_summary.py` | data-analysis | ~320 | numpy, scipy | data-to-paper 4-round review pattern | +| `format_pvalue.py` | data-analysis | ~145 | stdlib | data-to-paper pvalue.py | +| `design_experiments.py` | experiment-design | ~275 | stdlib | AI-Scientist-v2 4-stage pattern | +| `assembly_checker.py` | paper-assembly | ~290 | stdlib | New | +| `extract_paper_elements.py` | slide-generation | ~270 | stdlib | New | + +### Updated Scripts (5) + +| Script | Skill | Change | +|--------|-------|--------| +| `validate_citations.py` | citation-management | Added `--fix` auto-fix mode | +| `compile_paper.py` | paper-compilation | Added `--auto-fix` flag (fix + recompile 3 rounds) | +| `latex_checker.py` | latex-formatting | Added `--fix` flag (calls clean_latex.py) | +| `figure_template.py` | figure-generation | Added 4 templates: radar, violin, tsne, attention | +| `results_to_table.py` | table-generation | Added multi-dataset type, --significance, --underline-second | + +### Pre-existing Scripts (17) + +| Skill | Scripts | +|-------|---------| +| deep-research | search_semantic_scholar.py, search_arxiv.py, download_papers.py, extract_pdf.py, paper_db.py, bibtex_manager.py, compile_report.py | +| literature-search | search_openalex.py | +| citation-management | validate_citations.py (pre-upgrade) | +| figure-generation | figure_template.py (pre-upgrade) | +| idea-generation | novelty_check.py | +| latex-formatting | latex_checker.py (pre-upgrade) | +| paper-compilation | compile_paper.py (pre-upgrade) | +| self-review | extract_pdf_text.py | +| table-generation | results_to_table.py (pre-upgrade) | +| excalidraw-skill | 7 CJS files (MCP server) | + +--- + +## Verification Results + +Tested against real paper: RIGID metamaterials (OpenResearcher/2401.00003, with main.tex + references.bib + 14 figures). + +| Test | Script | Result | +|------|--------|--------| +| CrossRef API search | `search_crossref.py --query "attention mechanism transformer" --rows 3` | PASS — 3 results | +| LaTeX error fixer | `fix_latex_errors.py --tex main.tex --dry-run` | PASS — no fixes needed | +| PDF section parser | `parse_pdf_sections.py --pdf report.pdf --format json` | PASS — title + 7 sections | +| Citation validator | `validate_citations.py --tex main.tex --bib references.bib` | PASS — 71 citations, 7 unused entries | +| Beamer skeleton | `extract_paper_elements.py --tex main.tex --format beamer` | PASS — 19 sections, 12 figures, 3 equations | +| Assembly checker | `assembly_checker.py --dir paper/ --verbose` | PASS — 2/9 phases complete | +| LaTeX cleaner | `clean_latex.py --input main.tex --dry-run` | PASS — 2 legitimate changes | +| LaTeX checker | `latex_checker.py main.tex` | PASS — 7515 words, 12 sections | +| Traceability scan | `ref_numeric_values.py --scan main.tex` | PASS — 0 targets (paper has none) | +| P-value formatter | `format_pvalue.py --values "0.0001 0.003 0.012 0.048 0.067 0.5" --format latex` | PASS | +| Experiment design | `design_experiments.py --method "inverse design" --task classification` | PASS | +| Element extraction | `extract_paper_elements.py --tex main.tex --format json` | PASS | + +Bugs found and fixed during verification: +1. `parse_pdf_sections.py`: `len(doc)` called after `doc.close()` — fixed by saving page count before closing +2. `clean_latex.py`: Escaped `%` in comments and `#` in `\newcommand` — fixed by adding comment/command-definition skip patterns; removed `~` from escape chars (it's a valid LaTeX tie); added `tabular`/`array` to skip environments