mirror of
https://github.com/Orchestra-Research/AI-Research-SKILLs.git
synced 2026-09-19 05:04:39 +08:00
Trigger sync for new skills (verl, slime, miles, torchforge, torchtitan, ml-paper-writing)
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
@@ -355,3 +355,4 @@ Create seed checkpoint first (see Workflow 4, Step 1).
|
||||
- Paper: https://arxiv.org/abs/2410.06511
|
||||
- ICLR 2025: https://iclr.cc/virtual/2025/poster/29620
|
||||
- PyTorch Forum: https://discuss.pytorch.org/c/distributed/torchtitan/44
|
||||
|
||||
|
||||
@@ -312,3 +312,4 @@ See [slime API Reference](../slime/references/api-reference.md) for the complete
|
||||
- **Introduction Blog**: https://lmsys.org/blog/2025-11-19-miles/
|
||||
- **Slime (upstream)**: https://github.com/THUDM/slime
|
||||
- **SGLang**: https://github.com/sgl-project/sglang
|
||||
|
||||
|
||||
@@ -461,3 +461,4 @@ class CustomRewardModel:
|
||||
- **GitHub**: https://github.com/THUDM/slime
|
||||
- **Blog**: https://lmsys.org/blog/2025-07-09-slime/
|
||||
- **Examples**: See `examples/` directory for 14+ worked examples
|
||||
|
||||
|
||||
@@ -430,3 +430,4 @@ training:
|
||||
- **Discord**: https://discord.gg/YsTYBh6PD9
|
||||
- **TorchTitan**: https://github.com/pytorch/torchtitan
|
||||
- **Monarch**: https://github.com/meta-pytorch/monarch
|
||||
|
||||
|
||||
@@ -388,3 +388,4 @@ actor_rollout_ref:
|
||||
- **GitHub**: https://github.com/volcengine/verl
|
||||
- **Recipes**: https://github.com/verl-project/verl-recipe (DAPO, GSPO, etc.)
|
||||
- **Community**: Slack at verl-project
|
||||
|
||||
|
||||
@@ -934,3 +934,4 @@ See [templates/README.md](templates/README.md) for detailed setup instructions.
|
||||
**APIs:** [Semantic Scholar](https://api.semanticscholar.org/api-docs/) | [CrossRef](https://www.crossref.org/documentation/retrieve-metadata/rest-api/) | [arXiv](https://info.arxiv.org/help/api/basics.html)
|
||||
|
||||
**Venues:** [NeurIPS](https://neurips.cc/Conferences/2025/PaperInformation/StyleFiles) | [ICML](https://icml.cc/Conferences/2025/AuthorInstructions) | [ICLR](https://iclr.cc/Conferences/2026/AuthorGuide) | [ACL](https://github.com/acl-org/acl-style-files)
|
||||
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
# The Complete Guide to Writing Top-Quality ML Academic Papers
|
||||
|
||||
Writing successful ML papers for venues like NeurIPS, ICML, and ICLR demands mastery of a specific craft: translating rigorous technical work into a compelling narrative that busy reviewers can quickly evaluate. **The single most critical insight across all expert sources: your paper is not a collection of experiments—it's a story with one clear contribution supported by evidence.** This guide synthesizes advice from prominent researchers including Neel Nanda, Andrej Karpathy, Jacob Steinhardt, and Sebastian Farquhar, alongside official conference guidelines and practical tools for citation management.
|
||||
|
||||
The stakes are high: top ML conferences maintain **~25% acceptance rates**, and reviewers spend limited time per paper. Seminal work like Adam and Knowledge Distillation faced initial rejections. Success requires not just strong research but strategic communication—front-loading value, maintaining precision, and providing reproducibility details that build reviewer confidence.
|
||||
|
||||
---
|
||||
|
||||
## The narrative principle that separates accepted papers
|
||||
|
||||
Every successful ML paper centers on what Neel Nanda calls "the narrative": a short, rigorous, evidence-based technical story with a takeaway readers care about. This narrative rests on three pillars that must be crystal clear by the end of your introduction.
|
||||
|
||||
**The "What"** consists of one to three specific novel claims fitting within a cohesive theme. Vague contributions like "we study X" fail immediately—reviewers need precise, falsifiable claims. **The "Why"** provides rigorous empirical evidence that convincingly supports those claims, including strong baselines honestly tuned and experiments that distinguish between competing hypotheses rather than merely showing "decent results." **The "So What"** answers why readers should care, connecting your contribution to problems the community recognizes as important.
|
||||
|
||||
Andrej Karpathy reinforces this: "A paper is not a random collection of experiments you report on. The paper sells a single thing that was not obvious or present before. The entire paper is organized around this core contribution with surgical precision." This applies whether you're presenting a new architecture, a theoretical result, or improved understanding of existing methods—NeurIPS explicitly notes that "originality does not necessarily require an entirely new method."
|
||||
|
||||
The practical implication is severe: if you cannot state your contribution in one sentence, you don't yet have a paper. Everything else—experiments, related work, discussion—exists only to support that core claim.
|
||||
|
||||
---
|
||||
|
||||
## Front-load value: the title-to-methods pipeline
|
||||
|
||||
Readers encounter your paper in a predictable pattern: title → abstract → introduction → figures → maybe the rest. Nanda advises spending "about the same amount of time on each of: the abstract, the intro, the figures, and everything else." This isn't hyperbole—most reviewers form preliminary judgments before reaching your methods section.
|
||||
|
||||
**The abstract** follows a tight five-sentence structure perfected by Sebastian Farquhar: (1) What you achieved ("We introduce...", "We prove...", "We demonstrate..."), (2) Why this is hard and important, (3) How you do it with specialist keywords, (4-5) What evidence you have, including your most remarkable number. Generic openings like "Large language models have achieved remarkable success" waste precious space—Zachary Lipton's rule: "If the first sentence can be pre-pended to any ML paper, delete it."
|
||||
|
||||
**The introduction** should not exceed 1-1.5 pages and must include a bullet-point contribution list of 2-4 items (max 1-2 lines each in two-column format). Farquhar emphasizes: "Methods should start by page 2-3 maximum"—if your introduction runs longer, you're burying the actual contribution.
|
||||
|
||||
**Figure 1** deserves special attention because many readers skip directly to it. It should convey your core idea, approach, or most compelling result. Use vector graphics, ensure readability in black-and-white, and write captions that stand alone without requiring the main text.
|
||||
|
||||
---
|
||||
|
||||
## Section-by-section execution matters
|
||||
|
||||
**Related Work** should be organized methodologically, not paper-by-paper. Good: "One line of work uses Floogledoodle's assumption [refs] whereas we use Doobersnoddle's assumption because..." Bad: "Snap et al. introduced X while Crackle et al. introduced Y." Cite generously—reviewers likely authored relevant papers—and distribute citations throughout your paper rather than confining them to one section.
|
||||
|
||||
**Methods** must enable reimplementation. ICML's checklist requires: conceptual outline or pseudocode, clearly stated algorithms, all hyperparameters listed, and architectural details sufficient for reproduction. Present your final design decisions here; comparative ablations belong in experiments or appendix.
|
||||
|
||||
**Experiments** require explicit structure. For each experiment, state: what claim it supports, how it connects to your main contribution, the experimental setting (with details in appendix), and explicit guidance on what to observe in figures ("the blue line shows X, which demonstrates Y"). The ICML checklist mandates: error bars with methodology specified (standard deviation vs. standard error), hyperparameter search ranges, compute infrastructure (GPU type, total hours), and seed-setting methods.
|
||||
|
||||
**Limitations** deserve their own section—NeurIPS and ICML require this. Counter-intuitively, honesty helps: reviewers are explicitly instructed not to penalize papers for acknowledging limitations. Pre-empt criticisms by identifying weaknesses before reviewers do and explaining why they don't undermine your core claims.
|
||||
|
||||
---
|
||||
|
||||
## Writing style that signals quality
|
||||
|
||||
Gopen and Swan's "Science of Scientific Writing" establishes principles that ML papers routinely violate. **Place emphasis at sentence ends** (the "stress position"): readers naturally weight final words more heavily. **Put context first**: establish familiar information before introducing new concepts. **Keep subject and verb close together**: anything intervening reads as interruption. **One unit, one function**: each paragraph should make exactly one point.
|
||||
|
||||
Specific style rules from multiple sources converge on these practices. Minimize pronouns—if you must use "this" or "those," use them as adjectives ("this result") to provide clarity. Place verbs early in sentences for easier parsing. Use minimal-syllable words. Eliminate hedging unless genuine uncertainty exists—"may" and "can" should almost always be dropped. Lipton notes that "provides *very* tight approximation" drips with insecurity compared to "provides tight approximation."
|
||||
|
||||
Jacob Steinhardt emphasizes precision over brevity: replace "performance" with "accuracy" or "speed" depending on meaning. Use consistent phrasing—referring to the same concept with different terms creates confusion. Avoid vocabulary that signals incremental work: never "combine," "modify," or "expand"; instead "develop" or "propose."
|
||||
|
||||
For mathematical writing, state all assumptions formally, provide intuitive explanations alongside proofs, and use consistent notation. Ethan Perez's practical tip: unfold apostrophes ("X's Y" → "The Y of X") for clarity.
|
||||
|
||||
---
|
||||
|
||||
## Tables, figures, and visual communication
|
||||
|
||||
**Tables** should use the booktabs LaTeX package for professional appearance—avoid vertical lines, use horizontal rules sparingly. Bold the best value per metric and include symbols indicating direction (↑ higher is better, ↓ lower is better). Right-align numerical columns and maintain consistent decimal precision across all values.
|
||||
|
||||
**Figures** must be vector graphics (PDF, EPS) for plots and diagrams; raster formats (PNG at 600 DPI) only for photographs or dense visualizations. Critical accessibility requirement: **8% of men have color vision deficiency**. Use the Okabe-Ito or Paul Tol palettes, avoid red-green combinations, and verify your figures work in grayscale. The SciencePlots Python package provides publication-ready styles with a single line: `plt.style.use(['science', 'ieee'])`.
|
||||
|
||||
**Architecture diagrams** benefit from TikZ via PlotNeuralNet (GitHub: HarisIqbal88/PlotNeuralNet), which generates LaTeX code from Python. For training visualizations, include shaded regions showing variance across runs and use log scale when values span multiple orders of magnitude.
|
||||
|
||||
**Captions** should be self-contained—readers must understand figures without consulting main text. ICML explicitly states: "Do not include a title inside the figure; the caption should serve this function."
|
||||
|
||||
---
|
||||
|
||||
## Conference requirements every submission must meet
|
||||
|
||||
**NeurIPS 2025**: 9 content pages plus unlimited references; mandatory paper checklist covering reproducibility, ethics, and societal impact (desk rejection if missing); 6-point scoring system; lay summaries required for accepted papers. Reviews of accepted papers become public.
|
||||
|
||||
**ICML 2025**: 8 content pages plus one additional page allowed for camera-ready; Broader Impact Statement required at end before references (doesn't count toward limit); reciprocal reviewing requirement—all submissions need a designated reviewer from authors.
|
||||
|
||||
**ICLR 2025-2026**: 10 pages plus unlimited appendices; double-blind via OpenReview; new LLM policy requiring disclosure of AI use in writing (violations result in desk rejection).
|
||||
|
||||
All three conferences evaluate papers on four core dimensions: **quality** (technical soundness, well-supported claims), **clarity** (clear writing, reproducible by experts), **significance** (community impact, advances understanding), and **originality** (new insights, clear differentiation from prior work). Reviewers separate concerns into major issues (essential for publication) and minor issues (not essential), and strong reviews follow Daniel Dennett's rules: first re-express the position fairly, then list agreements and what you learned, only then critique.
|
||||
|
||||
---
|
||||
|
||||
## Citation APIs that prevent hallucination
|
||||
|
||||
AI-generated citations have a documented **~40% error rate**, including fabricated papers with real author names and fake titles. A reliable workflow requires programmatic verification through multiple APIs.
|
||||
|
||||
**Semantic Scholar** (api.semanticscholar.org) covers 214M papers with 2.49B citations. Rate limit: 1 RPS with free API key. Python library: `pip install semanticscholar`. Search, retrieve metadata, and access citation graphs—ideal for ML papers specifically.
|
||||
|
||||
**CrossRef** (api.crossref.org) is the primary source for DOI metadata and offers direct BibTeX retrieval via content negotiation:
|
||||
```python
|
||||
import requests
|
||||
def doi_to_bibtex(doi):
|
||||
return requests.get(f"https://doi.org/{doi}",
|
||||
headers={"Accept": "application/x-bibtex"}).text
|
||||
```
|
||||
|
||||
**arXiv API** (export.arxiv.org/api) provides metadata for preprints. Python library: `pip install arxiv`. No authentication required, but maintain 3-second delays between requests.
|
||||
|
||||
**OpenAlex** (api.openalex.org) offers 240M+ works under CC0 license—the open successor to Microsoft Academic Graph. 100K requests/day, 10 RPS with email in query string.
|
||||
|
||||
**Google Scholar has no official API**—scraping violates ToS. Use SerpApi ($75-275/month) only if Semantic Scholar coverage is insufficient.
|
||||
|
||||
---
|
||||
|
||||
## The verified citation workflow for AI assistants
|
||||
|
||||
For any AI-assisted paper writing, implement this verification pipeline to eliminate hallucinated citations:
|
||||
|
||||
1. **Search** using Semantic Scholar or OpenAlex APIs with specific queries
|
||||
2. **Verify existence** by confirming the paper appears in at least two sources (Semantic Scholar + CrossRef, or DOI resolution + arXiv)
|
||||
3. **Retrieve BibTeX** via DOI content negotiation for guaranteed accuracy
|
||||
4. **Verify claims** by accessing the actual paper (via DOI link or Semantic Scholar PDF) and confirming the attributed claim appears in the source
|
||||
5. **Maintain clean .bib files** using Zotero with Better BibTeX plugin for auto-export and consistent citation keys
|
||||
|
||||
Tools like Citely (citely.ai) and CiteSure provide batch verification of reference lists. ReciteWorks checks that in-text citations match your reference list.
|
||||
|
||||
For LaTeX, prefer **BibLaTeX with Biber backend** over legacy BibTeX—it provides full Unicode support, extended entry types (@online, @dataset), and flexible customization. Use `\citep{}` for parenthetical citations and `\citet{}` for textual citations.
|
||||
|
||||
---
|
||||
|
||||
## Conclusion
|
||||
|
||||
Writing top-quality ML papers is fundamentally about **reducing cognitive load for reviewers** while **maximizing evidence density for your claims**. The hierarchy of importance is clear: narrative clarity beats methodological complexity, front-loaded value beats comprehensive coverage, and verified reproducibility beats impressive numbers.
|
||||
|
||||
The most actionable insight: treat writing as iterative design. Nanda recommends paper swaps for mutual feedback; Karpathy suggests submitting a 5-page draft with all experiments two weeks before deadline to reveal critical gaps. Sebastian Farquhar captures the modern reality: "If you are a good writer, you are better than LLMs. If you are a bad writer, you need the practice"—but LLMs excel at identifying unclear passages through misinterpretation and simulating harsh reviewer feedback.
|
||||
|
||||
For citation workflows specifically, the combination of Semantic Scholar search → DOI content negotiation → BibLaTeX management provides a reliable, hallucination-resistant pipeline suitable for integration into AI writing assistants. Every citation must be verified to exist before inclusion—the alternative is contributing to the documented problem of fabricated references that has affected even NeurIPS accepted papers.
|
||||
@@ -0,0 +1,233 @@
|
||||
# Comprehensive Source List: Building an ML Paper Writing Skill
|
||||
|
||||
This document compiles authoritative sources for creating a Claude skill that writes high-quality ML/AI papers for venues like ICLR, NeurIPS, and ICML.
|
||||
|
||||
---
|
||||
|
||||
## Part 1: Writing Philosophy & Guides from ML Researchers
|
||||
|
||||
### Primary Sources (Must-Read)
|
||||
|
||||
| Source | Author | URL | Key Value |
|
||||
|--------|--------|-----|-----------|
|
||||
| **Highly Opinionated Advice on How to Write ML Papers** | Neel Nanda | https://www.alignmentforum.org/posts/eJGptPbbFPZGLpjsp/highly-opinionated-advice-on-how-to-write-ml-papers | Core narrative philosophy, "what/why/so what" framework, figure-first approach |
|
||||
| **How to Write ML Papers** | Sebastian Farquhar (DeepMind) | https://sebastianfarquhar.com/on-research/2024/11/04/how_to_write_ml_papers/ | 5-sentence abstract formula, structure templates, reader expectations |
|
||||
| **A Survival Guide to a PhD** | Andrej Karpathy | http://karpathy.github.io/2016/09/07/phd/ | Paper structure recipe, importance of reviewing bad papers, contribution framing |
|
||||
| **Heuristics for Scientific Writing (ML Perspective)** | Zachary Lipton (CMU) | https://www.approximatelycorrect.com/2018/01/29/heuristics-technical-scientific-writing-machine-learning-perspective/ | Snappy maxims for clear prose, vacuous intensifier warnings, section balance |
|
||||
| **Advice for Authors** | Jacob Steinhardt (UC Berkeley) | https://jsteinhardt.stat.berkeley.edu/blog/advice-for-authors | Precision over brevity, consistent terminology, reader-centric writing |
|
||||
| **Easy Paper Writing Tips** | Ethan Perez (Anthropic) | https://ethanperez.net/easy-paper-writing-tips/ | Practical micro-level tips, apostrophe unfolding, clarity tricks |
|
||||
|
||||
### Foundational Scientific Writing
|
||||
|
||||
| Source | Author | URL | Key Value |
|
||||
|--------|--------|-----|-----------|
|
||||
| **The Science of Scientific Writing** | Gopen & Swan | https://cseweb.ucsd.edu/~swanson/papers/science-of-writing.pdf | Topic/stress positions, old-before-new principle, sentence-level clarity |
|
||||
| **Summary of Science of Scientific Writing** | Lawrence Crowl | https://www.crowl.org/Lawrence/writing/GopenSwan90.html | Condensed version of Gopen & Swan principles |
|
||||
|
||||
### Additional Researcher Perspectives
|
||||
|
||||
| Source | URL | Key Value |
|
||||
|--------|-----|-----------|
|
||||
| How To Write A Research Paper In Machine Learning | https://grigorisg9gr.github.io/machine%20learning/research%20paper/how-to-write-a-research-paper-in-machine-learning/ | Practical walkthrough |
|
||||
| A Recipe for Training Neural Networks (Karpathy) | http://karpathy.github.io/2019/04/25/recipe/ | Debugging methodology that translates to paper structure |
|
||||
|
||||
---
|
||||
|
||||
## Part 2: Official Conference Guidelines
|
||||
|
||||
### NeurIPS
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| **Paper Checklist Guidelines** | https://neurips.cc/public/guides/PaperChecklist | Mandatory checklist items, reproducibility requirements |
|
||||
| **2025 Reviewer Guidelines** | https://neurips.cc/Conferences/2025/ReviewerGuidelines | What reviewers look for, scoring criteria |
|
||||
| **Formatting Instructions** | https://arxiv.org/html/2505.10292v1 | LaTeX template, page limits, style requirements |
|
||||
|
||||
### ICML
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| **Paper Guidelines** | https://icml.cc/Conferences/2024/PaperGuidelines | Submission requirements, ethics policy |
|
||||
| **Style & Author Instructions** | https://icml.cc/Conferences/2022/StyleAuthorInstructions | Formatting specifications |
|
||||
| **Reviewer Tutorial** | https://icml.cc/Conferences/2022/ReviewerTutorial | Evaluation criteria from reviewer perspective |
|
||||
| **Reviewer Guidelines (2020)** | https://icml.cc/Conferences/2020/ReviewerGuidelines | Detailed review criteria |
|
||||
| **ICML 2025 LaTeX Template** | https://www.overleaf.com/latex/templates/icml2025-template/dhxrkcgkvnkt | Official Overleaf template |
|
||||
|
||||
### ICLR
|
||||
|
||||
| Document | URL | Purpose |
|
||||
|----------|-----|---------|
|
||||
| **Author Guide 2026** | https://iclr.cc/Conferences/2026/AuthorGuide | Submission requirements, LLM disclosure policy |
|
||||
| **LLM Disclosure Policy** | https://eu.36kr.com/en/p/3443306502428032 | NEW: Mandatory AI use disclosure (desk rejection if missing) |
|
||||
|
||||
---
|
||||
|
||||
## Part 3: Citation APIs & Tools (Hallucination Prevention)
|
||||
|
||||
### Primary APIs for Paper Search & Metadata
|
||||
|
||||
| API | Documentation URL | Key Features | Rate Limits |
|
||||
|-----|-------------------|--------------|-------------|
|
||||
| **Semantic Scholar API** | https://api.semanticscholar.org/api-docs/ | 214M papers, citation graphs, AI-trained search | 1 RPS with API key |
|
||||
| **Semantic Scholar Tutorial** | https://www.semanticscholar.org/product/api/tutorial | Step-by-step usage guide | - |
|
||||
| **CrossRef REST API** | https://www.crossref.org/documentation/retrieve-metadata/rest-api/ | DOI metadata, direct BibTeX via content negotiation | Polite pool with mailto |
|
||||
| **arXiv API** | https://info.arxiv.org/help/api/basics.html | Preprint metadata, full-text access | 3-second delays |
|
||||
| **OpenAlex API** | https://docs.openalex.org/api-entities/works | 240M+ works, CC0 license, open successor to MAG | 100K/day, 10 RPS |
|
||||
|
||||
### Python Libraries
|
||||
|
||||
| Library | Install | Documentation | Purpose |
|
||||
|---------|---------|---------------|---------|
|
||||
| `semanticscholar` | `pip install semanticscholar` | https://semanticscholar.readthedocs.io/ | Official-ish Python wrapper |
|
||||
| `arxiv` | `pip install arxiv` | https://pypi.org/project/arxiv/ | arXiv search and download |
|
||||
| `habanero` | `pip install habanero` | https://github.com/sckott/habanero | CrossRef Python client |
|
||||
|
||||
### BibTeX Retrieval Code Pattern
|
||||
|
||||
```python
|
||||
import requests
|
||||
|
||||
def doi_to_bibtex(doi: str) -> str:
|
||||
"""Get BibTeX directly from DOI via CrossRef content negotiation."""
|
||||
response = requests.get(
|
||||
f"https://doi.org/{doi}",
|
||||
headers={"Accept": "application/x-bibtex"}
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.text
|
||||
|
||||
# Example: Get verified BibTeX for "Attention Is All You Need"
|
||||
bibtex = doi_to_bibtex("10.48550/arXiv.1706.03762")
|
||||
```
|
||||
|
||||
### Citation Verification Tools
|
||||
|
||||
| Tool | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| **Citely** | https://citely.ai/citation-checker | Batch verification of AI-generated citations |
|
||||
| **ReciteWorks** | https://reciteworks.com/ | Check in-text citations match reference list |
|
||||
|
||||
### LaTeX Citation Management
|
||||
|
||||
| Resource | URL | Key Info |
|
||||
|----------|-----|----------|
|
||||
| BibTeX vs BibLaTeX Guide | https://electricalvoice.com/latex-vs-bibtex-vs-biblatex/ | When to use which system |
|
||||
| BibLaTeX Comprehensive Guide | https://latextutorial.net/latex-vs-bibtex-vs-biblatex/ | Modern citation management |
|
||||
|
||||
---
|
||||
|
||||
## Part 4: The Verified Citation Workflow
|
||||
|
||||
### Recommended Pipeline for AI-Assisted Writing
|
||||
|
||||
```
|
||||
1. SEARCH: User specifies topic → Query Semantic Scholar API
|
||||
└─ Use paper/search endpoint with specific keywords
|
||||
|
||||
2. VERIFY EXISTENCE: For each candidate paper:
|
||||
└─ Confirm paper exists in 2+ sources (Semantic Scholar + CrossRef/arXiv)
|
||||
└─ Verify DOI resolves correctly
|
||||
|
||||
3. GET BIBTEX: Use DOI content negotiation
|
||||
└─ requests.get(f"https://doi.org/{doi}", headers={"Accept": "application/x-bibtex"})
|
||||
└─ NEVER generate BibTeX from memory - always fetch
|
||||
|
||||
4. VERIFY CLAIMS: Before citing paper for specific claim:
|
||||
└─ Retrieve paper abstract/full-text via Semantic Scholar
|
||||
└─ Confirm the attributed claim actually appears in source
|
||||
|
||||
5. BUILD BIBLIOGRAPHY:
|
||||
└─ Maintain .bib file with only verified entries
|
||||
└─ Use consistent citation keys (e.g., author_year_firstword)
|
||||
```
|
||||
|
||||
### Why This Matters
|
||||
|
||||
From research on AI citation hallucination:
|
||||
- ~40% of AI-generated citations contain errors (Enago Academy research)
|
||||
- NeurIPS 2025 found 100+ hallucinated citations slipped through review
|
||||
- Common errors: fabricated titles, wrong authors, non-existent papers with plausible metadata
|
||||
|
||||
---
|
||||
|
||||
## Part 5: Visualization & Formatting Resources
|
||||
|
||||
### Figure Creation
|
||||
|
||||
| Tool | URL | Purpose |
|
||||
|------|-----|---------|
|
||||
| **PlotNeuralNet** | https://github.com/HarisIqbal88/PlotNeuralNet | TikZ neural network diagrams |
|
||||
| **SciencePlots** | https://github.com/garrettj403/SciencePlots | Publication-ready matplotlib styles |
|
||||
| **Okabe-Ito Palette** | https://jfly.uni-koeln.de/color/ | Colorblind-safe color scheme |
|
||||
|
||||
### LaTeX Templates
|
||||
|
||||
| Venue | Template URL |
|
||||
|-------|--------------|
|
||||
| NeurIPS | https://neurips.cc/Conferences/2025/PaperInformation/StyleFiles |
|
||||
| ICML | https://www.overleaf.com/latex/templates/icml2025-template/dhxrkcgkvnkt |
|
||||
| ICLR | https://iclr.cc/Conferences/2026/AuthorGuide (links to template) |
|
||||
|
||||
---
|
||||
|
||||
## Part 6: Key Principles Summary (For Skill Encoding)
|
||||
|
||||
### From Neel Nanda
|
||||
1. Paper = short, rigorous, evidence-based technical story with a takeaway readers care about
|
||||
2. Spend equal time on: abstract, intro, figures, everything else
|
||||
3. Every experiment must support a specific claim connected to contribution
|
||||
4. "If you can't state your contribution in one sentence, you don't have a paper yet"
|
||||
|
||||
### From Karpathy
|
||||
1. "The paper sells a single thing that was not obvious before"
|
||||
2. Default structure: Intro → Related Work → Model → Experiments → Conclusions
|
||||
3. Review bad papers to learn what NOT to do (binary classifier training)
|
||||
|
||||
### From Zachary Lipton
|
||||
1. "If the first sentence can be pre-pended to any ML paper, delete it"
|
||||
2. Figures should tell coherent story even if reader skips text
|
||||
3. Sections should be balanced like bullets on slides
|
||||
4. "provides *very* tight approximation" drips with insecurity → "provides tight approximation"
|
||||
|
||||
### From Sebastian Farquhar
|
||||
1. Methods should start by page 2-3 maximum
|
||||
2. Abstract formula: (1) What achieved, (2) Why hard/important, (3) How with keywords, (4-5) Evidence + best number
|
||||
3. Introduction must have 2-4 bullet contribution list (max 1-2 lines each)
|
||||
|
||||
### From Gopen & Swan
|
||||
1. Place emphasis at sentence ends (stress position)
|
||||
2. Put context (old info) before new information
|
||||
3. Keep subject and verb close together
|
||||
4. One unit = one function (each paragraph = one point)
|
||||
|
||||
---
|
||||
|
||||
## Part 7: Additional Resources
|
||||
|
||||
### Hallucination & AI Writing Concerns
|
||||
|
||||
| Source | URL |
|
||||
|--------|-----|
|
||||
| AI Hallucinations in Research Citations | https://www.enago.com/academy/ai-hallucinations-research-citations/ |
|
||||
| Hallucination in AI-Generated Writing (PMC) | https://pmc.ncbi.nlm.nih.gov/articles/PMC10726751/ |
|
||||
| NeurIPS 2025 AI Hallucination Report | https://byteiota.com/neurips-2025-100-ai-hallucinations-slip-through-review/ |
|
||||
|
||||
### ML Conference Review System Analysis
|
||||
|
||||
| Source | URL |
|
||||
|--------|-----|
|
||||
| Position: ML Conferences Should Have Refutations Track | https://arxiv.org/html/2506.19882v1 |
|
||||
|
||||
---
|
||||
|
||||
## Usage Notes for Skill Development
|
||||
|
||||
1. **For paper structure**: Start with Nanda + Farquhar for high-level philosophy, use conference guidelines for specifics
|
||||
|
||||
2. **For writing style**: Combine Lipton's heuristics + Gopen & Swan's principles + Ethan Perez's micro-tips
|
||||
|
||||
3. **For citation workflow**: Implement Semantic Scholar → DOI verification → CrossRef BibTeX pipeline; NEVER generate citations from model memory
|
||||
|
||||
4. **For figures/tables**: Reference booktabs for tables, SciencePlots for figures, always use colorblind-safe palettes
|
||||
|
||||
5. **For reviewer simulation**: Study reviewer guidelines from all three venues to anticipate criticisms
|
||||
Reference in New Issue
Block a user