Studied via direct fetches: SakanaAI/AI-Scientist-v2, WecoAI/aideml,
anthropics/skills, obra/superpowers, UniM0cha/claude-self-improving-
skills, SamuelSchmidgall/AgentLaboratory, princeton-nlp/SWE-agent,
openai/preparedness (PaperBench), open-mmlab/mmsegmentation,
facebookresearch/dinov2, karpathy/nanoGPT. Applied the top verified
findings:
- extract_commands: join backslash-continued commands (every dinov2 run
command was previously truncated to an unrunnable `python ... \` stub)
and classify entrypoint-first with word-boundary keywords (nanoGPT's
`train.py ... --eval_iters=20` classified as evaluation, which would
execute training while bypassing the authorization gate). Quick-start
sections now count as run sections. Both real-repo patterns added as
regression fixtures.
- research-thinking-loop: draft/debug/improve iteration types with a
3-attempt debug cap (AIDE journal semantics), no-parsed-metric ⇒ buggy
and never best, replication across 3 seeds before a candidate replaces
current_research (AI-Scientist-v2 multi-seed evaluation), metric-only
best selection note, typed stop reasons, defaults table, ledger-first
grounding.
- annotate_readme: PaperBench-style evidence tiers per annotation
(code-development / execution / result-match) and a weighted 0-1
reproduction score in the header chip and readme_section_coverage.
- lessons_store: touch/prune lifecycle with usage-extended staleness
windows, credential-shape blocklist (AKIA/ghp_/sk-/xox/AIza),
best-effort security wording, what-NOT-to-record and human-reviewed
promotion flow in the policy.
- Demo bundles and preview images regenerated with score and tiers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HT2VAaQodjqTJdjBHSStxA
Research thinking loop (references/research-thinking-loop.md):
- Codifies the greedy, evidence-anchored cycle required in the explore
lane: observe -> ground (paper/code/prior-run/intuition anchors) ->
falsifiable hypothesis -> single-variable design -> bounded run ->
fair comparison vs current best -> greedy keep/rollback -> ledger
record. Greedy applies to selection, never to honesty: keeps require
comparable evidence under the frozen contract.
- Adapts AIDE's greedy code-space search and AI-Scientist-v2's managed
agentic tree search under RigorPilot's comparability-first gates.
- Wired into ai-research-explore SKILL.md and AGENTS.md.
Continuous learning (references/continuous-learning-policy.md):
- Immutable universal rigor core + user-owned lessons overlay.
- shared/scripts/lessons_store.py: JSONL store under ~/.rigorpilot/
(RIGORPILOT_HOME override, RIGORPILOT_LESSONS=0 opt-out), secret-line
refusal, dedup, bounded summarize into PERSONAL_RIGOR.md with an
explicit core-wins disclaimer.
- orchestrate_repro auto-records failure blockers and their later
resolutions per repo fingerprint; payload reports lesson_recorded.
- Hard rules: lessons are advisory only, never relax rigor gates, never
store secrets, never edit skill files; deleting the folder restores
base behavior.
- Installed alongside the skills (shared references + shared scripts).
Docs and tests:
- READMEs (en/zh): new Research Thinking Loop section with the greedy
cycle mermaid replacing the old explore-flow diagram, new Continuous
Learning section, two At-a-Glance rows, counts 48 scripts / 45 tests.
- scripts/test_lessons_store.py: persist/dedup/secret-refusal/summarize/
opt-out/CLI coverage (7 checks); full suite 46/46 green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HT2VAaQodjqTJdjBHSStxA
- Add root AGENTS.md so AGENTS.md-aware agents (OpenAI Codex, Cursor,
GitHub Copilot, Gemini CLI, Aider, Zed, ...) pick up the lane model,
entrypoint table, evidence contract, and hard rules with no install
step, complementing the Agent Skills (SKILL.md) standard the skills
already follow.
- Annotated README gains a rubric-style section-coverage scoreboard in
its header (e.g. 1 green / 2 yellow / 1 purple / 10 read-only of 14
sections), and the counts are recorded machine-readably as
readme_section_coverage in repro_outputs/status.json — inspired by how
replication benchmarks like PaperBench grade reproduction with
hierarchical rubrics instead of a single pass/fail. The annotation now
runs before the bundle writers so status.json carries the coverage and
the annotated-readme output path.
- READMEs: promote the annotated README to flagship position (moved
directly after Start Fast, new At-a-Glance rows), add a Multi-Agent /
Multi-Model section, and document the scoreboard. zh-CN mirrored.
- client-compatibility-policy.md: cover the Agent Skills open standard
and AGENTS.md consistency rules.
- Demo bundles regenerated with the scoreboard; renderer test now checks
the coverage line (17 checks).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HT2VAaQodjqTJdjBHSStxA
- run_training.py / run_command.py: subprocess.TimeoutExpired carries
captured output as bytes on POSIX even with text=True; decode it
before log combining so timed-out runs no longer crash on Linux.
- write_explore_bundle.py: guard selected_idea / patch_surface_summary
against explicit None so blocked manifests (no idea passes the gate)
render COMPARABILITY_REPORT.md instead of raising AttributeError.
Fixes test_training_lane_routing.py, test_research_explore_blocked_manifest.py,
and test_research_explore_campaign_abandon.py on Linux.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HT2VAaQodjqTJdjBHSStxA
- Introduced language policy to ensure human-readable outputs while maintaining machine-readable stability.
- Created output specification detailing required output files and their formats.
- Established patch policy to guide conservative modifications to repository code.
- Defined research safety principles to prioritize observable and reviewable research progress.
- Implemented orchestration script for minimal README-first reproduction, including command extraction and execution handling.
- Added functions for managing assets, estimating training duration, and building context for reproducibility.
- Implemented `lookup_sources.py` to facilitate a free-first, cache-first research lookup pass, including functions for deduplication, seed record collection, and provider record resolution.
- Introduced `source_mapping.py` for source module lookup and interface diffing, featuring functions to select source records, build target location maps, and generate minimal patch plans.
- Enhanced data normalization and evidence class handling to improve the accuracy of source mapping.
- Added utility functions for writing module candidates and interface diffs to markdown files for better documentation.
- Introduced `test_research_explore_campaign_abandon.py` to validate abandonment of far-from-SOTA baselines.
- Added `test_research_explore_campaign_checkpoint.py` to ensure human checkpoint gating for near-tied ideas.
- Created `test_research_explore_campaign_flow.py` to verify the overall campaign flow and execution policies.
- Documented the `research_campaign_spec.md` outlining the structure and requirements for campaign JSON/YAML configurations.
- Added `run_training.py` modifications to improve command execution handling.
- Introduced `explore-module-roadmap.md` to outline the evolution of the explore lane.
- Created `explore-variant-spec.md` for defining variant specifications in exploratory runs.
- Implemented regression tests for non-training and training variant execution in `test_research_explore_nontraining_execution.py` and `test_research_explore_variant_execution.py`.
- Developed `plan_code_changes.py` to generate exploratory code-change plans.
- Added `run_command.py` for executing short non-training commands and normalizing evidence.
- Added `research-explore` skill to facilitate end-to-end exploratory work on top of `current_research`.
- Updated `explore-code` and `explore-run` skills to clarify boundaries and usage.
- Enhanced documentation for all skills to reflect new orchestrator capabilities and requirements.
- Implemented orchestration logic in `scripts/orchestrate_explore.py` to manage exploratory code and run changes together.
- Created output structure for exploratory results in `explore_outputs/`.
- Added regression tests for the new orchestrator to ensure proper functionality and integration.
lama