lamaRigorPilot Skills
Run research repositories from their README, with bounded execution and auditable evidence. RigorPilot adds section-level results without rewriting the original README. Trusted reproduction is the default; candidate exploration requires explicit authorization.
Real examples · Install & use · Skill index · Validation · Engineering roadmap
📄 Real repositories, inspectable results
Original commands, prose, badges, images, videos and HTML stay in the source file. RigorPilot splits that file into sections and inserts one evidence-linked card per section. Removing its insertion blocks restores the retained original README byte for byte.
Each card below opens a full annotated README beside the original README in a retained repository checkout. Supporting repository files are kept so relative links and media retain their original context.
🟢 selected checks passed · 🔵 not executed · ⚪ read only · 🟡 partial · 🔴 blocked · 🟣 decision needed. Green does not automatically mean paper-result reproduction; blue is not an execution failure.
 micrograd · correctness checks 🟢 2 tests passed in 7.62 s 8 headings = 8 annotations · original bytes preserved Open full RigorPilot README → |
 minGPT · selection boundary 🔵 Test selected, not executed · no model download 11 headings = 11 annotations · original bytes preserved Open full RigorPilot README → |
 PyTorch MNIST · bounded startup 🟡 Partial training · observed loss 0.038893 1 heading = 1 annotation · original bytes preserved Open full RigorPilot README → |
 nanoGPT Shakespeare · bounded training 🟡 Partial · train loss 4.1676 · validation loss 4.1649 11 headings = 11 annotations · original bytes preserved Open full RigorPilot README → |
All four cases and upstream links · Recorded suite · Case definitions · Methodology
These are historical, commit-pinned deterministic runs: 4/4 case protocols
passed in 251.0 s, with a peak workspace of 98.67 MiB and 0 model API calls.
The zero-API count applies only to that suite. Selection-only and partial cases
are not completed evaluations, converged training or reproduced paper scores.
New: installed-skill micrograd trial, with before/after command reports, a retained failed attempt and independent checks—not a model-quality comparison.
Real model trial: public install → 2 tests passed → evidence verified → client timed out. Full outputs and failed attempts are retained; end-to-end acceptance and model uplift are not claimed.
🚀 Install and use
The installer needs Node.js/npm. Tested with skills@1.5.26 and Node 22.20.0;
that installer requires Node ≥22.20.0. If you see EBADENGINE, check the requested version.
Install all skills:
npx skills add lllllllama/rigorpilot-skills --all
Or install only the self-contained reproduction skill:
npx skills add lllllllama/rigorpilot-skills --skill ai-research-reproduction
Open the target repository in a Skills-capable agent, then ask:
Use ai-research-reproduction: run the smallest README-documented evaluation, preserve the source and write evidence to repro_outputs/, plus an annotated copy beside the original README. Ask before large downloads or long training.
The main skill works alone; choose all skills for companion and leaf entrypoints. Your existing agent loads the skill. The standalone model runner is optional. Client compatibility
Start with the RIGORPILOT_README.md reported in source_adjacent_readme.path,
then follow its command and log links. If a conflicting file blocks the extra
copy, that file stays intact; inspect repro_outputs/SUMMARY.md for the outcome
and next action.
What it does—and does not do
README → documented target → reviewed setup → bounded execution → verification → evidence.
- Preserves source meaning; records assumptions, deviations, failures and blockers.
- Records process state, logs and attempt lineage; supports explicit cancellation, recovery and retry through the persistent runtime.
- Separates trusted reproduction from explicitly authorized, candidate-only exploration.
- Checks execution criteria independently of the model's completion claim.
This is local execution, not an OS sandbox. Approved commands can access the host and network; use trusted repositories. Resource admission and between-action budget checks are not hard OS quotas or subscription-balance monitoring.
The optional model loop currently supports Anthropic Messages and reviewed command IDs, not unrestricted source repair. This standalone runner has no successful live-model acceptance recorded yet: three provider attempts returned HTTP 502. Other model profiles are metadata, not proof of working transports or equivalent model performance. Runner and recovery contract · Implementation evidence and limits
🎯 Skill index
| Task | Skill |
|---|---|
| Reproduce from README commands | ai-research-reproduction |
| Read-only repository analysis | analyze-project |
| Prepare environment, data and weights | env-and-assets-bootstrap |
| Run documented inference or evaluation | minimal-run-and-audit |
| Start or verify training conservatively | run-train |
| Diagnose before proposing a patch | safe-debug |
| Coordinate authorized candidate exploration | ai-research-explore |
| Implement a candidate change on an isolated branch | explore-code |
| Execute a bounded candidate experiment | explore-run |
Two helpers support orchestration: repo-intake-and-plan and paper-context-resolver.
Exploration requires a durable current_research anchor and a frozen comparison
contract. Candidate results never become trusted baseline results by declaration.
Routing · Research loop ·
Campaign inputs
📦 Evidence bundle
| Artifact | What to inspect |
|---|---|
repro_outputs/ANNOTATED_README.md |
Original README with inserted section verdicts |
SUMMARY.md, COMMANDS.md, LOG.md, status.json |
Outcome, exact commands, observations and machine-readable status |
PATCHES.md, SCIENTIFIC_CHANGELOG.md, COMPARABILITY_REPORT.md |
Changes, scientific meaning and comparison boundaries |
_runtime/<run_id>/ |
Process state, events, resource samples and stdout/stderr |
agent_state.json, trajectory.jsonl |
Optional model runner's checkpoints, tool calls and reported usage |
🟢 success · 🔵 not executed · ⚪ read only · 🟡 partial · 🔴 blocked · 🟣 decision required
Standard evidence stays under repro_outputs/. Both main runners accept
--source-adjacent-readme to also write RIGORPILOT_README.md beside the original,
preserving the context of its relative media/file links. Only inserted evidence
links are rebased. The same output directory may refresh its unchanged owned
copy, never an unrelated or manually edited file. Retain supporting repository
files and the evidence directory's readme_delivery.json.
Output contract · Rigor principles
✅ Offline validation
From a clone of this project, with Python 3.11+ and Git:
python benchmarks/run_skill_acceptance.py --output tmp/skill-check
Runs the installed-layout skill runtime on three small cases: missing data,
matching metrics, and exit-zero/wrong metrics. Independently checks raw logs,
predictions, original files and README insertions. With existing PyTorch/pytest,
add --include-micrograd to run the two unchanged upstream tests too.
No model calls, downloads or package installs; use a fresh output directory each time.
Recorded outcomes and full evidence — 4/4 functional
checks, not four successful reproductions or measured model uplift.
For installation/environment problems (replace the skill path after installation):
python skills/ai-research-reproduction/scripts/doctor.py --repo /path/to/target
This read-only check reports the actual Python, Git, bundled-file integrity and
README availability. Add --require-module torch --require-module pytest for
dependency discovery; it does not install anything or execute target code.
For the failure-and-recovery walkthrough:
python scripts/run_harness_lab.py
This offline example uses scripted decisions and actual processes. It exercises
failure → preparation → pause → controller restart → independent verification,
without API calls, GPU use or model downloads. Inspect the printed REPORT.json
path and its linked artifacts. Existing output is never overwritten; use
--output tmp/check-2 to repeat. It is not evidence of live-model capability.
Example source and checks
Run the repository regression suite:
python scripts/run_all_tests.py
Latest local Windows record (2026-09-13): 71/71 scripts passed in 173.1 s. The CI badge links to the current Windows, Linux and macOS results. Local tests do not substitute for live-model or held-out evaluation.
For model comparisons, the small paired-evaluation kit provides frozen tasks and actual grader-calibration logs. The six planned model trials remain unrun; calibration is not evidence of skill uplift.
Controlled-trial checks add real failure/recovery logs, restricted tools and unknown-usage stops, with scripted model responses. The guide also provides a bounded A/B command-line entrypoint for model transport → reviewed commands → independent grading → sealed summary. Local HTTP integration is tested; real-provider effectiveness is not yet measured.
Engineering and contributions
Engineering roadmap · Contributing · Security and reporting · CI workflow · Reproduction feedback · MIT license
Keep acceptance checks independent, retain failed evidence and review traces before publication. Do not publish credentials or unreviewed private repository data. Agent guidance · Operating principles · Personalization policy
Historical interface illustration—not execution evidence
First attempt · After setup. This older MiniSeg preview illustrates error, metric and authorization displays. Its execution provenance is not independently verified; it is excluded from benchmarks.