Files
Udit Goenka 9f51f726e5 feat(autoresearch): autonomous goal-directed orchestrator (v2.2.0) (#105)
* feat(autoresearch): add autonomous goal-directed orchestrator (v2.2.0)

Overload bare `/autoresearch` so a plain-language goal drives the whole
subcommand suite autonomously, the way `/ck:cook` does for implementation.
The user states a goal; the orchestrator classifies it, derives a concrete
success predicate, and loops the right subcommands until the predicate holds.

Dispatch (bare invocation):
- `Metric:`/`Verify:` present  -> Classic metric loop (unchanged)
- free-form natural-language goal -> Orchestrator
- nothing                       -> setup wizard
- `--classic` / `--auto` force the respective mode; mode printed in banner.

Two orchestrator modes by goal archetype:
- Orchestration loop for predicate-bearing archetypes (ship-ready,
  optimize-metric, fix-broken, harden, build-feature, explore) — loops until
  the mechanical predicate is met.
- Single-pass dispatch for subjective/terminal archetypes (document,
  what-to-build, decide-design) — routes once to learn/improve/reason.

All routing logic lives in one deterministic seam, scripts/orchestrate.sh,
mirroring the existing scripts/score-regression.sh <-> tests/test-regression.sh
pattern. Subcommands: classify, next-hop, units, plateau, screen-cmd, verdict.

Safety invariants:
- Never auto-approves ship/deploy/push; deploy always needs user approval.
- Bounded by plateau detection (5 cycles no net progress) and a hard ceiling
  (50, override --max-cycles); repeated unknown-units cycles route to BLOCKED.
- Every derived command is safety-screened (rm recursive+force in any flag
  arrangement, curl|sh, credential patterns, fork bombs) and re-screened when
  read from persisted state on resume.
- Data-migration stays behind the anchored DB-URL allowlist (host must be
  localhost/127.0.0.1/single-label container, or dbname carries _test/_ci
  suffix); bare substring does not qualify.

Distribution parity: spec mirrored across .claude, claude-plugin, .agents
(codex), .opencode, and plugins/autoresearch; manifests bumped to 2.2.0;
scripts/transform.sh propagates the new reference and also fixes a latent
:regression colon-drift in the codex/opencode adapters.

Tests: tests/test-orchestrator.sh (85 assertions) green; existing
test-regression.sh (50) and test-hooks.sh (105) still green.

* fix(autoresearch): harden screen-cmd against path-qualified rm and alt-shell curl pipes

The screen-cmd safety gate anchored its rm matcher on the bare command name,
so a path-qualified invocation (/bin/rm, ./rm, /usr/local/bin/rm) bypassed the
recursive+force refusal. Likewise the curl/wget pipe matcher only caught sh and
bash, missing zsh/dash/fish/ksh/python/perl/ruby/node/php and path-qualified
shells. Both are autonomous-loop command-screening bypasses.

- rm: allow an optional path prefix ending in /rm on the command-name anchor.
- curl/wget pipe: match an optional path prefix plus an enumerated interpreter
  set. Enumerated rather than "refuse any curl pipe" so a legitimate derived
  predicate that pipes curl output to a parser (jq/grep/awk) is not refused.

Added 12 assertions: path-qualified rm (4 forms) refuse; non-rm words
(confirm/perform) stay ok; alt-shell/path-qualified curl pipes refuse;
curl piped to a parser stays ok. test-orchestrator.sh now 97/97.

---------

Co-authored-by: uditgoenka <udit@firstsales.io>
2026-06-20 13:09:10 +05:30

6.6 KiB

Autoresearch Domain Glossary

Terms meaningful to domain experts. Implementation details live in code, not here.

Output Types (per subcommand)

Term Subcommand Definition
Constraint probe A requirement extracted from persona interrogation. Atomic, deduplicated, has confidence and evidence. Constraints are things the product MUST NOT violate.
Finding predict, security A code-derived observation from expert personas or STRIDE/OWASP analysis. Has severity, confidence, and file:line evidence. Findings describe what IS (current state).
Hypothesis debug A falsifiable claim about a bug's root cause. Tested via investigation, classified as confirmed/disproven/inconclusive.
Scenario scenario An edge case generated across 12 dimensions. Classified as new/extension/duplicate with severity.
Insight improve An externally-researched product improvement opportunity, structured as {problem, affected_persona, proposed_mechanism, expected_outcome}. Unlike findings (code-derived) or hypotheses (falsifiable bug claims), insights originate from market data, user research, or competitive analysis and represent synthesized-but-unvalidated understanding.

Loop Shapes

Shape Subcommands Pattern Notes
Metric loop core, fix commit → verify metric → keep/discard based on direction
Saturation loop probe iterate until net-new output drops below threshold for N consecutive rounds Internal extraction
Saturation loop improve iterate until net-new output drops below threshold for N consecutive rounds External research; LLM-judged dedup; triangulation replaces git-as-memory
Hypothesis loop debug, security form hypothesis → investigate → classify → repeat
Refinement loop reason generate candidates → critique → judge → converge
Exploration loop scenario, learn generate → classify → check coverage/saturation
One-shot plan, predict, evals, ship No iteration loop. Phased pipeline or single-pass analysis.
Orchestration loop orchestrator (predicate-bearing goals) classify → route subcommand → run → recompute Units remaining → repeat until Success predicate met Plateau

Scoring Systems

System Subcommand How it works
Severity ranking debug, security, predict Critical/High/Medium/Low/Info per finding
Composite metric security score = (owasp_tested/10)*50 + (stride_tested/6)*30 + min(findings, 20)
Tiered ranking improve ICP binary gate → Must-have/Nice-to-have/Moonshot tiers → pairwise within Must-have → confidence indicator
Convergence reason Incumbent wins N consecutive judge rounds → converged
Saturation probe, improve Net-new below threshold for 3 consecutive rounds → saturated

Key Concepts

Term Definition
ICP Ideal Customer Profile. The specific customer segment the product targets. Used by improve to filter and prioritize insights by relevance to the target buyer.
Product context Background understanding of what a product does, derived from existing docs. Sourced (in priority order) from: learn summary (autoresearch/learn-*/summary.md), README.md (≥500 chars), package manifest description (≥10 chars), or conditional auto-discover scan. NOT docs/codebase-summary.md which is autoresearch's own doc.
Chain Sequential handoff between subcommands via handoff.json. Each command reads upstream findings and passes its own downstream.
Terminal emitter A command whose output is consumed by humans or external tools, not by other autoresearch subcommands. Writes handoff.json for protocol consistency but is the last autoresearch link. Example: improve produces PRDs for /ck:plan and /ck:cook, not for autoresearch re-entry.
Guard An optional safety command (e.g., npm test) that must pass for a "keep" decision. Reverts on failure regardless of metric improvement.
Metric direction higher_is_better or lower_is_better. Written as TSV comment on line 1. Determines whether improvement means going up or down.
Saturation The state where a loop produces diminishing returns. Detected when net-new output drops below a threshold for N consecutive iterations.
Keep/discard (improve) Improve reuses the standard keep/discard vocabulary but applies it to insights, not code commits. A novel insight is "kept" (logged as keep); a duplicate is "discarded" (logged as discard). This preserves evals compatibility.
Goal archetype A classification of a natural-language goal that selects a starting pipeline and mode (Orchestration loop or Single-pass dispatch). The nine archetypes are: fix-broken, ship-ready, optimize-metric, harden, build-feature, explore, document, decide-design, what-to-build.
Success predicate A mechanical goal-met check: a concrete shell command and its expected result (e.g., npm test → exit 0). Generalizes metric+threshold, errors==0, and tests-green into one form. Confirmed once upfront, before the loop starts. Distinct from Guard (which checks safety on every iteration) and Metric (which measures directional improvement).
Units remaining The Orchestration loop's goal-distance scalar. Lower is better. Composite; default weights: each failing test = 1, each open HARD regression = 1, metric delta normalized to its target. A cycle that cannot compute Units returns unknown — never counted as zero-progress.
Plateau Units remaining flat or worse for N=5 consecutive computed cycles. Stops the Orchestration loop and produces a checkpoint report. Catches both stalls and thrash (oscillation netting zero). Distinct from Saturation, which measures net-new output in discovery loops (probe, improve), not goal-distance.
Orchestration Dynamic state-driven routing among subcommands toward a Success predicate. Distinct from Chain, which is static, linear, and user-specified. Built on Chain's handoff.json bridge — each subcommand hop writes handoff.json as usual; the orchestrator folds it into orchestrator-state.json.
Single-pass dispatch The orchestrator mode for subjective or terminal goals (document, what-to-build, decide-design). No mechanical predicate exists, so the orchestrator routes to one self-terminating subcommand, lets it run, and reports. No loop, no Plateau, no ceiling, no ship gate.