mirror of
https://github.com/uditgoenka/autoresearch.git
synced 2026-09-14 20:07:09 +08:00
9f51f726e5
* feat(autoresearch): add autonomous goal-directed orchestrator (v2.2.0) Overload bare `/autoresearch` so a plain-language goal drives the whole subcommand suite autonomously, the way `/ck:cook` does for implementation. The user states a goal; the orchestrator classifies it, derives a concrete success predicate, and loops the right subcommands until the predicate holds. Dispatch (bare invocation): - `Metric:`/`Verify:` present -> Classic metric loop (unchanged) - free-form natural-language goal -> Orchestrator - nothing -> setup wizard - `--classic` / `--auto` force the respective mode; mode printed in banner. Two orchestrator modes by goal archetype: - Orchestration loop for predicate-bearing archetypes (ship-ready, optimize-metric, fix-broken, harden, build-feature, explore) — loops until the mechanical predicate is met. - Single-pass dispatch for subjective/terminal archetypes (document, what-to-build, decide-design) — routes once to learn/improve/reason. All routing logic lives in one deterministic seam, scripts/orchestrate.sh, mirroring the existing scripts/score-regression.sh <-> tests/test-regression.sh pattern. Subcommands: classify, next-hop, units, plateau, screen-cmd, verdict. Safety invariants: - Never auto-approves ship/deploy/push; deploy always needs user approval. - Bounded by plateau detection (5 cycles no net progress) and a hard ceiling (50, override --max-cycles); repeated unknown-units cycles route to BLOCKED. - Every derived command is safety-screened (rm recursive+force in any flag arrangement, curl|sh, credential patterns, fork bombs) and re-screened when read from persisted state on resume. - Data-migration stays behind the anchored DB-URL allowlist (host must be localhost/127.0.0.1/single-label container, or dbname carries _test/_ci suffix); bare substring does not qualify. Distribution parity: spec mirrored across .claude, claude-plugin, .agents (codex), .opencode, and plugins/autoresearch; manifests bumped to 2.2.0; scripts/transform.sh propagates the new reference and also fixes a latent :regression colon-drift in the codex/opencode adapters. Tests: tests/test-orchestrator.sh (85 assertions) green; existing test-regression.sh (50) and test-hooks.sh (105) still green. * fix(autoresearch): harden screen-cmd against path-qualified rm and alt-shell curl pipes The screen-cmd safety gate anchored its rm matcher on the bare command name, so a path-qualified invocation (/bin/rm, ./rm, /usr/local/bin/rm) bypassed the recursive+force refusal. Likewise the curl/wget pipe matcher only caught sh and bash, missing zsh/dash/fish/ksh/python/perl/ruby/node/php and path-qualified shells. Both are autonomous-loop command-screening bypasses. - rm: allow an optional path prefix ending in /rm on the command-name anchor. - curl/wget pipe: match an optional path prefix plus an enumerated interpreter set. Enumerated rather than "refuse any curl pipe" so a legitimate derived predicate that pipes curl output to a parser (jq/grep/awk) is not refused. Added 12 assertions: path-qualified rm (4 forms) refuse; non-rm words (confirm/perform) stay ok; alt-shell/path-qualified curl pipes refuse; curl piped to a parser stays ok. test-orchestrator.sh now 97/97. --------- Co-authored-by: uditgoenka <udit@firstsales.io>
6.6 KiB
6.6 KiB
Autoresearch Domain Glossary
Terms meaningful to domain experts. Implementation details live in code, not here.
Output Types (per subcommand)
| Term | Subcommand | Definition |
|---|---|---|
| Constraint | probe | A requirement extracted from persona interrogation. Atomic, deduplicated, has confidence and evidence. Constraints are things the product MUST NOT violate. |
| Finding | predict, security | A code-derived observation from expert personas or STRIDE/OWASP analysis. Has severity, confidence, and file:line evidence. Findings describe what IS (current state). |
| Hypothesis | debug | A falsifiable claim about a bug's root cause. Tested via investigation, classified as confirmed/disproven/inconclusive. |
| Scenario | scenario | An edge case generated across 12 dimensions. Classified as new/extension/duplicate with severity. |
| Insight | improve | An externally-researched product improvement opportunity, structured as {problem, affected_persona, proposed_mechanism, expected_outcome}. Unlike findings (code-derived) or hypotheses (falsifiable bug claims), insights originate from market data, user research, or competitive analysis and represent synthesized-but-unvalidated understanding. |
Loop Shapes
| Shape | Subcommands | Pattern | Notes |
|---|---|---|---|
| Metric loop | core, fix | commit → verify metric → keep/discard based on direction | |
| Saturation loop | probe | iterate until net-new output drops below threshold for N consecutive rounds | Internal extraction |
| Saturation loop | improve | iterate until net-new output drops below threshold for N consecutive rounds | External research; LLM-judged dedup; triangulation replaces git-as-memory |
| Hypothesis loop | debug, security | form hypothesis → investigate → classify → repeat | |
| Refinement loop | reason | generate candidates → critique → judge → converge | |
| Exploration loop | scenario, learn | generate → classify → check coverage/saturation | |
| One-shot | plan, predict, evals, ship | No iteration loop. Phased pipeline or single-pass analysis. | |
| Orchestration loop | orchestrator (predicate-bearing goals) | classify → route subcommand → run → recompute Units remaining → repeat until Success predicate met | Plateau |
Scoring Systems
| System | Subcommand | How it works |
|---|---|---|
| Severity ranking | debug, security, predict | Critical/High/Medium/Low/Info per finding |
| Composite metric | security | score = (owasp_tested/10)*50 + (stride_tested/6)*30 + min(findings, 20) |
| Tiered ranking | improve | ICP binary gate → Must-have/Nice-to-have/Moonshot tiers → pairwise within Must-have → confidence indicator |
| Convergence | reason | Incumbent wins N consecutive judge rounds → converged |
| Saturation | probe, improve | Net-new below threshold for 3 consecutive rounds → saturated |
Key Concepts
| Term | Definition |
|---|---|
| ICP | Ideal Customer Profile. The specific customer segment the product targets. Used by improve to filter and prioritize insights by relevance to the target buyer. |
| Product context | Background understanding of what a product does, derived from existing docs. Sourced (in priority order) from: learn summary (autoresearch/learn-*/summary.md), README.md (≥500 chars), package manifest description (≥10 chars), or conditional auto-discover scan. NOT docs/codebase-summary.md which is autoresearch's own doc. |
| Chain | Sequential handoff between subcommands via handoff.json. Each command reads upstream findings and passes its own downstream. |
| Terminal emitter | A command whose output is consumed by humans or external tools, not by other autoresearch subcommands. Writes handoff.json for protocol consistency but is the last autoresearch link. Example: improve produces PRDs for /ck:plan and /ck:cook, not for autoresearch re-entry. |
| Guard | An optional safety command (e.g., npm test) that must pass for a "keep" decision. Reverts on failure regardless of metric improvement. |
| Metric direction | higher_is_better or lower_is_better. Written as TSV comment on line 1. Determines whether improvement means going up or down. |
| Saturation | The state where a loop produces diminishing returns. Detected when net-new output drops below a threshold for N consecutive iterations. |
| Keep/discard (improve) | Improve reuses the standard keep/discard vocabulary but applies it to insights, not code commits. A novel insight is "kept" (logged as keep); a duplicate is "discarded" (logged as discard). This preserves evals compatibility. |
| Goal archetype | A classification of a natural-language goal that selects a starting pipeline and mode (Orchestration loop or Single-pass dispatch). The nine archetypes are: fix-broken, ship-ready, optimize-metric, harden, build-feature, explore, document, decide-design, what-to-build. |
| Success predicate | A mechanical goal-met check: a concrete shell command and its expected result (e.g., npm test → exit 0). Generalizes metric+threshold, errors==0, and tests-green into one form. Confirmed once upfront, before the loop starts. Distinct from Guard (which checks safety on every iteration) and Metric (which measures directional improvement). |
| Units remaining | The Orchestration loop's goal-distance scalar. Lower is better. Composite; default weights: each failing test = 1, each open HARD regression = 1, metric delta normalized to its target. A cycle that cannot compute Units returns unknown — never counted as zero-progress. |
| Plateau | Units remaining flat or worse for N=5 consecutive computed cycles. Stops the Orchestration loop and produces a checkpoint report. Catches both stalls and thrash (oscillation netting zero). Distinct from Saturation, which measures net-new output in discovery loops (probe, improve), not goal-distance. |
| Orchestration | Dynamic state-driven routing among subcommands toward a Success predicate. Distinct from Chain, which is static, linear, and user-specified. Built on Chain's handoff.json bridge — each subcommand hop writes handoff.json as usual; the orchestrator folds it into orchestrator-state.json. |
| Single-pass dispatch | The orchestrator mode for subjective or terminal goals (document, what-to-build, decide-design). No mechanical predicate exists, so the orchestrator routes to one self-terminating subcommand, lets it run, and reports. No loop, no Plateau, no ceiling, no ship gate. |