mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
test(overnight): cover live tree hash statuses
Refs na-1iv
This commit is contained in:
+5
-1
@@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
- **Native Windows smoke gate** — `tests/windows/test-windows-smoke.ps1` now exercises the PowerShell `ao` and Codex installers, local `ao doctor` Windows guidance, and focused Windows-sensitive Go tests, with a blocking `windows-smoke` job in `validate.yml`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Dream RunLoop status invariants** — live-tree hash coverage now exercises every terminal RunLoop status, and `degraded` reflects the current pre-commit MEASURE rollback semantics.
|
||||
|
||||
## [2.36.0] - 2026-04-11
|
||||
|
||||
### Added
|
||||
@@ -19,7 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Autodev program contract** — root `PROGRAM.md` gives evolve/autodev a repo-local operating contract with mutable and immutable scope, validation commands, escalation policy, and stop conditions.
|
||||
- **Beads stale-scope tooling** — `ao beads verify|lint|harvest` adds first-class stale-citation checks for bead-driven planning and RPI recovery.
|
||||
- **RPI discovery artifacts** — RPI can now persist and consume discovery artifacts, with tests and docs covering the `--discovery-artifact` path.
|
||||
- **Dream RunLoop invariant coverage** — `TestRunLoop_LiveTreeHashInvariant_AllStatuses` locks the `IsCorpusCompounded()` and live-tree mutation invariant across deterministically reproducible terminal statuses, with remaining fixture statuses tracked in `na-1iv`.
|
||||
- **Dream RunLoop invariant coverage** — `TestRunLoop_LiveTreeHashInvariant_AllStatuses` locks the `IsCorpusCompounded()` and live-tree mutation invariant across deterministically reproducible terminal statuses.
|
||||
- **Dream failed-summary contract coverage** — regression tests now lock the `finalizeOvernightSummary` contract for MEASURE consecutive-failure halts and persisted iteration history.
|
||||
- **Dream operator mode** — `ao overnight start|run|report|setup` adds a private overnight lane with shared `dream.*` config, keep-awake defaults, scheduler/bootstrap guidance, council-ready runner packets, and DreamScape-style morning summaries
|
||||
- **Nightly live retrieval proof** — the dream-cycle now runs `ao retrieval-bench --live --json`, emits retrieval proof in nightly summaries, and keeps a visible artifact trail for flywheel health
|
||||
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -20,8 +21,8 @@ import (
|
||||
//
|
||||
// Micro-epic 3 landed TestRunLoop_LiveTreeHashInvariant in loop_resume_test.go
|
||||
// with only the StatusDone happy-path case implemented. na-1iv asks for
|
||||
// coverage of the remaining 4 IterationStatus values. This file extends that
|
||||
// coverage via a table-driven L2 test that forces RunLoop into each status
|
||||
// coverage of the other terminal IterationStatus values. This file extends
|
||||
// that coverage via a table-driven L2 test that forces RunLoop into each status
|
||||
// deterministically (using SetTestFitnessInjector for fitness-driven paths)
|
||||
// and asserts the core invariant:
|
||||
//
|
||||
@@ -35,39 +36,20 @@ import (
|
||||
// file's helper declaration order.
|
||||
//
|
||||
// Semantic notes (from the M8 Option A consolidation council):
|
||||
// StatusDone -> corpus compounded (live tree mutated)
|
||||
// StatusDegraded -> legacy meaning: post-commit MEASURE
|
||||
// failure. Under M8 Option A the MEASURE
|
||||
// moved pre-commit, so StatusDegraded now
|
||||
// fires PRE-commit and rollback runs;
|
||||
// the live tree is NOT mutated. This
|
||||
// creates tension with the legacy
|
||||
// IsCorpusCompounded() == true mapping
|
||||
// in types.go - that predicate is
|
||||
// preserved for backward compatibility
|
||||
// with persisted pre-M8 iterations. We
|
||||
// therefore t.Skip the StatusDegraded
|
||||
// case with a pointer to this comment.
|
||||
// StatusHaltedOnRegressionPostCommit -> legacy post-M8 path only reachable via
|
||||
// pm-V7 late-stage metadata integrity
|
||||
// checks; no deterministic fault
|
||||
// injector exists for that path from
|
||||
// tests. t.Skip with the reason.
|
||||
// StatusRolledBackPreCommit -> REDUCE stage failure. Requires
|
||||
// injecting a failure into RunReduce;
|
||||
// no test hook exists yet. t.Skip.
|
||||
// StatusHaltedOnRegressionPreCommit -> strict-mode fitness regression.
|
||||
// Reproducible via the M8 pattern from
|
||||
// loop_fitness_injection_test.go.
|
||||
// Tested directly.
|
||||
// StatusFailed -> INGEST/CHECKPOINT/COMMIT error. No
|
||||
// deterministic injector from the test
|
||||
// side; t.Skip.
|
||||
// StatusDone -> corpus compounded (live tree mutated)
|
||||
// StatusDegraded -> MEASURE failed pre-commit; rollback
|
||||
// ran, live tree unchanged.
|
||||
// StatusHaltedOnRegressionPostCommit -> commit succeeded, then late metadata
|
||||
// integrity caught post-commit drift.
|
||||
// StatusRolledBackPreCommit -> REDUCE stage failure; rollback ran.
|
||||
// StatusHaltedOnRegressionPreCommit -> strict-mode fitness regression before
|
||||
// commit; rollback ran.
|
||||
// StatusFailed -> unrecoverable INGEST/CHECKPOINT/COMMIT
|
||||
// error path.
|
||||
//
|
||||
// Net result: this file locks two additional statuses beyond the pre-existing
|
||||
// StatusDone case - StatusHaltedOnRegressionPreCommit (the Option A strict
|
||||
// regression path) - with a table scaffold that documents what's pending so
|
||||
// future contributors can plug new injectors in under a single entry-point.
|
||||
// This file locks all statuses RunLoop can emit today. Rows use narrow,
|
||||
// test-only hooks where needed so production options stay free of artificial
|
||||
// fault-injection knobs.
|
||||
|
||||
// liveTreeHashInvariantCase describes a single row of the table test.
|
||||
type liveTreeHashInvariantCase struct {
|
||||
@@ -83,9 +65,12 @@ type liveTreeHashInvariantCase struct {
|
||||
// buildOpts returns the RunLoopOptions for the case. cwd and outputDir
|
||||
// are pre-filled by the driver.
|
||||
buildOpts func(cwd, outputDir, runID string) RunLoopOptions
|
||||
// skipReason, when non-empty, causes the case to t.Skip before any
|
||||
// work is done. Used to document non-reproducible statuses.
|
||||
skipReason string
|
||||
// seedPriorDone writes a prior done iteration so the next RunLoop call
|
||||
// can produce a regression status as its first new iteration.
|
||||
seedPriorDone bool
|
||||
// wantErr is true for RunLoop paths that intentionally return an error
|
||||
// after appending the terminal iteration to result.Iterations.
|
||||
wantErr bool
|
||||
}
|
||||
|
||||
// TestRunLoop_LiveTreeHashInvariant_AllStatuses drives the live-tree hash
|
||||
@@ -121,12 +106,14 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "StatusHaltedOnRegressionPreCommit_StrictRegression",
|
||||
wantStatus: StatusHaltedOnRegressionPreCommit,
|
||||
name: "StatusHaltedOnRegressionPreCommit_StrictRegression",
|
||||
wantStatus: StatusHaltedOnRegressionPreCommit,
|
||||
seedPriorDone: true,
|
||||
setup: func(t *testing.T) func() {
|
||||
// iter 1 commits at 0.9 (no prev baseline), iter 2
|
||||
// drops to 0.1 -> strict regression -> pre-commit halt.
|
||||
SetTestFitnessInjector(injectRegressionOnSecondIteration(0.9, 0.1))
|
||||
// A prior persisted StatusDone iteration supplies the 0.9
|
||||
// baseline; the first new live iteration reports 0.1 and
|
||||
// trips the strict pre-commit regression halt.
|
||||
SetTestFitnessInjector(injectConstantFitness(0.1))
|
||||
return func() { SetTestFitnessInjector(nil) }
|
||||
},
|
||||
buildOpts: func(cwd, outputDir, runID string) RunLoopOptions {
|
||||
@@ -135,7 +122,7 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
OutputDir: outputDir,
|
||||
RunID: runID,
|
||||
RunTimeout: 30 * time.Second,
|
||||
MaxIterations: 5,
|
||||
MaxIterations: 2,
|
||||
PlateauEpsilon: 0.01,
|
||||
PlateauWindowK: 2,
|
||||
RegressionFloor: 0.05,
|
||||
@@ -147,52 +134,126 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
{
|
||||
name: "StatusDegraded_MeasureFailurePreCommit",
|
||||
wantStatus: StatusDegraded,
|
||||
skipReason: "M8 Option A moved MEASURE pre-commit, so a measure failure " +
|
||||
"now triggers Rollback() and leaves the live tree unchanged. " +
|
||||
"IsCorpusCompounded() still maps StatusDegraded->true for backward " +
|
||||
"compat with persisted pre-M8 iterations, but live runs break " +
|
||||
"the hash-equals-compounded invariant. Semantic tension tracked " +
|
||||
"in types.go docstring. Deferring behavioural coverage to a " +
|
||||
"separate invariant that distinguishes legacy vs. live iterations.",
|
||||
setup: func(t *testing.T) func() {
|
||||
SetTestFitnessInjector(func(int) (FitnessSnapshot, error) {
|
||||
return FitnessSnapshot{}, errors.New("synthetic measure failure")
|
||||
})
|
||||
return func() { SetTestFitnessInjector(nil) }
|
||||
},
|
||||
buildOpts: func(cwd, outputDir, runID string) RunLoopOptions {
|
||||
return RunLoopOptions{
|
||||
Cwd: cwd,
|
||||
OutputDir: outputDir,
|
||||
RunID: runID,
|
||||
RunTimeout: 30 * time.Second,
|
||||
MaxIterations: 1,
|
||||
PlateauEpsilon: 0.01,
|
||||
PlateauWindowK: 2,
|
||||
WarnOnly: true,
|
||||
LogWriter: io.Discard,
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "StatusHaltedOnRegressionPostCommit_LegacyPath",
|
||||
wantStatus: StatusHaltedOnRegressionPostCommit,
|
||||
skipReason: "Legacy post-commit halt only reachable via pm-V7 late-stage " +
|
||||
"metadata-integrity checks under warn-only rescue paths. No " +
|
||||
"deterministic test injector exists for that path; it requires " +
|
||||
"planting a corrupt metadata artifact inside the staging tree " +
|
||||
"after commit. Follow-up fixture engineering needed.",
|
||||
setup: func(t *testing.T) func() {
|
||||
SetTestFitnessInjector(injectConstantFitness(0.8))
|
||||
SetTestPostCommitFaultInjector(func(_ int, cwd string) error {
|
||||
path := filepath.Join(cwd, ".agents", "learnings", "learning-000.md")
|
||||
return os.WriteFile(path, []byte("# Fixture\n\nNo frontmatter here.\n"), 0o644)
|
||||
})
|
||||
return func() {
|
||||
SetTestFitnessInjector(nil)
|
||||
SetTestPostCommitFaultInjector(nil)
|
||||
}
|
||||
},
|
||||
buildOpts: func(cwd, outputDir, runID string) RunLoopOptions {
|
||||
return RunLoopOptions{
|
||||
Cwd: cwd,
|
||||
OutputDir: outputDir,
|
||||
RunID: runID,
|
||||
RunTimeout: 30 * time.Second,
|
||||
MaxIterations: 1,
|
||||
PlateauEpsilon: 0.01,
|
||||
PlateauWindowK: 2,
|
||||
WarnOnly: false,
|
||||
LogWriter: io.Discard,
|
||||
}
|
||||
},
|
||||
},
|
||||
{
|
||||
name: "StatusRolledBackPreCommit_ReduceFailure",
|
||||
wantStatus: StatusRolledBackPreCommit,
|
||||
skipReason: "REDUCE-stage failure injection requires a new test hook " +
|
||||
"(analogous to testFitnessInjector) in reduce.go. Out of scope " +
|
||||
"for this bead; tracked for W1h fixture work.",
|
||||
setup: func(t *testing.T) func() {
|
||||
prev := refreshInjectCacheFn
|
||||
refreshInjectCacheFn = func(_ context.Context, stagingCwd string, _ io.Writer) (*InjectRefreshResult, error) {
|
||||
path := filepath.Join(stagingCwd, ".agents", "learnings", "learning-000.md")
|
||||
if err := os.WriteFile(path, []byte("# Fixture\n\nNo frontmatter here.\n"), 0o644); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &InjectRefreshResult{
|
||||
Attempted: true,
|
||||
Succeeded: true,
|
||||
Method: "in-process",
|
||||
Duration: time.Millisecond,
|
||||
}, nil
|
||||
}
|
||||
return func() { refreshInjectCacheFn = prev }
|
||||
},
|
||||
buildOpts: func(cwd, outputDir, runID string) RunLoopOptions {
|
||||
return RunLoopOptions{
|
||||
Cwd: cwd,
|
||||
OutputDir: outputDir,
|
||||
RunID: runID,
|
||||
RunTimeout: 30 * time.Second,
|
||||
MaxIterations: 1,
|
||||
PlateauEpsilon: 0.01,
|
||||
PlateauWindowK: 2,
|
||||
WarnOnly: false,
|
||||
LogWriter: io.Discard,
|
||||
}
|
||||
},
|
||||
wantErr: true,
|
||||
},
|
||||
{
|
||||
name: "StatusFailed_IngestOrCheckpointError",
|
||||
wantStatus: StatusFailed,
|
||||
skipReason: "INGEST/CHECKPOINT/COMMIT error injection requires a new " +
|
||||
"test hook in ingest.go or checkpoint.go. No deterministic " +
|
||||
"reproducer from the test side today.",
|
||||
setup: func(t *testing.T) func() {
|
||||
SetTestIngestFaultInjector(func(int) error {
|
||||
return errors.New("synthetic ingest failure")
|
||||
})
|
||||
return func() { SetTestIngestFaultInjector(nil) }
|
||||
},
|
||||
buildOpts: func(cwd, outputDir, runID string) RunLoopOptions {
|
||||
return RunLoopOptions{
|
||||
Cwd: cwd,
|
||||
OutputDir: outputDir,
|
||||
RunID: runID,
|
||||
RunTimeout: 30 * time.Second,
|
||||
MaxIterations: 1,
|
||||
PlateauEpsilon: 0.01,
|
||||
PlateauWindowK: 2,
|
||||
WarnOnly: false,
|
||||
LogWriter: io.Discard,
|
||||
}
|
||||
},
|
||||
wantErr: true,
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range cases {
|
||||
tc := tc
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
if tc.skipReason != "" {
|
||||
t.Skip(tc.skipReason)
|
||||
}
|
||||
t.Setenv("HOME", t.TempDir())
|
||||
restore := stubInjectRefresh(t)
|
||||
defer restore()
|
||||
|
||||
if tc.setup != nil {
|
||||
cleanup := tc.setup(t)
|
||||
t.Cleanup(cleanup)
|
||||
if cleanup != nil {
|
||||
defer cleanup()
|
||||
}
|
||||
}
|
||||
|
||||
dir := t.TempDir()
|
||||
@@ -203,30 +264,31 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
agentsDir := filepath.Join(dir, ".agents")
|
||||
runID := "hash-invariant-" + sanitizeRunID(tc.name)
|
||||
outputDir := filepath.Join(dir, ".agents", "overnight", runID)
|
||||
priorCount := 0
|
||||
if tc.seedPriorDone {
|
||||
seedPriorDoneIteration(t, outputDir, runID)
|
||||
priorCount = 1
|
||||
}
|
||||
|
||||
opts := tc.buildOpts(dir, outputDir, runID)
|
||||
|
||||
// Capture the baseline hash BEFORE RunLoop executes. Then, after
|
||||
// RunLoop returns, walk the iteration list and assert:
|
||||
// IsCorpusCompounded(lastIter) == (hashAfter != hashBefore)
|
||||
//
|
||||
// We assert on the LAST iteration only because intermediate
|
||||
// iterations cannot be observed without snapshotting between
|
||||
// them (RunLoop is a single call). For the single-iter cases
|
||||
// (StatusDone, StatusHaltedOnRegressionPreCommit via iter 2
|
||||
// halt) this is the only iter that matters; for longer loops
|
||||
// the invariant on the tail iter is what proves the predicate
|
||||
// aligns with disk state at the termination point.
|
||||
// Capture the baseline hash BEFORE RunLoop executes. Then compare
|
||||
// it to the hash after the newly emitted terminal iteration. Prior
|
||||
// seeded history, when present, is already included in the baseline.
|
||||
hashBefore, err := liveTreeHash(agentsDir)
|
||||
if err != nil {
|
||||
t.Fatalf("hashBefore: %v", err)
|
||||
}
|
||||
|
||||
result, err := RunLoop(context.Background(), opts)
|
||||
if err != nil {
|
||||
if tc.wantErr {
|
||||
if err == nil {
|
||||
t.Fatalf("RunLoop err=nil, want error for %s", tc.wantStatus)
|
||||
}
|
||||
} else if err != nil {
|
||||
t.Fatalf("RunLoop: %v", err)
|
||||
}
|
||||
if result == nil || len(result.Iterations) == 0 {
|
||||
if result == nil || len(result.Iterations) <= priorCount {
|
||||
t.Fatalf("result has no iterations: result=%+v", result)
|
||||
}
|
||||
|
||||
@@ -235,31 +297,23 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
t.Fatalf("hashAfter: %v", err)
|
||||
}
|
||||
|
||||
lastIter := result.Iterations[len(result.Iterations)-1]
|
||||
newIters := result.Iterations[priorCount:]
|
||||
if len(newIters) != 1 {
|
||||
t.Fatalf("new iteration count = %d, want 1; statuses=%s",
|
||||
len(newIters), iterStatusSummary(result.Iterations))
|
||||
}
|
||||
lastIter := newIters[len(newIters)-1]
|
||||
if tc.wantStatus != "" && lastIter.Status != tc.wantStatus {
|
||||
t.Fatalf("last iter status = %q, want %q", lastIter.Status, tc.wantStatus)
|
||||
}
|
||||
|
||||
// Core invariant for the terminal iteration. For multi-iter
|
||||
// cases where an earlier iter compounded and a later iter
|
||||
// halted pre-commit (e.g. iter 1 StatusDone + iter 2
|
||||
// StatusHaltedOnRegressionPreCommit), the live tree HAS been
|
||||
// mutated by iter 1 - so hashAfter != hashBefore even though
|
||||
// lastIter.IsCorpusCompounded() == false. We therefore assert
|
||||
// the stronger invariant: there exists at least one compounded
|
||||
// iter iff the tree changed.
|
||||
anyCompounded := false
|
||||
for _, it := range result.Iterations {
|
||||
if it.Status.IsCorpusCompounded() {
|
||||
anyCompounded = true
|
||||
break
|
||||
}
|
||||
}
|
||||
// Core invariant for the newly emitted terminal iteration.
|
||||
treeChanged := hashBefore != hashAfter
|
||||
if anyCompounded != treeChanged {
|
||||
t.Fatalf("invariant broken: anyCompounded=%v treeChanged=%v "+
|
||||
if lastIter.Status.IsCorpusCompounded() != treeChanged {
|
||||
t.Fatalf("invariant broken: status=%s compounded=%v treeChanged=%v "+
|
||||
"(hashBefore=%s hashAfter=%s). Iterations: %s",
|
||||
anyCompounded, treeChanged, hashBefore, hashAfter,
|
||||
lastIter.Status, lastIter.Status.IsCorpusCompounded(), treeChanged,
|
||||
hashBefore, hashAfter,
|
||||
iterStatusSummary(result.Iterations))
|
||||
}
|
||||
|
||||
@@ -276,15 +330,27 @@ func TestRunLoop_LiveTreeHashInvariant_AllStatuses(t *testing.T) {
|
||||
if lastIter.Status.IsCorpusCompounded() {
|
||||
t.Fatalf("StatusHaltedOnRegressionPreCommit.IsCorpusCompounded()=true; want false")
|
||||
}
|
||||
// Pre-commit halt must leave iter 2's staging discarded.
|
||||
// The earlier iter 1 (StatusDone) DID mutate the live tree,
|
||||
// so hashBefore != hashAfter is expected here - that's the
|
||||
// anyCompounded branch above. No additional hash check.
|
||||
if hashBefore != hashAfter {
|
||||
t.Fatalf("StatusHaltedOnRegressionPreCommit mutated the live tree")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func seedPriorDoneIteration(t *testing.T, outputDir, runID string) {
|
||||
t.Helper()
|
||||
iterDir := filepath.Join(outputDir, runID, "iterations")
|
||||
if err := writeIterationAtomic(iterDir, IterationSummary{
|
||||
ID: IterationID(runID + "-iter-1"),
|
||||
Index: 1,
|
||||
Status: StatusDone,
|
||||
FitnessAfter: map[string]any{"composite": 0.9},
|
||||
}); err != nil {
|
||||
t.Fatalf("seed prior iteration: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
// sanitizeRunID lowercases and replaces underscores in a subtest name so the
|
||||
// resulting runID is safe to use as a path component.
|
||||
func sanitizeRunID(name string) string {
|
||||
|
||||
@@ -262,7 +262,13 @@ func RunLoop(ctx context.Context, opts RunLoopOptions) (*RunLoopResult, error) {
|
||||
}
|
||||
|
||||
// --- INGEST ---
|
||||
ingest, ingestErr := RunIngest(loopCtx, opts, log)
|
||||
var ingest *IngestResult
|
||||
var ingestErr error
|
||||
if injector := getTestIngestFaultInjector(); injector != nil {
|
||||
ingestErr = injector(iterIndex)
|
||||
} else {
|
||||
ingest, ingestErr = RunIngest(loopCtx, opts, log)
|
||||
}
|
||||
if ingestErr != nil {
|
||||
iter.Status = StatusFailed
|
||||
iter.Error = fmt.Sprintf("ingest: %v", ingestErr)
|
||||
@@ -541,6 +547,12 @@ func RunLoop(ctx context.Context, opts RunLoopOptions) (*RunLoopResult, error) {
|
||||
fmt.Fprintf(log, "overnight: iteration %d commit failed: %v\n", iterIndex, commitErr)
|
||||
return result, fmt.Errorf("overnight: iteration %d commit: %w", iterIndex, commitErr)
|
||||
}
|
||||
if injector := getTestPostCommitFaultInjector(); injector != nil {
|
||||
if err := injector(iterIndex, opts.Cwd); err != nil {
|
||||
result.Degraded = append(result.Degraded,
|
||||
fmt.Sprintf("iter-%d post-commit fault injection: %v", iterIndex, err))
|
||||
}
|
||||
}
|
||||
|
||||
// Post-commit metadata integrity check (ratchet-forward per pm-V7).
|
||||
// Cannot unwind a successful commit; record a findings entry and
|
||||
@@ -550,6 +562,7 @@ func RunLoop(ctx context.Context, opts RunLoopOptions) (*RunLoopResult, error) {
|
||||
// stage defect (not a fitness regression).
|
||||
if postReport := VerifyMetadataRoundTripPostCommit(cp); !postReport.Pass {
|
||||
msg := fmt.Sprintf("post-commit metadata integrity: %d stripped field(s)", len(postReport.StrippedFields))
|
||||
iter.Status = StatusHaltedOnRegressionPostCommit
|
||||
iter.Degraded = append(iter.Degraded, msg)
|
||||
fmt.Fprintf(log, "overnight: iteration %d %s\n", iterIndex, msg)
|
||||
// Log a structured finding the router will intentionally skip
|
||||
|
||||
@@ -335,13 +335,11 @@ func TestRunLoop_PostCommitHalt_RehydratesAsBaseline(t *testing.T) {
|
||||
// iteration whose IsCorpusCompounded() is false MUST NOT have
|
||||
// mutated it.
|
||||
//
|
||||
// Scope excision: the full 5-status fixture engineering is tracked
|
||||
// as na-1iv. This test currently covers the StatusDone happy-path
|
||||
// case directly and documents the remaining 4 cases as pending
|
||||
// fixtures. The predicate logic is exhaustively unit-tested by
|
||||
// TestIterationStatus_IsCorpusCompounded in types_test.go; this L2
|
||||
// test proves the predicate aligns with real-world on-disk mutation
|
||||
// for the case we can deterministically produce.
|
||||
// Historical scope: this test covers the StatusDone happy-path shape that
|
||||
// originally exposed the invariant. TestRunLoop_LiveTreeHashInvariant_AllStatuses
|
||||
// covers every deterministic terminal status. The predicate logic is
|
||||
// exhaustively unit-tested by TestIterationStatus_IsCorpusCompounded in
|
||||
// types_test.go.
|
||||
func TestRunLoop_LiveTreeHashInvariant(t *testing.T) {
|
||||
t.Setenv("HOME", t.TempDir())
|
||||
restore := stubInjectRefresh(t)
|
||||
@@ -396,13 +394,7 @@ func TestRunLoop_LiveTreeHashInvariant(t *testing.T) {
|
||||
hashBefore, hashAfter)
|
||||
}
|
||||
|
||||
// Pending cases (tracked in bd issue na-1iv):
|
||||
// StatusDegraded → hash changed
|
||||
// StatusHaltedOnRegressionPostCommit → hash changed
|
||||
// StatusRolledBackPreCommit → hash UNCHANGED
|
||||
// StatusFailed (ingest/checkpoint) → hash UNCHANGED
|
||||
t.Log("Partial coverage: StatusDone case locked. " +
|
||||
"Remaining 4 status cases tracked in bd issue na-1iv.")
|
||||
// Non-StatusDone cases are covered in live_tree_hash_invariant_test.go.
|
||||
}
|
||||
|
||||
// agentsHash returns a deterministic SHA-256 over every regular file
|
||||
|
||||
@@ -68,3 +68,43 @@ func getTestFitnessInjector() func(iterIndex int) (FitnessSnapshot, error) {
|
||||
defer testFitnessInjectorMu.RUnlock()
|
||||
return testFitnessInjector
|
||||
}
|
||||
|
||||
var (
|
||||
testIngestFaultInjectorMu sync.RWMutex
|
||||
testIngestFaultInjector func(iterIndex int) error
|
||||
|
||||
testPostCommitFaultInjectorMu sync.RWMutex
|
||||
testPostCommitFaultInjector func(iterIndex int, cwd string) error
|
||||
)
|
||||
|
||||
// SetTestIngestFaultInjector installs a deterministic INGEST failure hook for
|
||||
// RunLoop tests. It is intentionally narrower than RunIngest itself: tests use
|
||||
// it to exercise RunLoop's StatusFailed bookkeeping without corrupting the
|
||||
// fixture directory or depending on filesystem timing.
|
||||
func SetTestIngestFaultInjector(f func(iterIndex int) error) {
|
||||
testIngestFaultInjectorMu.Lock()
|
||||
defer testIngestFaultInjectorMu.Unlock()
|
||||
testIngestFaultInjector = f
|
||||
}
|
||||
|
||||
func getTestIngestFaultInjector() func(iterIndex int) error {
|
||||
testIngestFaultInjectorMu.RLock()
|
||||
defer testIngestFaultInjectorMu.RUnlock()
|
||||
return testIngestFaultInjector
|
||||
}
|
||||
|
||||
// SetTestPostCommitFaultInjector installs a deterministic hook that runs after
|
||||
// Commit succeeds and before the post-commit metadata verification pass. Tests
|
||||
// use it to exercise StatusHaltedOnRegressionPostCommit without changing the
|
||||
// Checkpoint implementation or relying on nondeterministic disk faults.
|
||||
func SetTestPostCommitFaultInjector(f func(iterIndex int, cwd string) error) {
|
||||
testPostCommitFaultInjectorMu.Lock()
|
||||
defer testPostCommitFaultInjectorMu.Unlock()
|
||||
testPostCommitFaultInjector = f
|
||||
}
|
||||
|
||||
func getTestPostCommitFaultInjector() func(iterIndex int, cwd string) error {
|
||||
testPostCommitFaultInjectorMu.RLock()
|
||||
defer testPostCommitFaultInjectorMu.RUnlock()
|
||||
return testPostCommitFaultInjector
|
||||
}
|
||||
|
||||
@@ -23,9 +23,8 @@ const (
|
||||
// mode). The iteration contributed forward progress.
|
||||
StatusDone IterationStatus = "done"
|
||||
|
||||
// StatusDegraded: MEASURE failed post-commit. The corpus IS
|
||||
// compounded on disk (commit already happened), but we could not
|
||||
// compute a fitness delta for this iteration. The loop continues
|
||||
// StatusDegraded: MEASURE failed before commit. The checkpoint is
|
||||
// rolled back and the live tree is unchanged. The loop continues
|
||||
// with a stale prevSnapshot from the last fully-done iteration.
|
||||
StatusDegraded IterationStatus = "degraded"
|
||||
|
||||
@@ -100,18 +99,18 @@ func (s IterationStatus) Validate() error {
|
||||
}
|
||||
|
||||
// IsCorpusCompounded reports whether the iteration's corpus mutation
|
||||
// landed on disk. True for StatusDone, StatusDegraded, and
|
||||
// StatusHaltedOnRegressionPostCommit — all three represent states where
|
||||
// landed on disk. True for StatusDone and
|
||||
// StatusHaltedOnRegressionPostCommit — both represent states where
|
||||
// cp.Commit() succeeded before the iteration terminated. False for
|
||||
// StatusRolledBackPreCommit (no mutation happened) and StatusFailed
|
||||
// (may have partial state; RecoverFromCrash handles).
|
||||
// StatusDegraded and StatusRolledBackPreCommit (no mutation happened),
|
||||
// and StatusFailed (may have partial state; RecoverFromCrash handles).
|
||||
//
|
||||
// This is the single source of truth for rehydration logic: an iteration
|
||||
// with IsCorpusCompounded() == true is a valid prevSnapshot baseline
|
||||
// regardless of whether the loop then halted.
|
||||
func (s IterationStatus) IsCorpusCompounded() bool {
|
||||
switch s {
|
||||
case StatusDone, StatusDegraded, StatusHaltedOnRegressionPostCommit:
|
||||
case StatusDone, StatusHaltedOnRegressionPostCommit:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
|
||||
@@ -31,17 +31,17 @@ func TestIterationStatus_Validate(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// TestIterationStatus_IsCorpusCompounded locks the truth table: done,
|
||||
// degraded, and halted-on-regression-post-commit all have corpus on
|
||||
// disk; pre-commit rollback and failed do not. This is the rehydration
|
||||
// predicate; if it drifts, resume semantics break.
|
||||
// TestIterationStatus_IsCorpusCompounded locks the truth table: done and
|
||||
// halted-on-regression-post-commit have corpus on disk; degraded,
|
||||
// pre-commit rollback, and failed do not. This is the rehydration predicate;
|
||||
// if it drifts, resume semantics break.
|
||||
func TestIterationStatus_IsCorpusCompounded(t *testing.T) {
|
||||
cases := []struct {
|
||||
s IterationStatus
|
||||
want bool
|
||||
}{
|
||||
{StatusDone, true},
|
||||
{StatusDegraded, true},
|
||||
{StatusDegraded, false},
|
||||
{StatusHaltedOnRegressionPostCommit, true},
|
||||
{StatusRolledBackPreCommit, false},
|
||||
{StatusFailed, false},
|
||||
|
||||
+5
-1
@@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
|
||||
- **Native Windows smoke gate** — `tests/windows/test-windows-smoke.ps1` now exercises the PowerShell `ao` and Codex installers, local `ao doctor` Windows guidance, and focused Windows-sensitive Go tests, with a blocking `windows-smoke` job in `validate.yml`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- **Dream RunLoop status invariants** — live-tree hash coverage now exercises every terminal RunLoop status, and `degraded` reflects the current pre-commit MEASURE rollback semantics.
|
||||
|
||||
## [2.36.0] - 2026-04-11
|
||||
|
||||
### Added
|
||||
@@ -19,7 +23,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
||||
- **Autodev program contract** — root `PROGRAM.md` gives evolve/autodev a repo-local operating contract with mutable and immutable scope, validation commands, escalation policy, and stop conditions.
|
||||
- **Beads stale-scope tooling** — `ao beads verify|lint|harvest` adds first-class stale-citation checks for bead-driven planning and RPI recovery.
|
||||
- **RPI discovery artifacts** — RPI can now persist and consume discovery artifacts, with tests and docs covering the `--discovery-artifact` path.
|
||||
- **Dream RunLoop invariant coverage** — `TestRunLoop_LiveTreeHashInvariant_AllStatuses` locks the `IsCorpusCompounded()` and live-tree mutation invariant across deterministically reproducible terminal statuses, with remaining fixture statuses tracked in `na-1iv`.
|
||||
- **Dream RunLoop invariant coverage** — `TestRunLoop_LiveTreeHashInvariant_AllStatuses` locks the `IsCorpusCompounded()` and live-tree mutation invariant across deterministically reproducible terminal statuses.
|
||||
- **Dream failed-summary contract coverage** — regression tests now lock the `finalizeOvernightSummary` contract for MEASURE consecutive-failure halts and persisted iteration history.
|
||||
- **Dream operator mode** — `ao overnight start|run|report|setup` adds a private overnight lane with shared `dream.*` config, keep-awake defaults, scheduler/bootstrap guidance, council-ready runner packets, and DreamScape-style morning summaries
|
||||
- **Nightly live retrieval proof** — the dream-cycle now runs `ao retrieval-bench --live --json`, emits retrieval proof in nightly summaries, and keeps a visible artifact trail for flywheel health
|
||||
|
||||
@@ -147,17 +147,18 @@ Schema v2 (2026-04-09) introduces the compounding iteration loop. It is **additi
|
||||
|
||||
### Status Precedence Truth Table
|
||||
|
||||
The `IterationSummary.Status` field uses an exhaustive five-value enum. Each value has distinct semantics that downstream consumers (morning report, rehydration logic, invariant tests) depend on.
|
||||
The `IterationSummary.Status` field uses an exhaustive enum. Each value has distinct semantics that downstream consumers (morning report, rehydration logic, invariant tests) depend on.
|
||||
|
||||
| Status | Commit succeeded? | Corpus on disk? | Rehydration baseline? | Typical trigger |
|
||||
|---|---|---|---|---|
|
||||
| `done` | yes | yes | yes | Happy path — all stages succeeded, fitness delta non-regressing |
|
||||
| `degraded` | yes | yes | yes | MEASURE failed post-commit; no fitness delta available but corpus compounded |
|
||||
| `degraded` | no | no (unchanged) | no | MEASURE failed pre-commit; checkpoint rolled back before the live corpus changed |
|
||||
| `rolled-back-pre-commit` | no | no (unchanged) | no | REDUCE failed before commit; checkpoint was rolled back |
|
||||
| `halted-on-regression-post-commit` | yes | yes | yes | Post-commit regression check fired; corpus is in live tree but loop halted |
|
||||
| `halted-on-regression-pre-commit` | no | no (unchanged) | no | Fitness regression or plateau halt fired before commit; checkpoint was rolled back |
|
||||
| `failed` | partial/no | indeterminate | no | Unrecoverable error in INGEST/CHECKPOINT/COMMIT; RecoverFromCrash handles partial state on restart |
|
||||
|
||||
**Invariant:** an iteration with `status ∈ {done, degraded, halted-on-regression-post-commit}` is a valid rehydration baseline for `prevSnapshot` on resume, because the compounded corpus is on disk. Statuses `rolled-back-pre-commit` and `failed` are NOT valid baselines — rehydration walks past them.
|
||||
**Invariant:** an iteration with `status ∈ {done, halted-on-regression-post-commit}` is a valid rehydration baseline for `prevSnapshot` on resume, because the compounded corpus is on disk. Statuses `degraded`, `rolled-back-pre-commit`, `halted-on-regression-pre-commit`, and `failed` are NOT valid baselines — rehydration walks past them.
|
||||
|
||||
**Companion marker file:** when an iteration has `status = halted-on-regression-post-commit`, Dream writes a sentinel file `committed-but-flagged.iter-<N>.marker` in the same `<outputDir>/<runID>/iterations/` directory. Operators can find flagged iterations via directory listing without parsing every iter-<N>.json.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user