This reverts commit 0cf3b3ca09.
57 KiB
PROPOSAL: Pluggable Storage Backends
Status: Historical — partially implemented, then superseded on 2026-07-16
Date: 2026-07-02
Informed by: the bts-rs Rust spike (a from-scratch Rust reimplementation
of the gc-contract surface that validated a backend-agnostic storage seam at 299/299 byte-parity
across two radically different backends), a five-dimension recon of this repo's storage
architecture, and a two-reviewer adversarial verification pass. All file:line citations verified
against worktree-beads-new.
Superseded scope: This proposal is retained as the design record for the storage interface, shared issue core, SQLite adapter, and differential conformance work that grew from it. Those pieces remain. The direct PostgreSQL and MySQL adapters were rolled back before entering a tagged release because supporting additional general-purpose server databases adds substantial dialect, credential, schema-lifecycle, migration, CI, and operational complexity. Our goal is to keep Beads as simple as possible and consume as few resources as possible. The supported paths are now embedded Dolt, Dolt server, and SQLite; Dolt server's use of the MySQL wire protocol is unchanged. See Storage Backends for the current design.
Everything below this notice is retained as historical design material, not as current backend support or migration guidance.
1. End state
bd init --backend=dolt|sqlite|postgres— the user picks the engine at init. The workspace locator records the choice; opening consults the locator, never the environment.- One core command set (the gc-contract 16 plus the portable CORE inventory, ~67 commands) behaves identically on every backend, proven by a differential conformance harness — not by code review.
- Storage-specific commands are capability-gated addons.
bd history,bd dolt push/pull,bd vc,bd branch,bd diff,bd federation,bd backup,bd sqlare live only when the opened store advertises the matching capability; otherwise they are deterministic stubs that explain what backend provides them (Decision D2, resolved: stubs, not hidden — see §6). Pick Dolt → you get history and remotes. Pick SQLite → you get a fast, zero-dep, pure-Go single-writer store and those commands stub out. - Dolt remains the default and richest backend. Back-compat guarantee: a workspace created before this migration keeps opening exactly as it does today — enforced by an explicit legacy-locator rule (§4.3), not by hope.
- Adoption path for existing workspaces:
bd export→ re-init on the new backend →bd import, with documented fidelity limits (§5, Phase 4 gate). In-place cross-backend data migration is a separate initiative.
2. What the Rust spike proved (the transferable payload)
Evidence: bts-rs/docs/ (SEAM_VALIDATION_FINDINGS.md, CONFORMANCE_GAPS.md,
RED_TEAM_FINDINGS.md, HISTORY_ON_YUGA_DESIGN.md, PERF_PG_VS_YB.md, REMOTE_DEPLOYMENT.md).
2.1 The seam shape
Six small core traits (33 data methods) + optional capability traits reached via accessors defaulting to "absent". Two backends (Postgres, file-backed memory) hit byte-parity with real bd on 299/299 scenarios with zero command-handler changes — handlers never name a backend.
Go gets the spike's hardest problem for free. Rust's non-object-safe async traits forced a
399-line delegation enum; in Go, storage.Storage is already an interface and Open() returns
the interface value. Budget the port on semantics and conformance, not plumbing.
Seam-size caveat (from review): the spike's 33-method core is NOT evidence that beads' much larger core (§4.1: ~107+24 methods after the fold) is cheap to implement per-backend. The spike's "~1–10k LOC per adapter" anchor was measured against 33 methods. Phase 0 therefore includes a core-shrink review: which core methods are truly primitive vs derivable-above-the-seam vs capability material. Sizing in §8 assumes the core does NOT shrink (worst case).
2.2 The four seam-leak rules (each was a real bug in the spike)
- Construction is ON the seam.
Open(locator, identity)/Init(backend, ...)are seam methods; otherwise every entry point names concrete backends. - Identity is ON the seam — actor identity is construction-time config in the seam package.
- The locator is backend-neutral and self-describing (
{backend, workspace_id}); backend is read FROM the locator at open;--backend/env applies only at init. - Capabilities OPTIMIZE core behavior; they never GATE it. For every capability use inside a core command there must be a core-interface fallback, OR the whole command is capability-gated. (The spike's only runtime divergence violated this.)
2.3 Transactional-integrity rules that apply to beads' CORE (not just capabilities)
The spike's red team hit this bug class three times; in beads it lives on the core surface:
- Same-tx side effects: beads writes the events table inside the mutation transaction
(
issueops/create.go:601,labels.go:151,helpers.go:147; read via coreGetAllEventsSince,storage.go:84, andbd audit). Any new backend must do the same — events emitted outside the mutation tx strand or duplicate audit rows on crash. - Denormalized
is_blockedis the most conformance-dangerous core semantic. It is maintained across 10+ issueops write paths (dependencies/close/reopen/delete/promote/…) and read by ready queries (sqlbuild/ready.go:96). Delete/purge must recompute surviving neighbours. A fresh backend reimplements this propagation on every mutation; the harness must carry required transition fixtures (close-blocker→ready, purge-orphan re-seed).
2.4 The conformance oracle
- Differential, not static: scenarios as real processes in throwaway workspaces; volatile
normalization (
<TS>/<UUID>/actor); per-step exit + stderr + JSON-aware stdout diff. A static canonicalized corpus "pins a formatter, not a system". - Multiset array diffing by default, with a per-scenario
orderedflag valid only where ordering is implementation-independent. - Property tests for non-byte-diffable paths (minted IDs,
--graph). - Concurrency proofs as capability gates: the 24-process
ready --claimdrain applies only to backends advertising multi-writer. - Epistemic honesty — what green does NOT prove (§7 risk table): hook-firing semantics
(post-commit, rollback-suppressed — invisible to stdout/stderr diffing), equal-priority and
wall-clock-hybrid ordering (
ready's 48h recency rule), and anything outside the in_scope predicate. Each gate below pins its in_scope coverage explicitly so "proven identical" is never claimed beyond what ran. - Operational: unique candidate-binary path per run (
cpover an exec-mapped binary fails silently and scores a stale binary).
2.5 History without Dolt (later, not now)
bd history's minimal substrate is a per-issue, ordered, never-pruned field-delta changelog
written in the same tx as each mutation + HistoryViewer{History, AsOf, Diff} as a backward
fold. Dolt provides all three natively (dolt_history_*, AS OF, dolt_diff). So "pick Dolt →
get history" can later relax to "any backend with the Changelog capability" without
re-architecting.
2.6 Backend guidance from measured perf
Postgres bounded ops flat to 100k (list ~2ms). Remote cost = round-trips × RTT plus ~6 RTT of
connection setup per command in the shell-out model (~300ms pure overhead at 50ms RTT) —
mitigations: co-location, local PgBouncer ≥1.21 transaction mode, daemon endgame. Collapse write
paths to 1–2 statements; FOR UPDATE SKIP LOCKED IS the claim guarantee. YugabyteDB ≈ 2–3×
latency / 10× write cost — multi-region only. An embedded pure-Go store is the cheapest second
backend and the one that validates the seam.
3. Where beads is today (recon summary, review-corrected)
3.1 The good news — the seam is already 80% drawn
storage.DoltStorage(internal/storage/storage.go:200-213) composes coreStorage(62 methods) + 11 sub-interfaces (144 total; +Transaction24). The Dolt-shaped surface is already isolated in 5 named sub-interfaces — VersionControl (16), HistoryViewer (3), RemoteStore (12), SyncStore (2), FederationStore (4) = 37 methods, ~87 genuine call sites in non-test cmd/bd. The other 6 sub-interfaces are backend-neutral (BulkIssueStore, DependencyQueryStore, AnnotationStore, ConfigMetadataStore, CompactionStore, AdvancedQueryStore — ~45 methods, ~155 cmd/bd call sites serving CORE commands likeready --claim,dep,comment) and fold INTO the core seam (§4.1).- The capability-gating pattern already ships: 10 optional interfaces
(
storage.go:217-287),UnwrapStore+ type-assert at 22 cmd/bd sites, graceful errors (sql.go:54-57). - The public API is not frozen to the god interface: root
beads.go:20aliasesStorage = beads.Storage(=storage.Storage); RemoteStore/SyncStore are exported as separately assertable interfaces. The retype does not break the public type surface — but see §4.4 for the publicOpen*constructors and the extension-compat note. - Real prior art for the oracle:
tests/regression(built for the SQLite→Dolt migration; found 70+ bugs) contributes workspace isolation, binary management, container TestMain, and a normalizer that already encodes cross-backend representational divergences (regression_test.go:587local-vs-UTC midnight;:647metadata{}vs absent). Its execution core does NOT transfer (§5 Phase 1). - Integrations are portable for free (all 6 trackers bind to narrow
storage.Storage); the tracker engine already demonstrates capability-with-fallback (tracker/engine.go:695-717). - Dialect exposure is concentrated: ~90% of SQL flows through
issueops/sqlbuild/domain/db. Dolt-specific SQL inside issueops sits in 7 files:as_of.go,commit_pending.go,diff.go,federation.go,blocked_merge.go,history.go(dolt_history_issues), andblocked_consistency.go(dolt_statusguard — backend-internal, moves with the backend). The first six map 1:1 onto gateable capabilities.
3.2 The hazards — what actually blocks a second backend
- H1. Two competing seams, mid-flight — and BOTH are alive. The
DoltStoragepath covers 100% of commands. The uow/domain layer covers 13 commands via*_proxied_server.goduals and is an ACTIVE workstream (one command per PR since 2026-05-10, latest merges 2026-06-24; the init gate atinit.go:135-140is deliberate staging per PR #4488, and the CI lane is dark only until the gate lifts). It is Dolt-flavored at its Tx layer (uow/doltserver_tx.go:28=CALL DOLT_COMMIT), duplicates issueops (kept honest by Seam-A parity tests), and is already feature-diverging from the embedded path (--offsetis proxied-only). Proxied-server mode exists ONLY on the uow path —newDoltStorerefuses it (store_factory.go:48-53). D1 must resolve the seam relationship AND proxied's fate, with the uow workstream owner participating (§6). - H2. Dolt lifecycle is fused into every command's write path, with real protocol structure.
Not just "auto-commit in PostRun":
transactHonoringAutoCommit(dolt_autocommit.go:28-47) blanks the commit message to suppress mid-tx commits depending on mode; a non-blank message + success setscommandDidExplicitDoltCommitwhich suppresses PostRun auto-commit (main.go:1223-1227); the tips system defers metadata writes and PostRun issues a SECOND commit for them (tips.go:154-183,main.go:1233-1257); auto-export/backup freshness readsGetCurrentCommitAFTER the commit; auto-push gates and error semantics differ from auto-commit; the autocommit-mode default itself is chosen by the CLI fromusesSQLServer()(main.go:1099-1106).commitPendingIfEmbedded/transactHonoringAutoCommithave 21 call sites across 18 files. This is a protocol, not a hook — it gets a specified contract (§4.2) and its own phase (2b). - H3. Global store + FIVE construction paths.
store storage.DoltStorage(main.go:45; 159 non-test refs / 61 files, 15 more test files). Paths: PreRun choke point; lazydirect_mode.go; read-only; ~13 files opening routed/extra stores; and the public API — rootbeads.go:63-73Open/OpenFromConfighard-codedolt.New*(+beads_nocgo.goOpenBestAvailable), used by external Go extensions. The registry must own all five. TheHookFiringStoredecorator EMBEDS DoltStorage (hook_decorator.go:28-29) and is applied only on the main path today — the registry must own decorator application or lazy/read-only/routed opens keep silently skipping hooks. - H4. The migration stream is not portable. 53 tracked + 10 dolt-local
*.up.sql, MySQL dialect; migrations 0019/0028/0040/0041 executedolt_ignore/CALL DOLT_COMMITinline. New backends need per-backend migration sources (theschemarunner's two-stream design is the extension point). - H5. Core-signature Dolt leaks.
RunInTransaction(ctx, commitMsg, fn)— where the message is load-bearing suppression semantics per H2, NOT advisory;GetCurrentCommitas freshness token (export_auto.go:85,backup_auto.go:127);ApplyCompaction(commitHash);bd edit's raw*sql.DBkeepalive (edit.go:150); "dolt-ignored" LocalMetadata contract. - H6. Command metadata in name-keyed string lists.
noDbCommands(27),readOnlyCommands(14), dolt-subcommand patch lists (main.go:770-808,122-137). - H7. bd doctor — 37/45 files Dolt-tinged, raw SQL on
dolt_remotes/dolt_status. - H8. Mode-divergent semantics are unformalized. The draft claimed this "includes a LIVE
BUG":
with server-mode autocommit OFF,CORRECTED AFTER VERIFICATION: REFUTED. The claimed "live bug" does not exist. The server-mode store (GetCurrentCommitnever advances, so auto-export/backup freshness checks silently stop firing in server mode.internal/storage/dolt.DoltStore) commits to HEAD per write method, unconditionally — independent of--dolt-auto-commitand ofmaybeAutoCommit(dolt/issues.go:49CreateIssue →CALL DOLT_COMMIT; UpdateIssue/CloseIssue/DeleteIssue likewise;RunInTransaction→versioncontrolops.StageAndCommit). So in non-proxied server mode HEAD advances on every persistent write,DOLT_HASHOF('HEAD')changes, and the auto-backup freshness check DOES fire. Separately, auto-export never even reaches the freshness check in server mode — it early-returns (export_auto.go:38-41,if serverMode { return nil }). ThemaybeAutoCommitmachinery governs the EMBEDDED working-set-persistence flow, not the server store's per-method commits; the H8 draft was written against an inaccurate model of the server store. The rest of H8 stands: auto-commit ON embedded / OFF server; single-writer flock embedded-only (store_factory.go:66-85); server DoltStore has real streaming iterators (dolt/iter_issues.go:48) while embedded shipsNewSliceIterstubs;bd sql/bd adminwork in server mode but error in embedded (sql.go:45,admin.go:17) — and that embedded error string is gc contract class 3. - H9. Auto-import is a silent data-mutation path on the core surface (missed in the first
draft; review-added).
maybeAutoImportJSONLruns in PreRun for every command (main.go:1163-1164), takesstorage.DoltStorage, auto-imports JSONL into empty databases, and has a documented data-clobber history (auto_import_upgrade_unit_test.go:74-87). Its "auto-importing … into empty database" strings are gc error class 4 (the write-loss guard). A second backend that is empty-on-fresh-open is exactly the trigger condition; if its strings differ, gc's guard goes blind. - H10. Env precedence can override the locator at OPEN.
BEADS_DOLT_SERVER_MODE=1is precedence #1 over metadata.json (configfile.go:232-243); shared-server env/config overrides pinneddolt_modewith a Notice (main.go:247);--globalhard-requires shared-server (main.go:1071-1076). Parsing is scattered (configfile, doltserver, init, bootstrap). The spike's "locator wins at open" rule must extend to topology, with an explicit ownership and inertness rule (§4.3).
3.3 The command surface (census, corrected)
~271 cobra commands (~111 top-level), all registered unconditionally in package init() —
before any workspace resolution (this kills naive hide-at-registration; see D2). Classified:
CORE ≈ 67 (incl. the gc-contract 16), STORAGE-SPECIFIC = 11 (+3 mixed: bd gc, bd migrate,
bd restore's history fallback), INTEGRATION = 6 families, INFRA ≈ 27. 12 of the gc-16 have
dual (store+uow) implementations; purge, count, version, stats do not — and purge
is precisely where the spike's red team found re-seeding bugs, so it is a required Phase 1
scenario, not a "covered elsewhere" case.
4. Target architecture — the clean-room object model, and the bridge to it
Two things are described here and it matters to keep them distinct: §4.0 the destination (the clean-room object model we would build greenfield, which the migration accretes toward) and §4.5 the bridge (the flat-interface waypoint the mechanical phases pass through). The destination is what new code is written AGAINST from Phase 4 onward; the bridge is scaffolding that de-risks the retype and is thinned afterward (Phase 7).
4.0 The clean-room object model (the destination)
┌────────────────────────────────────────────────────────────────────────┐
│ cmd/bd — commands, formatting, streams, contract strings │
│ command registry: declared metadata {needsStore, readOnly, │
│ requires: Capability} — replaces the name-keyed string lists (H6) │
└──────────────────────────────┬─────────────────────────────────────────┘
│ consumes ONLY:
┌──────────────────────────────▼─────────────────────────────────────────┐
│ package store — THE SEAM (small, frozen, documented) │
│ Driver / Registry / Locator / Identity (construction, rules 1–3) │
│ Store / Tx + ~7 domain interfaces (core; Tx-first) │
│ Capability interfaces + CapabilitySet (optional surface, rule 4)│
│ Error taxonomy + retry classes (typed, rendering-free) │
│ storetest.RunContract(t, driver) (the compliance suite) │
└──────┬───────────────┬───────────────┬───────────────┬─────────────────┘
│ │ │ │
backend/dolt backend/postgres backend/sqlite backend/fake
(topologies: (multi-writer; (pure-Go; (in-memory/file;
embedded | no history v1) single-writer; unit-test keystone)
server | fills nocgo)
proxied
INTERNAL)
The core, sketched (signatures indicative, not final):
// construction — the spike's leak rules 1–3, verbatim
type Driver interface {
Open(ctx context.Context, loc Locator, id Identity) (Store, error)
Init(ctx context.Context, opts InitOptions) (Locator, error)
}
type Locator struct{ Backend string; Raw json.RawMessage } // self-describing; env only at Init
type Identity struct{ Actor, Email string } // on the seam; injectable for tests
type Store interface {
// Run is the PRIMARY seam op: one retryable unit of work. The seam owns
// phase-aware retry (Serialization -> bounded auto-replay of fn;
// CommitIndeterminate -> surfaced, never auto-retried; fn must be replayable,
// no external I/O while the Tx is open). Semantics lifted from uow/run_in_tx.go.
Run(ctx context.Context, op OpInfo, fn func(Tx) error) error
Begin(ctx context.Context, op OpInfo) (Tx, error) // low-level escape hatch
Capabilities() CapabilitySet // per-INSTANCE (topology-aware)
Limits() Limits // {MaxAtomicTxBytes, MaxValueBytes, MaxTxDuration,
// ConsistentScanBudget} — CLI pre-flights + chunks instead
// of discovering backend ceilings as commit failures
ChangeToken(ctx context.Context) (string, error) // equality-only freshness cursor
Close() error
}
// OpInfo + CommandScope are where the H2 commit protocol collapses to. NOTE
// (red-team blocker): commit messages are OUTCOME-derived in bd (composed after
// close/claim results — close_proxied_server.go:242-255, uow.RunInTxMsg), so the
// description travels on Commit, NOT on Begin. A CommandScope above Tx owns the
// one-version-commit-per-command rule (N txs -> ONE dolt commit), the tips
// deferred-metadata second commit, and PostWritePush ordering.
type OpInfo struct{ Command, Actor string; ReadOnly bool }
// the transaction is the first-class object — one per command/logical op
type Tx interface {
Issues() Issues // CRUD, search, ready, claims; wisps as a TIER param, not a
Deps() Dependencies // parallel ×2 method family
Labels() Labels
Comments() Comments
Events() EventReader // reads; event WRITES happen implicitly inside mutations, same tx
Config() Config
Commit(ctx context.Context, desc string) error // desc outcome-derived;
// Dolt: DOLT_COMMIT(desc); PG: COMMIT; reads: Rollback, never Commit
Rollback(ctx context.Context) error
}
// capabilities: extend core behavior, never gate it (rule 4)
type CapabilitySet struct {
History, VersionControl, RemoteSync, Backup, RawSQL bool
// MultiWriter bool was too coarse (red-team, 2 lenses independently):
Concurrency struct {
ConcurrentReaders, ConcurrentWritersSafe bool
WriterExclusion string // enforced-lock | lease | none
ContentionModel string // row | workspace-CAS
}
}
func History(s Store) (HistoryViewer, bool)
type HistoryViewer interface{ /* History, AsOf, Diff */ }
// + VersionControl, RemoteSync, Backup, RawSQL, Maintenance(GC/Flatten/Compact)
// errors: typed below the seam, rendered above it
var ErrNotFound, ErrClaimConflict error
type ErrUnsupported struct{ Op, Backend string }
type RetryClass int // Transient | Serialization | CommitIndeterminate | Permanent
Every commitment traces to a scar in the current system or the spike:
| Commitment | Scar it heals |
|---|---|
Tx-first seam (Begin → domain ifaces → Commit) |
DoltStore's per-method commits = the stale-snapshot data-loss class (#3822) that forced the May pivot; command-scoped short tx is the proven concurrency unit. This is uow.UnitOfWork promoted to the seam. |
| ~7 small domain interfaces, no god interface | the 805-line panic wall (PR #3792); "can't easily mock DoltStorage"; 168 methods before one command works |
OpInfo on Begin |
H2: commit protocol currently lives in CLI globals (message-blanking, commandDidExplicitDoltCommit, tips double-commit) |
ChangeToken() on core |
backend-neutral freshness: GetCurrentCommit/DOLT_HASHOF is Dolt-specific, so export/backup freshness must key on a neutral opaque token any backend can supply (SQLite/Postgres have no Dolt commit hash). (NB: the earlier "H8 live bug" rationale is withdrawn — see H8; ChangeToken stands on backend-neutrality alone.) |
per-instance CapabilitySet, extend-never-gate |
bd sql works on Dolt-server, stubs on Dolt-embedded — capability varies by topology; the spike's only runtime divergence was a capability gating core behavior |
topologies INSIDE backend/dolt |
embedded/server/proxied are three connections to ONE engine; CLI knowledge of them produced 38 scattered mode checks |
| events written inside mutations, same tx | the spike's red team hit emit-outside-tx three times; the use-cases already do it right — make it structural + contract-tested |
backend/fake + storetest.RunContract |
nothing tests against the interface today; every suite needs a Dolt container |
| wisps as a tier param | today every use-case method is duplicated ×2 (41 where ~25 would do); the --offset union-paging divergence is a symptom |
| slices, not streaming iterators, in v1 | all 10 Iter* methods are slice stubs in the embedded backend anyway; streaming returns later as a measured need |
One-sentence summary: the clean-room seam is Dustin's UnitOfWork widened to the full
domain surface, wearing bts-rs's construction/capability/error discipline.
4.1 Bridge-seam arithmetic (the flat waypoint; review-corrected)
Core = today's Storage (62) + the six backend-neutral sub-interfaces (~45) = ~107 methods,
Transaction(24), all mandatory for every backend. The ~155 cmd/bd call sites of the six folded interfaces need zero conversion. The ~87 call sites of the 5 Dolt-shaped interfaces convert to capability asserts.DoltStorageis deleted; the Dolt backends keep avar _compile assertion for core + all capabilities. Phase 0's core-shrink review may pull some of the ~107 above the seam (derivable) — treat any shrink as upside, not plan.
4.2 The WriteLifecycle contract (replaces the naive "PostWrite hook")
A specified seam protocol, designed in Phase 2b BEFORE code moves:
WriteSummary{didWrite, explicitCommitOccurred, readOnlyCommand, messageParts, deferredMeta}carried per command (absorbs today'scommandDidWrite/commandDidExplicitDoltCommitglobals and the tips deferred-write protocol).- Two seam calls bracketing the CLI's own export/backup steps:
PostWriteCommit(summary)(Dolt: auto-commit incl. the second tips commit; others: no-op) then — after the CLI's auto-export/auto-backup, whose freshness check consumes a neutral ChangeToken() (core, with the capability-shortcut-plus-fallback shape; motivated by backend-neutrality, sinceDOLT_HASHOFis Dolt-specific — NOT by the withdrawn H8 "live bug", see H8) —PostWritePush(summary)(Dolt: auto-push with its swallowed-error semantics; others: no-op). - In-transaction suppression (
transactHonoringAutoCommit's blank-message trick) becomes an explicit seam behavior: the backend decides mid-tx checkpointing from the summary + its own mode; the message parameter onRunInTransactionstays but is re-specified as "operation description; versioned backends MAY use it as a commit message" (D3). - Autocommit-mode POLICY (today chosen by the CLI from
usesSQLServer()) moves into the Dolt backend's topology config — the CLI no longer knows modes exist.
4.3 Locator, legacy rule, and env precedence
- Locator in
.beads/metadata.json:{backend: "dolt"|"sqlite"|"postgres", ...}.dolt_modebecomes Dolt-internal topology config. - Legacy-value rule (review blocker): the existing
backendfield is UNTRUSTED legacy — workspaces in the wild may carrybackend:"sqlite"from the pre-removal era (removed in87493ce91), andGetBackend()'s hard-return of "dolt" (configfile.go:207-210) is a deliberate shim over exactly that. Rule: introduce a NEW locator key (e.g.storage_backend, plus a metadata schema version); absent/legacy/unknown values + presence of a Dolt data dir resolve to dolt, never to a new backend. A Phase 1 harness fixture with a legacybackend:"sqlite"metadata.json pins this. This closes the trap where a legacy value resolves to the new SQLite backend, opens an empty store, and H9 auto-import silently re-seeds/forks the workspace. - Env precedence:
BEADS_DOLT_*(19 vars) are owned by the Dolt backend and are inert for non-Dolt locators; env parsing consolidates behind the registry; the Phase 1 harness pins the precedence table (H10).--global/shared-server: the CLI keeps a seam-level "workspace redirection" concept whose only current provider is Dolt shared-server topology — on SQLite/Postgres,--globalerrors with a capability stub (D2 pattern).
4.4 Errors, formatting, and the public API
- Typed errors below the seam (NotFound/ClaimConflict/Unsupported{op,backend}/retry-class);
ALL envelope/stream routing and contract strings above it — including gc error class 3
(
bd sqlembedded string), class 4 (H9 auto-import strings — now backend-neutral pinned contract), and the claim-conflict text. - Public API:
beads.Open*route through the registry (H3 fifth path). Compat note for extension authors: on non-Dolt workspaces, type-assertingRemoteStoreetc. on the returnedStoragestarts returning false — document in release notes; the assertion pattern itself (beads.go:27-31) is unchanged. - Single binary, no dynamic plugins. "Addons/packs" = capability-gated stubs inside one binary
- build tags where deps demand (embedded Dolt is cgo-only; modernc SQLite is pure-Go → nocgo
builds gain a local backend for the first time,
beads_nocgo.go:27).
- build tags where deps demand (embedded Dolt is cgo-only; modernc SQLite is pure-Go → nocgo
builds gain a local backend for the first time,
4.5 Bridge vs destination — how the two relate
The mechanical phases (2a/2b/3) retype the CLI onto the FLAT core interface (§4.1) because
that keeps a 159-reference migration mechanical. That flat interface is scaffolding, not
the destination: from Phase 4 onward, the uowStore adapter's inner surface is built as
the PUBLIC clean-room seam (§4.0 Tx + domain interfaces) rather than as a private adapter
detail — so the destination accretes underneath the migration instead of requiring a second
one. Phase 7 then thins the flat layer opportunistically (or keeps it as sugar). Existing
components map as: uow.UnitOfWork/use-cases/domain/db → the seam's embryo;
EmbeddedDoltStore/DoltStore/issueops → backend/dolt interior; the 13 duals,
DoltStorage, the string lists, and the Seam-A parity tests → deleted.
4.6 Future-backend expandability (FDB, redb-class embedded KV, S3)
The seam must not assume SQL — two of the three named future backends are key-value and one is object storage. Design constraints this imposes, to be validated by adversarial review:
- FoundationDB: interactive transactions exist but carry hard limits (~5s / ~10MB);
bd import-scale work cannot be oneTx. The seam therefore must NOT promise unbounded transaction size — bulk operations need an explicit chunked/bulk path on the seam, and the contract suite must pin what atomicity bulk ops actually guarantee. Ordered scans and secondary indexes are backend-built (the bts-rs readiness-projection pattern applies). - redb-class embedded KV: single-writer ACID — same shape as the SQLite slot. (redb itself is a Rust library; in-process use from Go implies cgo/FFI or it arrives via the bts-rs lineage — the seam question is only "can an embedded KV implement the contract", and the spike's file-backed mem store already answered yes.)
- S3: no multi-key transactions. A conforming backend is manifest-CAS shaped (commit =
conditional-PUT of a new manifest;
Commitreturns a Serialization retry class on CAS loss) — which theTx+RetryClassmodel can express, but claim latency and contention behavior must be honestly capability-flagged (likelyMultiWriter: falseor claims-unsupported). S3 may be BETTER fitted as the target of RemoteSync/Backup/cold history (the bts-rs BlockStore/RefStore substrate is S3-native with CAS on refs) than as a primary Store; the seam should permit both without contortion.ChangeTokenmaps naturally (manifest ETag).
4.7 Red-team revisions (Fable panel, 5 lenses, all verdicts holds-with-fixes) — NORMATIVE
These contract additions override/extend §4.0 where they conflict; fold into the sketch on the next full edit. Convergent = demanded independently by 2+ lenses.
Convergent (adopted):
Store.Run(closure)is the primary op; commands migrate ONTO it in Phase 7 (not off it). Retry ownership lives on the seam; typed errors gain backend-suppliedRetryAdvice(backoff hint + budget); storetest includes an error-injecting fake and requires that a conforming driver's advice lets 24 contending claimers converge (no livelock).- CommandScope above Tx (see §4.0 note): one version-commit per command, tips second commit, commit→backup/export→push ordering; dolt-log divergence scenarios join the Phase 2b gate (stdout diffing alone cannot see N-commits-instead-of-one).
- Read-consistency contract: statements within one Tx observe ONE snapshot (PG: REPEATABLE READ or single-statement collapse; the two-phase ready-union read is the motivating bug); within-Tx read-your-writes is contract-tested; read families are classified single-snapshot-required (ready, claim-candidate, mutation-feeding reads) vs chunk-consistent-permitted (export, audit, stats) — FDB's ~5s limit applies to READS too.
- Bulk/chunked port with pinned atomicity: per-chunk atomic, idempotent upsert,
resumable journal; crash-mid-bulk + auto-import-resume (H9) fixtures in Phase 1;
create --graphatomicity resolved againstLimits(). - Wisp/tier respec: tier is a DERIVED attribute (ephemeral/no_history/infra-config
routing — never caller-chosen); query methods take visibility flags, not a tier enum;
explicit cross-tier ops (PromoteFromEphemeral, AffectedByDeletion); tiered-atomicity
contract (Commit atomic per tier, pinned ordering,
PartialCommiterror class orAtomicTiersflag); a tier×capability locality matrix (History/RemoteSync/Backup/export MUST exclude local-tier data) with fixtures: purge-spares-no_history, backup-excludes-wisps. All of this feeds D5. - Ordering/predicate semantics become a written contract: binary-codepoint collation
(PG DDL pins COLLATE "C"), second-precision timestamps, full tiebreak chains incl.
NULL-first, LIKE-wildcard/escape behavior, metadata
{"k": null}counts as present; clock ownership moves client-side (aClockprovider joins Identity on the construction seam) so relative-time predicates and the 48h-hybrid ordering are deterministic + contract-testable.
Single-lens (adopted):
7. Mutation-observation contract (repo lens, blocker): Tx.Commit yields a
MutationSummary (feedable from the same-tx event rows) consumed by a registry-owned
hook dispatcher; the decorator wraps Tx, not just Store; hook-artifact fixtures join the
Phase 1 harness; Phase 7 conversions gate on hook parity.
8. Interleaving rule: capability methods may not run between Begin and Commit/Rollback
on the same Store (typed error, storetest-pinned); per-topology connection budgets are
documented backend-author contract (embedded's eager second dolt_ignore connection binds
lazily or every command pays issue-#4303's dominant cost).
9. Sync/archive TARGETS are a first-class seam concept distinct from Store (S3 lens):
BlockStore{Put,Get}/RefStore{Get,CAS} target interfaces + per-scheme drivers
(s3/gs/file); CapabilitySet.RemoteSync rescoped to backend-NATIVE remotes (Dolt);
Phase 6 cold history and portable backup-to-URL route through targets — S3's best fit,
without pretending it is (or isn't) a primary Store. Backends whose commit protocol can
strand garbage (CAS-race orphans) MUST implement + declare Maintenance.GC; doctor
surfaces it.
10. Error taxonomy additions: ErrBusy (Transient; the embedded flock string is its
pinned rendering); claim errors carry structured holder/status (the gc class-2 string
is rendered ABOVE the seam from data, incl. idempotent self-reclaim success);
Open/Begin blocking-vs-fail-fast behavior specified per concurrency class; backends MAY
resolve CommitIndeterminate internally (verify-after-timeout) and return a definite
class.
11. ChangeToken contract: opaque, equality-only, data-changed ⇒ token-changed
(false positives permitted); Dolt = hash over non-ignored core tables (NOT HEAD);
per-tier advance semantics pinned; S3 = seq embedded in the manifest, not raw ETag;
FDB = versionstamped last-mutation key.
12. Testing-story honesty (repo lens): backend/fake + storetest get their own phase,
gate, and sizing row; evaluate a go-mysql-server-backed fake running the REAL SQL
(reuse, not a fifth semantics copy); Seam-A parity-suite deletion re-gated on
single-SQL-stack convergence inside backend/dolt (embedded TxProvider over the embedded
engine → delete issueops), not on dual-file collapse; §3.1's "integrations are free" is
WRONG — the tracker keys its Dolt fast path on DB() presence and must key on the
History capability (same audit for every RawDBAccessor consumer: SQL-speaking ≠
has-Dolt-system-tables).
13. Streaming: unbounded consumers (audit, sync, export) keep a cursor/since+limit
path (or a StreamingReads capability) before Phase 7 converts them — "slices only" is
bounded-by-workspace materialization, stated as such.
§4.6 corrections from the future-backend lenses: FDB limits apply to reads too (~5s
read-version expiry; 100kB value cap vs bd's unbounded LONGTEXT — chunked content or
declared Limits); the readiness-projection pattern transplants only with rework (chunked
catch-up outside the op Tx, lease-based projector); redb: what the spike validated is "an
embedded single-file backend behind the seam for the gc-16 surface" — the 9-sort-order
index design, ~107-method surface, and cross-process behavior remain unvalidated, and redb
has no C API (a Go binding does not exist; that backend arrives via FFI work or the bts-rs
lineage); S3 claims stay CORE (never "claims-unsupported") — the drain gate keys on
Concurrency.ConcurrentWritersSafe with a disclosed contention model; 412/409/503/timeout
→ RetryClass mapping table to be added.
5. Migration phases
Each phase lands green with the default backend byte-identical; each has a verification gate that names its in_scope coverage.
Phase 0 — Decisions + contract pinning (docs only)
- Resolve D1–D6 (§6) with maintainers.
- Leak-4 audit over capability-sourced behavior in core commands (restore's history
fallback; export/backup freshness → ChangeToken;
bd gc's portable decay phase; tracker fast path). Classification defines the core command set. - Core-shrink review (§2.1 caveat): triage the ~107 core methods into primitive / derivable-above-seam / capability.
- Census additions from review: H9 auto-import; the 21
commitPendingIfEmbeddedsites; the seven issueops Dolt files; the WriteLifecycle protocol inventory (H2). - Pin H8 behavior contracts (incl. the iterator and autocommit divergences) into
BackendCapabilities(configfile.go:183-204). - Gate: signed-off decision doc; core-vs-capability census table; WriteLifecycle contract spec reviewed.
Phase 1 — The oracle before the surgery
A new differential runner reusing tests/regression scaffolding (workspace isolation,
binary management, container TestMain, the battle-tested normalizer) — NOT an extension of its
execution core: today's runner is 177 Go closures compared by end-state snapshot only, t.Fatals
on any non-zero exit, and merges stdout+stderr (regression_test.go:363-374,811-831). New:
per-step recorder (exit code, split streams, JSON-aware diff), replayable scenario format,
multiset-default + ordered flag, in_scope predicate, backend/mode parameter on workspace
setup, property tests for minted IDs, the claim drain (multi-writer only), unique
candidate-binary paths.
- Scope: gc-contract 16 × their flags first (incl. purge — no uow dual, red-team history), plus fixtures: legacy-locator (§4.3), auto-import class-4 trigger (H9), close-blocker→ready and purge-orphan re-seed transitions (§2.3), the env-precedence table (H10).
- Shake-out: Dolt-embedded vs Dolt-server. Premise (review-corrected): these are KNOWN DIVERGENT (H8) — the run formalizes the divergences, it does not validate on identical engines. Allowlist discipline: entries are backend-PAIR-scoped, field-scoped, tagged intentional-vs-known-bug, and expire with the Phase 2 scrub. Mode waivers do not carry into SQLite runs. Expect the autocommit asymmetry to be the dominant waiver class. (The earlier "expect to confirm the H8 live bug" line is withdrawn — H8's server-mode export-freshness bug was refuted on verification; the server store commits per write and auto-export is server-mode-disabled.)
- Gate: harness green on Dolt-vs-Dolt with a fully attributed allowlist; coverage report artifact (scenarios × commands × flags) checked in.
Phase 2a — Neutralize the core seam (the mechanical part)
- Fold the six neutral sub-interfaces into core (§4.1); retype the global
store+ decorator +molecules.NewLoader+remotecache.StoreOpener+ helpers fromDoltStorageto the core interface (159 refs / 61 files + 15 test files); convert the ~87 Dolt-shaped call sites to UnwrapStore asserts; deleteDoltStorage. - Consolidate the FIVE construction paths (incl. public
beads.Open*) into the registry; registry owns decorator application (closing the existing lazy/read-only/routed hook gap); new locator key + legacy rule (§4.3). - Convert H6 string lists to declared per-command properties.
- Gate: full suite + Phase-1 harness Dolt-vs-Dolt: zero NEW divergences vs the Phase-1 allowlist.
Phase 2b — The WriteLifecycle migration (the semantic part; own phase, own gate)
- Implement the §4.2 contract; migrate the 21
commitPendingIfEmbedded/transact*call sites, PersistentPostRunE, the tips deferred-write protocol, ChangeToken for export/backup freshness (on backend-neutrality grounds — the "fixes the silent server-mode export-freshness bug" rationale is withdrawn; that bug was refuted on verification, so ChangeToken is a portability requirement, not a bug fix, and needs no behavior-change release note), retype H9 auto-import and pin its strings. - Gate: Phase-1 harness re-run; the ONLY acceptable diffs are the enumerated bug fixes; autocommit/push/export ordering pinned by new dedicated scenarios.
Phase 3 — Capability-gate the addon surface
Capabilities()is instance-level (computed after open, backend × topology — review fix:bd sqlworks on Dolt-server but not Dolt-embedded, same backend). Gating happens at RunE-time via stubs (D2 resolved): registration stays ininit()(no bootstrap restructuring;bd --helpand the CI docs-regen stay deterministic); absent capability → deterministic stub error naming the backend that provides it (federation_nocgo precedent;bd sql's embedded string preserved verbatim as contract).- Surface:
bd dolt(16 subcmds),bd vc,bd branch,bd history,bd diff,bd show --as-of,bd federation,bd backup,bd sql,bd flatten, Dolt legs ofbd compact/bd gc,db-proxy-child,--global(§4.3). bd doctor: per-backend check registry (D6); Dolt checks (incl.blocked_consistency'sdolt_statusguard) move into the Dolt backend's set.- Audit + flip the
--backend=sqlitedeprecation contract (init.go:215-225hard-errors today; docs/scripts written against that string need a coordinated, release-noted flip). - Gate: on Dolt, behavior +
--helpbyte-identical; a test-only capability-less backend exercises every stub; docs regen green.
Phase 4 — Second backend: Postgres (THE PROOF backend; reordered per owner direction)
Owner-stated success criterion: beads runs on a Postgres backend and the bd CLI simply doesn't have the history functionality. Postgres — not SQLite — is therefore the seam validator and the critical-path deliverable. (The first draft's SQLite-first ordering followed the spike's cheapest-second-backend guidance; the owner's proof criterion overrides it.)
Two candidate build routes — choosing is part of D1, because they hinge on the uow relationship:
- Route A (recommended if D1 lands the adapter): ride the uow stack. A Postgres
TxProvider(plain BEGIN/COMMIT; the commit-message param is ignored) + a dialect port of thedomain/dbrepositories (~75 methods of EXISTING, Seam-A-parity-tested SQL; the MySQL-isms are enumerable —ON DUPLICATE KEY→ON CONFLICT, backticks, noCALL DOLT_*)- the
uowStoreadapter + the five gap units + per-backend DDL. This REUSES the already-implemented dangerous semantics (same-tx events,is_blockedpropagation) instead of re-deriving them, and leans on the workstream's own stated invariant (issue #4489): "the local-vs-proxied difference is confined to the uow transaction layer (the mode-polymorphic db.Runner)."
- the
- Route B (fallback, if D1 keeps the layers separate): direct core implementation
(~107+24 methods over a fresh schema), cribbing the bts-rs PG adapter wholesale (schema,
order/ready indexes + batched hydration, single-statement claim via
FOR UPDATE SKIP LOCKED, typed retry classes, optional readiness projection). Independent of the uow roadmap; more code; a THIRD implementation of the business semantics to keep honest — the hard part is semantics, not SQL (§2.3: same-tx events,is_blockedpropagation, delete/purge neighbour recompute, wisp/tier semantics, D5 local-tables).
Proof-wedge milestone (demoable before Phases 2–3 fully land): minimal locator backend
field + one factory arm + the PG stack + a TEMPORARY full-DoltStorage compat shell whose 37
Dolt-shaped methods return typed Unsupported errors (graceful errors, NOT panics — the D1
design-history lesson) + PersistentPostRun neutralized for non-Dolt. This demonstrates "bd on
Postgres, bd history absent" without waiting for the 159-ref retype; the shell is deleted
when Phase 2a lands. Accepted sharp edges for the demo: auto-export/backup freshness and
parts of bd doctor degrade on the wedge; the gc-16 harness subset is the demo's acceptance
test.
- Gate: Phase-1 harness PG-vs-Dolt green over the gc-16 in_scope set, THEN widened to the CORE-67 surface before the backend is announced (coverage artifact enforces the widening — no shipping green-on-16-only). Transition fixtures (§2.3) green. 24-process claim drain (PG advertises multi-writer). Remote deployment guidance shipped — per-command connection-setup cost measured, PgBouncer/co-location documented (§2.6).
Phase 5 — Third backend: SQLite (the nocgo/embedded story; now optional)
Pure-Go (modernc.org/sqlite), documented single-writer (claim-drain gate N/A); fills the
nocgo hole (beads_nocgo.go:27 — non-CGO builds currently have NO local backend). Under
Route A it rides the same production line (a SQLite TxProvider + dialect port); under
Route B, a direct implementation with the WAL lessons (synchronous=NORMAL,
wal_autocheckpoint=0 + explicit checkpoint cadence, pragmas on the read pool, no unclosed
rows). Slice-iterator stubs are legal (embedded Dolt ships them; server streams — both
satisfy the interface).
- Gate: as Phase 4 minus the claim drain; single-writer lock UX tested. Export→import adoption-path fidelity documented (events history, wisps, LocalMetadata do NOT obviously round-trip — measure and state).
Phase 6 — History as a portable capability (optional)
Changelog capability (same-tx field-delta log) + HistoryViewer-as-fold (§2.5). Dolt keeps its
native implementation. Mirror the spike-documented wire quirks (bare JSON array, PascalCase
keys, full snapshots, client-side --limit, volatile CommitHash normalized).
Phase 7 — The end-game: promote the clean-room seam, thin the bridge (opportunistic)
Migrate commands off the flat conveniences onto Begin/Tx/Commit directly — file by file,
no flag day, each conversion deleting a RunInTransaction wart — until the flat layer is
thin enough to delete or small enough to keep as sugar. No deadline: the system is already
multi-backend and capability-gated before this phase starts. Exit criterion: the store
package's godoc IS the backend-author contract, and storetest.RunContract is the only
thing a new backend must pass before the differential harness.
Explicitly deferred / out of scope
- In-place cross-backend DATA migration (
bd migrate --to-backend=X) — the interim adoption path is export→re-init→import with Phase-4-documented fidelity limits. - Retiring the uow layer's 13 dual command paths (fate fixed by D1; execution follows Phase 3).
- YugabyteDB/FDB backends; dynamic plugin loading; backend-neutral federation.
6. Decisions required
- D1 — Canonical seam AND proxied-server's fate (one decision, not two). Corrected after
git-history review: the uow/domain layer is NOT dormant — it is an active workstream
(Dustin Brown / coffeegoddd, DoltHub), converting one command per PR since 2026-05-10 with
merges through 2026-06-24 (#4039–#4488: provider, list, update, delete, show, close, reopen,
ready, dep, config, context, query). The init gate is deliberate staging ("lifting the gate
is a separate, deliberate change", PR #4488), not abandonment. Its motivation is
multi-process concurrency: embedded Dolt is single-writer; a shared server + short
per-command transactions serves concurrent agents. "Freeze uow" is therefore not a viable
recommendation.
Design-history fact that reframes this decision:
a
storage.DoltStorageimplementation over the proxy WAS built first —DoltServerStore, PR #3792, May 6–7 2026 — and died in four days as an 805-line wall ofpanic("unimplemented")(128/132 methods), pivoted to the UOW design with only a two-line TODO as rationale. It died because the god interface cannot be implemented incrementally from scratch — a cause the uow workstream's own six weeks of use-case building has since removed (the adapter delegates to them), and which Phase 2's interface shrink + capability split removes for every future backend. "We tried a store over the proxy already" is therefore true but not dispositive. Recommended reconciliation — compose, don't pick:storage.Storageremains the backend seam (what the CLI consumes; what SQLite/Postgres implement; where the capability/addon model lives — use-cases have no answer for command gating). Build the missinguowStoreadapter (coreStorageimplemented overUnitOfWorkuse-cases; today no such adapter exists andstore_factory.go:51/96/169TODOs point the other way — this is the crux to align with the uow owner). Consequences: proxied-server becomes a topology of the Dolt backend behind the registry; the 13*_proxied_server.goduals collapse; the domain repositories overdb.Runner(3 methods) become an ACCELERATOR for future SQL backends (supply Runner+TxProvider+dialect, reuse the repos) rather than a rival seam; theTx.Commit=CALL DOLT_COMMIT(uow/doltserver_tx.go:28) adopts the §4.2 WriteLifecycle contract one layer down. Full sketch with the method-by-method mapping and gap analysis (headline: ~60% of core maps 1:1 to existing use-case methods; the true remaining cost of full proxied coverage is five small gap units, not 54 more command conversions). The alternative (full use-case trajectory): commands migrate to use-cases everywhere and backends supply repositories. Coherent, but: ~54 more command conversions, dual implementations + parity tests for the duration, non-SQL backends become impossible (Runnerassumes SQL), and the capability model is still needed regardless. Divergence governance either way: the two paths already diverge in features (--offsetis proxied-only because the embedded issues+wisps union cannot page — PR #4488) — assign an owner for keeping the paths semantically converged until they reunify, and add the divergences to the Phase 1 allowlist as tagged, expiring entries. This decision requires the uow workstream owner in the room. - D2 — Absent-capability UX. RESOLVED by review: stubs, not hidden. Cobra registration
happens in package
init()before workspace resolution; hiding requires pre-cobra locator resolution and breaks help/docs determinism. Stubs preserve the gc error contract and the federation_nocgo precedent. (If hiding is ever wanted, it is its own sized work item.) - D3 —
RunInTransactionmessage param: keep, re-specified per §4.2 (the blank-message suppression semantics become explicit seam behavior; "advisory" was wrong). - D4 — MergeSlot*/Slot*: keep in core (bead-backed, portable); revisit when Postgres lands (Phase 4) — real row locking may obviate them.
- D5 — Wisps' dolt-local tables on backends without an ignore concept: table-prefix convention (recommended) vs separate file/schema. Feeds the Phase 4 schema design.
- D6 — Doctor scoping: per-backend check registry (recommended) vs neutral check interfaces.
7. Risks
| Risk | Mitigation |
|---|---|
Two-seam divergence widens while D1 is undecided (uow conversions merge weekly; --offset-class proxied-only features accumulate) |
D1 decided WITH the uow owner in Phase 0; until then, new proxied features carry a convergence note + Phase 1 allowlist entry; the uowStore adapter collapses the duals once agreed |
Legacy backend:"sqlite" metadata resolves to the new backend and auto-import forks data |
§4.3 legacy rule (new locator key, dolt-default fallback) + Phase 1 fixture — was a review blocker, now designed-out |
| Silent drift during the Phase 2a retype | Harness EXISTS FIRST; zero-new-divergence gate; commit per file-cluster |
| WriteLifecycle migration changes semantics invisibly | Own phase (2b) + dedicated ordering scenarios; only enumerated bug-fix diffs accepted |
| Allowlist waivers rot and mask real regressions | Pair-scoped, field-scoped, tagged, expiring entries; no carryover to new-backend runs |
| Green harness over-trusted | §2.4 epistemics: hooks/timing/coverage limits stated per gate; coverage artifact required; CORE-67 widening enforced before Phase-4 announce |
| A from-scratch backend (Route B) re-opens the spike's transactional bug class | §2.3 rules are Phase 4/5 implementation requirements with mandatory transition fixtures; Route A sidesteps by reusing the parity-tested use-case semantics |
bd doctor/backup partial coverage confuses users |
Capability-aware stubs + doctor prints per-backend coverage |
| Public-API extensions break on non-Dolt workspaces | §4.4 compat note; assertions return false (defined behavior), release-noted |
| Schema drift between per-backend migration streams | Shared logical-schema contract suite each backend's fresh DB must pass |
8. Sizing (rough, sequential; review-adjusted)
| Phase | Size | Notes |
|---|---|---|
| 0 | ~1 week | decisions + leak-4 audit + core-shrink review + WriteLifecycle spec |
| 1 | 2–3 weeks | new runner on old scaffolding; gc-16 scope + fixtures (was 1–2; review: per-step core is new code) |
| 2a | 2–3 weeks | fold + retype (159 refs/61 files) + registry (5 paths) + metadata properties |
| 2b | 1–2 weeks | WriteLifecycle contract migration (semantic; own gate) |
| 3 | 1–2 weeks | instance-level capabilities + stubs + doctor registry + deprecation flip |
| 4 | Route A: 4–6 weeks / Route B: 5–8 weeks | Postgres proof: A = PG TxProvider + repo dialect port (~75 methods of existing SQL) + adapter + 5 gap units + DDL; B = ~107-method direct core + is_blocked/events semantics from scratch + schema; both + conformance to CORE-67 |
| 5 | 2–4 weeks (Route A) / 4–7 (Route B) | SQLite riding the same production line (A) or direct (B); nocgo hole |
| 7 | opportunistic | thin the flat bridge onto the §4.0 seam; no deadline |
| 6 | 2–4 weeks | optional; independent — and under Route A, the Changelog capability could later give PG history too (§2.5), making "no history on Postgres" a v1 statement, not a permanent one |
Phases 4/5 are independent of each other; 6 independent of both. Critical path to the owner's PROOF ("bd on Postgres, no history"): 0 (D1 + route choice) → 1 (gc-16 harness subset) → Phase-4 proof wedge (~6–9 weeks). Critical path to the full end state ("users choose a backend", shipped properly): 0 → 1 → 2a → 2b → 3 → 4 (~12–19 weeks sequential); the wedge work is on that path, not thrown away — only the compat shell is deleted when 2a lands.
Appendix: review provenance
Recon: 5 parallel read-only explorers (storage interface census; Dolt leak census; command
surface census; existing-abstraction assessment; bts-rs learnings extraction) — ~578k tokens,
311 tool calls. Adversarial pass: 2 reviewers (feasibility-vs-repo, completeness-vs-spike),
verdicts sound-with-fixes; 2 blockers, 8 majors, 5 minors, 13 missing items — all folded into
this revision. Notable review discoveries retained above: the six-neutral-interfaces blocker
(§4.1), the legacy-locator trap (§4.3), the WriteLifecycle protocol structure (§4.2/H2), the
instance-level capability requirement (Phase 3), the H8 live bug (server-mode export
freshness) (corrected after verification: refuted — the server store commits per write and
auto-export is server-mode-disabled; ChangeToken is retained on backend-neutrality grounds),
and the H9 auto-import hazard. Clean-room shred: 5-lens Fable panel
(object-model, repo-reality, FDB, embedded-KV, S3) — all verdicts holds-with-fixes; 2
blockers (outcome-derived commit messages vs OpInfo-at-Begin; hook-observation contract), 17
majors; all folded as §4.7.