mirror of
https://github.com/gastownhall/beads.git
synced 2026-09-14 20:17:24 +08:00
test(protocol): golden-JSON contract corpus + diff guard (Beads↔Gas City contract, Phase 2) (#4490)
* test(protocol): add canonicalized golden-JSON contract corpus + diff guard (contract Phase 2)
Producer half of the Beads<->Gas City cross-version contract-test system.
Generates a deterministic, canonicalized golden-JSON corpus of bd's --json
wire surface (create/show/list/ready/dep/count/version + the
{error,schema_version} envelope, in both flat and BD_JSON_ENVELOPE=1
variants) and fails on any unreviewed diff. Gas City vendors this corpus
to detect cross-version drift without a live bd.
- corpus.go: deterministic command plan, canonicalizer (timestamps -> <TS>,
object arrays stable-sorted, sorted keys), provenance manifest.
- corpus_test.go: TestCorpusGolden (byte-compare vs committed corpus;
-corpus.update regenerates; Dolt-boot is an infra skip, a diff is a hard
fail). Pins IDs with --force for determinism; captures stdout only.
- canonicalize_test.go: canonicalizer unit tests + TestCorpusDoubleRunByteIdentical.
- make corpus-regen; cmd/bd/protocol/CATALOG.md.
Pre-commit hook bypassed: it fails in golangci-lint (built with Go 1.25,
repo targets 1.26.2) unrelated to this change; gofmt, go vet, and the full
corpus test suite were run manually and pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: gate the contract corpus in beads CI (contract-corpus job)
Adds a Docker-enabled contract-corpus job to the PR gate that regenerates
the canonicalized golden corpus from this branch's bd and byte-compares it
to the committed testdata/corpus/, plus the determinism double-run. An
unreviewed bd --json wire change is now a hard CI failure on the producer
side, mirroring the gascity consumer gate. Wired into ci-gate/CI_GATE_REQUIRED.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(protocol): drop build-provenance from the corpus so it is reproducible across build envs
The contract-corpus CI gate caught a real reproducibility bug: bd version
--json embeds a 'commit' field from Go's VCS stamping that varies by build
environment (a local worktree build omits it; a CI clean-clone build embeds
the SHA), so the committed golden corpus failed the diff in CI. Canonicalization
now drops the 'commit' key entirely (value AND presence vary), and the manifest
no longer records bd_commit. bd_version plus the per-blob checksums remain the
reproducible provenance.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* ci: publish the contract corpus as a release artifact (contract Phase 3)
Ships cmd/bd/protocol/testdata/corpus as beads_<version>_contract_corpus.tar.gz
with each release and appends its checksum to the release checksums.txt, so Gas
City can anchor its vendored-corpus drift-check to a signed beads release rather
than a self-recomputed manifest. Runs in the goreleaser job before the macOS
job's serial checksums.txt append (goreleaser-macos needs goreleaser), so the
corpus entry is preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(protocol): extend the contract corpus to the full command surface
Adds update, close, reopen, delete, dep remove, and two more create beads to
the corpus plan, taking it to 17 commands x {flat, envelope}. New shapes pinned:
the update/close/reopen issue arrays (incl. close_reason + closed_at), the
dual-key dep edge {issue_id, depends_on_id, type, status} for add AND remove,
the delete confirmation {deleted, dependencies_removed, references_updated}, and
metadata coercion (--set-metadata phase=2 -> integer 2) plus label round-trip.
Mutations run after the show/dep_list reads so those long-standing blobs stay
byte-identical; list/ready/count now reflect post-mutation state. Golden +
double-run determinism verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(protocol): make the corpus version-agnostic (placeholder version/branch/build)
The contract-corpus gate failed when beads main bumped to 1.1.0-rc.1: the corpus
version.json pinned the literal bd version, which changes every release (CI builds
the PR merged with current main). The corpus pins wire SHAPES, not release
identity, so canonicalization now replaces version/branch/build values with
placeholders and the manifest drops bd_version/bd_commit entirely. schema_version
(the coordination canary) + per-blob checksums remain. gc's real version
parsing/gating is covered by the cross-version matrix + bd_version_pin_test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(protocol): satisfy build-tag policy + misspell lint for contract corpus
Pre-review CI repair for the golden-JSON contract corpus PR (#4490):
- Makefile: add -tags "$(BUILD_TAGS)" to the corpus-regen target so it
passes scripts/check-build-tags.sh (ICU build-tag policy, see
docs/ICU-POLICY.md), matching every other go test/build target.
- corpus.go: fix "re-marshalled" -> "re-marshaled" (misspell, US locale).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(protocol): harden the contract corpus per maintainer review
Maintainer review fixups for the golden-JSON contract corpus (PR #4490),
folded into one maintainer commit across review iterations.
- CanonicalizeJSON now rejects trailing stdout after the first JSON value.
A command that printed valid JSON followed by a warning line (or a second
JSON value) would otherwise be silently truncated to the leading value and
pass the corpus gate, while a real consumer decoding the whole stream would
choke. Add regression tests for trailing text, multiple values, and the
still-allowed trailing-whitespace case.
- TestCorpusGolden now asserts the committed corpus file set is exactly the
generated blobs plus manifest.json, and `make corpus-regen` recreates each
mode directory before writing. A capture removed or renamed in CorpusPlan can
no longer leave a stale blob committed and shipped in the release archive.
- The required contract-corpus CI job sets BEADS_PROTOCOL_REQUIRE_DOLT=1 and
the corpus tests hard-fail (not skip) when the Dolt store is unavailable, so
the gate cannot report success without exercising the golden and double-run
checks.
- Document the intentionally global (bare-key) scope of provenance
canonicalization and the required future action if a non-version command adds
a commit/version/branch/build field.
- Make the release corpus-checksum append idempotent so a re-run of the
goreleaser job does not leave a duplicate row in checksums.txt.
- generateCorpus now pins each capture's subprocess exit status instead of
discarding cmd.Run(): every capture must exit 0 except the dedicated error
capture, which must exit non-zero with bd's not-found code. Otherwise the
corpus could pass while a "successful" command exited non-zero, or while the
error capture stopped failing, leaving the exit-codes-and-errors half of the
error blob's contract (CATALOG.md) unpinned. Add a checkCaptureExit helper
and TestCheckCaptureExit, and run the test in the required CI job.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: CI Bot <ci@beads.test>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Julian Knutsen
@@ -475,6 +475,44 @@ jobs:
|
||||
BEADS_TEST_BD_BINARY: ${{ github.workspace }}/ci-build-artifacts/bd-linux-gms-pure
|
||||
run: go test -tags gms_pure_go -race -count=1 -v ./internal/storage/domain/... ./internal/storage/uow/... ./internal/tracker/...
|
||||
|
||||
# Producer-side guard for the Beads<->Gas City contract corpus. Regenerates
|
||||
# the canonicalized golden corpus from this branch's bd and byte-compares it
|
||||
# to the committed testdata/corpus/; an unreviewed wire change is a hard
|
||||
# failure (run `make corpus-regen`). The double-run test proves determinism.
|
||||
# Needs the Dolt sql-server image (the protocol harness spins a container).
|
||||
contract-corpus:
|
||||
name: Contract corpus (golden + determinism)
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v6
|
||||
|
||||
- name: Set up Go
|
||||
uses: actions/setup-go@4a3601121dd01d1626a1e23e37211e3254c1c06c # v6
|
||||
with:
|
||||
go-version-file: 'go.mod'
|
||||
|
||||
- name: Install Dolt CLI
|
||||
run: curl -fsSL https://github.com/dolthub/dolt/releases/latest/download/install.sh | sudo bash
|
||||
|
||||
- name: Configure Git and Dolt identity
|
||||
run: |
|
||||
git config --global user.name "CI Bot"
|
||||
git config --global user.email "ci@beads.test"
|
||||
dolt config --global --add user.name "CI Bot"
|
||||
dolt config --global --add user.email "ci@beads.test"
|
||||
|
||||
- name: Pull Dolt sql-server image
|
||||
# Keep tag in sync with internal/testutil/testdoltcommon.go:DoltDockerImage.
|
||||
run: docker pull dolthub/dolt-sql-server:2.1.0
|
||||
|
||||
- name: Corpus golden + determinism
|
||||
# Require a live Dolt store here: this is the gate that must exercise the
|
||||
# golden + double-run checks, so a failed container is a hard failure,
|
||||
# not a silent skip (see requireDoltStore in cmd/bd/protocol tests).
|
||||
env:
|
||||
BEADS_PROTOCOL_REQUIRE_DOLT: "1"
|
||||
run: go test -tags gms_pure_go -count=1 -run 'TestCorpusGolden|TestCanonicalize|TestCorpusDoubleRunByteIdentical|TestCheckCaptureExit' ./cmd/bd/protocol
|
||||
|
||||
fmt-check:
|
||||
name: Check formatting
|
||||
runs-on: ubuntu-latest
|
||||
@@ -525,6 +563,7 @@ jobs:
|
||||
- pr-core-wrapper
|
||||
- pr-lint-wrapper
|
||||
- test-domain-uow
|
||||
- contract-corpus
|
||||
- fmt-check
|
||||
- lint
|
||||
if: ${{ always() }}
|
||||
@@ -550,6 +589,7 @@ jobs:
|
||||
PR_CORE_WRAPPER
|
||||
PR_LINT_WRAPPER
|
||||
TEST_DOMAIN_UOW
|
||||
CONTRACT_CORPUS
|
||||
FMT_CHECK
|
||||
LINT
|
||||
BUILD_ARTIFACTS: ${{ needs.build-artifacts.result }}
|
||||
@@ -567,6 +607,7 @@ jobs:
|
||||
PR_CORE_WRAPPER: ${{ needs.pr-core-wrapper.result }}
|
||||
PR_LINT_WRAPPER: ${{ needs.pr-lint-wrapper.result }}
|
||||
TEST_DOMAIN_UOW: ${{ needs.test-domain-uow.result }}
|
||||
CONTRACT_CORPUS: ${{ needs.contract-corpus.result }}
|
||||
FMT_CHECK: ${{ needs.fmt-check.result }}
|
||||
LINT: ${{ needs.lint.result }}
|
||||
run: |
|
||||
|
||||
Reference in New Issue
Block a user