fix(coding-agents): not every source tag names a kind of claim

Running per_source on real repositories showed two of the five source tags are
provenance labels rather than axes, and each produced a scope worth less than it
cost.

source:git-log is a bookkeeping alias. git.ts tags the commit-message seed with
it AND source:git, so emitting a scope for each gave two near-identical belief
sets — 307 observations against 302 on one repo — and doubled the consolidation
for that pair. It is the same claim as source:git: what the commits say. Mapped
onto it rather than dropped, because that seed is where a cold repo's entire
commit history arrives.

source:survey-baseline marks the "researching…" status document, whose retain
strategy is meant to extract nothing. It still yielded a scope holding exactly
one observation — a belief set that exists only to be noise. Excluded.

Emitting a scope per DISTINCT source tag, sorted, is still right: taking the
first made the result depend on the order git.ts assembled its tags, which a
test pins. It is the vocabulary that carries two non-semantic entries, not the
rule.
This commit is contained in:
Shaun Eccles
2026-08-29 16:25:40 +10:00
parent a9a729f79c
commit 0786de87ba
5 changed files with 76 additions and 8 deletions
@@ -557,6 +557,13 @@ carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
`tags: [], tags_match: "exact"`.
Two `source:` tags are provenance labels rather than kinds of claim, and are handled specially —
both found by running this on real repositories. `source:git-log` maps onto `source:git`: the
commit-message seed carries both, so a scope for each produced two near-identical belief sets (307
observations against 302 on one repo) for double the consolidation. `source:survey-baseline` is
excluded outright: it marks the "researching…" status document, whose retain strategy extracts
nothing, and it yielded a scope holding a single observation.
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
unconditional — it is not filtered against the memory's own tags — so a configured
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
@@ -560,6 +560,13 @@ carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
`tags: [], tags_match: "exact"`.
Two `source:` tags are provenance labels rather than kinds of claim, and are handled specially —
both found by running this on real repositories. `source:git-log` maps onto `source:git`: the
commit-message seed carries both, so a scope for each produced two near-identical belief sets (307
observations against 302 on one repo) for double the consolidation. `source:survey-baseline` is
excluded outright: it marks the "researching…" status document, whose retain strategy extracts
nothing, and it yielded a scope holding a single observation.
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
unconditional — it is not filtered against the memory's own tags — so a configured
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
@@ -362,6 +362,13 @@ carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
`tags: [], tags_match: "exact"`.
Two `source:` tags are provenance labels rather than kinds of claim, and are handled specially —
both found by running this on real repositories. `source:git-log` maps onto `source:git`: the
commit-message seed carries both, so a scope for each produced two near-identical belief sets (307
observations against 302 on one repo) for double the consolidation. `source:survey-baseline` is
excluded outright: it marks the "researching…" status document, whose retain strategy extracts
nothing, and it yielded a scope holding a single observation.
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
unconditional — it is not filtered against the memory's own tags — so a configured
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
@@ -262,17 +262,37 @@ describe("HindsightClient.retain — per_source scoping", () => {
expect(item.observation_scopes).toEqual([[]]);
});
// The commit-message seed is tagged `source:git` AND `source:git-log` (git.ts keeps both: the
// cold-repo check filters on `source:git`). Taking every distinct source tag is the only rule
// that needs no arbitrary tie-break and does not depend on the order git.ts happens to emit.
it("gives a document carrying two source tags a scope for each", async () => {
// The commit-message seed carries `source:git` AND `source:git-log` (git.ts keeps
// both so the cold-repo check can find it). `git-log` is a bookkeeping alias for
// the same kind of claim — what the commits say — not a second axis. Emitting a
// scope for each produced two near-identical belief sets on a real repository:
// 307 observations against 302, for double the consolidation.
it("folds the commit-log alias into the git scope rather than duplicating it", async () => {
const item = await retainItem(perSource(), ["source:git", "source:git-log", "gitlog-head:abc"]);
expect(item.observation_scopes).toEqual([[], ["source:git"], ["source:git-log"]]);
expect(item.observation_scopes).toEqual([[], ["source:git"]]);
});
it("gives the commit-log seed the git scope even when only the alias is present", async () => {
const item = await retainItem(perSource(), ["source:git-log"]);
expect(item.observation_scopes).toEqual([[], ["source:git"]]);
});
// The survey baseline is the "researching…" status marker, whose retain strategy
// is meant to extract nothing. On a real repository it still yielded a scope
// holding exactly one observation — a belief set that exists only to be noise.
it("gives the survey status marker no scope of its own", async () => {
const item = await retainItem(perSource(), ["source:survey-baseline"]);
expect(item.observation_scopes).toEqual([[]]);
});
it("still scopes a real source alongside an excluded one", async () => {
const item = await retainItem(perSource(), ["source:chat", "source:survey-baseline"]);
expect(item.observation_scopes).toEqual([[], ["source:chat"]]);
});
it("orders the scopes independently of the order the tags arrive in", async () => {
const item = await retainItem(perSource(), ["source:git-log", "source:git"]);
expect(item.observation_scopes).toEqual([[], ["source:git"], ["source:git-log"]]);
const item = await retainItem(perSource(), ["source:upload", "source:chat"]);
expect(item.observation_scopes).toEqual([[], ["source:chat"], ["source:upload"]]);
});
it("never lets a volatile provenance tag become a scope", async () => {
@@ -59,12 +59,39 @@ export type ObservationScopes =
* The empty scope is always first and always present, so the untagged observations that knowledge
* pages read (they match with `tags_match: "all"`) keep being written exactly as under `shared`.
*/
/**
* `source:` tags are provenance labels, and not every one of them names a KIND of
* claim. Two do not, and both were found by running this on real repositories:
*
* - `source:git-log` is a bookkeeping alias. `git.ts` tags the commit-message seed
* with it AND `source:git`, so a scope for each produced two near-identical
* belief sets — 307 observations against 302 on one repo — and doubled the
* consolidation for the pair. It is the same claim as `source:git`: what the
* commits say.
* - `source:survey-baseline` marks the "researching…" status document, whose retain
* strategy is meant to extract nothing. It still yielded a scope holding exactly
* one observation: a belief set that exists only to be noise.
*
* Mapping rather than excluding keeps the seed's facts in the git scope instead of
* dropping them, which matters because that document is where a cold repo's whole
* commit history arrives.
*/
const SOURCE_SCOPE_ALIASES: Record<string, string> = { "source:git-log": "source:git" };
const SOURCE_SCOPE_EXCLUDED = new Set(["source:survey-baseline"]);
export function resolveRetainScopes(
tags: string[] | undefined,
configured: ObservationScopes
): ObservationScopes {
if (configured !== "per_source") return configured;
const sources = [...new Set((tags ?? []).filter((t) => t.startsWith("source:")))].sort();
const sources = [
...new Set(
(tags ?? [])
.filter((t) => t.startsWith("source:"))
.filter((t) => !SOURCE_SCOPE_EXCLUDED.has(t))
.map((t) => SOURCE_SCOPE_ALIASES[t] ?? t)
),
].sort();
return [[], ...sources.map((s) => [s])];
}