mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
feat(coding-agents): add per_source observation scoping (#3872)
* feat(coding-agents): add per_source observation scoping On a repo worked by a coding agent, commit diffs and session transcripts make different kinds of claim. A diff records what the code does; a transcript records what someone intended, argued for, or discarded. Under the `shared` default both consolidate into one undifferentiated belief set, so an idea floated in chat and never implemented is indistinguishable from a belief derived from the commits, and "what does the codebase actually do" cannot be answered from commit-derived knowledge alone. This cannot be fixed by configuration. The server treats an explicit scope list as unconditional: `_resolve_obs_tags_list` and `_resolve_write_scopes` in the consolidator both return the parsed list verbatim, without filtering it against the memory's own tags. A configured `[[], ["source:git"], ["source:chat"]]` therefore writes EVERY document into all three scopes, and the `source:git` scope fills with observations built from chat transcripts. Only a per-document decision separates them. `per_source` is resolved client-side, per document, in the new exported `resolveRetainScopes`, and never reaches the server. It expands to the global scope plus one per distinct `source:` tag the document carries, sorted — a document with two source tags (the commit-message seed keeps `source:git` alongside `source:git-log` so the cold-repo check still sees it) gets a scope for each, which needs no arbitrary tie-break and does not depend on the order the caller assembled its tags in. Reading only `source:` is what keeps this safe. `per_tag` splits on the right axis but also on every other one: it would reinstate the per-agent `harness:` fork that #3564 and #3575 removed, and any volatile tag such as a session id from `retainTags` would become its own scope — the fragmentation bug itself. The empty scope is always emitted first and unchanged, so the merged view matches `shared` exactly and the untagged observations that knowledge pages read (`tags_match: "all"`, per #3664) are unaffected. The cost is honest: one extra consolidation pass per document. That is the price of the axis, and it is why this is opt-in — `DEFAULT_OBSERVATION_SCOPES` remains `shared` and no existing value changes meaning. Closes #3871 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01CdYchB9yKUWw8oa16RWQu9 * fix(coding-agents): not every source tag names a kind of claim Running per_source on real repositories showed two of the five source tags are provenance labels rather than axes, and each produced a scope worth less than it cost. source:git-log is a bookkeeping alias. git.ts tags the commit-message seed with it AND source:git, so emitting a scope for each gave two near-identical belief sets — 307 observations against 302 on one repo — and doubled the consolidation for that pair. It is the same claim as sourcewhat the commits say. Mapped onto it rather than dropped, because that seed is where a cold repo's entire commit history arrives. source:survey-baseline marks the "researching…" status document, whose retain strategy is meant to extract nothing. It still yielded a scope holding exactly one observation — a belief set that exists only to be noise. Excluded. Emitting a scope per DISTINCT source tag, sorted, is still right: taking the first made the result depend on the order git.ts assembled its tags, which a test pins. It is the vocabulary that carries two non-semantic entries, not the rule. * Revert "fix(coding-agents): not every source tag names a kind of claim" This reverts
0786de8. The evidence behind it was an artefact. The claim was that source:git and source:git-log produced near-identical belief sets (307 observations against 302). They did — but only because at the time of measurement the bank held exactly ONE git document, the commit-message seed, which carries both tags. The per-commit diff backfill under gitIngest: "full" had not run yet, so one document was feeding both scopes. Once it does, the two diverge and are genuinely different questions: source:git-log is fed only by the seed — what the commit MESSAGES say — while source:git also collects every per-commit diff. Intent against implementation. The reasoning was wrong at a deeper level too. Overlapping content across scopes is not duplication to be engineered away: consolidation already distills and deduplicates WITHIN each scope, and a fact that legitimately answers two questions belonging to both is the design working. Excluding tags to avoid overlap misreads what a scope is for. source:survey-baseline goes back for the same reason. Its scope holding a single observation suggests the marker document's retain strategy is extracting facts it is meant to suppress — which is worth fixing where it happens, not papering over in the scope resolver. --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Nicolò Boschi <boschi1997@gmail.com>
This commit is contained in:
committed by
GitHub
parent
aee42254ab
commit
247e9b3138
@@ -454,7 +454,7 @@ hook by Codex...), so one shared config serves several agents side by side:
|
||||
| `retainTags` | — | extra tags on every document written by the integration, e.g. `["project:{gitProject}"]` — see **Recording where a memory came from** below |
|
||||
| `retainMetadata` | — | extra metadata on every document written by the integration, e.g. `{"repo": "{gitProject}"}` |
|
||||
| `manageBankConfig` | `true` | let the plugin shape the bank's own configuration — the retain strategies it writes under, the `knowledge` entity-label group, and, on a bank that has none, the missions. Writing is strictly **additive**: it adds what the bank does not define and never overwrites what is there, so your control-plane edits survive. Set `false` to keep it out of the bank config entirely — see **A bank you shape yourself** below |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]` |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]`; `"per_source"` adds a scope per `source:` kind alongside the global one, so commit knowledge and conversation knowledge consolidate apart |
|
||||
| `disabled` | `false` | hard off-switch (inert plugin/hook — a no-memory baseline) |
|
||||
| `reflectTimeoutMs` | `120000` | **automatic** session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded) |
|
||||
| `reflectToolTimeoutMs` | `330000` | timeout for the agent-invoked `hindsight_reflect` tool — a call the agent waits on, whose high-budget synthesis on a populated bank runs for minutes. Defaults above the server's own reflect wall timeout (`HINDSIGHT_API_REFLECT_WALL_TIMEOUT`, 300s) so the server decides when to give up. Unset, it inherits an explicitly raised `reflectTimeoutMs`, but a short one never lowers it |
|
||||
@@ -640,6 +640,36 @@ scope per bank, which is what a bank already is — one project's memory. Set th
|
||||
}
|
||||
```
|
||||
|
||||
### Splitting code from conversation — `per_source`
|
||||
|
||||
`shared` puts every document a repo produces into one belief set. `"per_source"` keeps that set and
|
||||
adds one per origin, so "what the commits say" and "what was decided in conversation" can be asked
|
||||
apart:
|
||||
|
||||
```jsonc
|
||||
{ "observationScopes": "per_source" }
|
||||
```
|
||||
|
||||
Each document consolidates into the global scope **plus** one named for each `source:` tag it
|
||||
carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git"]]` for a commit
|
||||
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
|
||||
`tags: [], tags_match: "exact"`.
|
||||
|
||||
A document carrying two `source:` tags gets a scope for each, and that is deliberate rather than
|
||||
duplication. The commit-message seed is tagged `source:git` and `source:git-log`, so
|
||||
`source:git-log` is fed only by the seed — what the commit _messages_ say — while `source:git` also
|
||||
collects every per-commit diff under `gitIngest: "full"`. Two questions, two answers, each
|
||||
deduplicated within itself by consolidation. A fact belonging to more than one axis is the point.
|
||||
|
||||
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
|
||||
unconditional — it is not filtered against the memory's own tags — so a configured
|
||||
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
|
||||
scope fills with beliefs built from chat transcripts. Only a per-document decision separates them.
|
||||
|
||||
It costs one extra consolidation pass per document, and it reads only `source:`, so a volatile
|
||||
provenance tag never becomes a scope. The global scope is still written first and unchanged, so the
|
||||
untagged observations knowledge pages read are unaffected.
|
||||
|
||||
`"per_tag"` and `"all_combinations"` split further still, and an explicit `[["project:demo"], …]`
|
||||
declares the scopes literally. `HINDSIGHT_OBSERVATION_SCOPES` sets the scalar modes; a scope list is
|
||||
file-only. Changing this does not rewrite observations already consolidated under the old scoping —
|
||||
|
||||
@@ -457,7 +457,7 @@ hook by Codex...), so one shared config serves several agents side by side:
|
||||
| `retainTags` | — | extra tags on every document written by the integration, e.g. `["project:{gitProject}"]` — see **Recording where a memory came from** below |
|
||||
| `retainMetadata` | — | extra metadata on every document written by the integration, e.g. `{"repo": "{gitProject}"}` |
|
||||
| `manageBankConfig` | `true` | let the plugin shape the bank's own configuration — the retain strategies it writes under, the `knowledge` entity-label group, and, on a bank that has none, the missions. Writing is strictly **additive**: it adds what the bank does not define and never overwrites what is there, so your control-plane edits survive. Set `false` to keep it out of the bank config entirely — see **A bank you shape yourself** below |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]` |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]`; `"per_source"` adds a scope per `source:` kind alongside the global one, so commit knowledge and conversation knowledge consolidate apart |
|
||||
| `disabled` | `false` | hard off-switch (inert plugin/hook — a no-memory baseline) |
|
||||
| `reflectTimeoutMs` | `120000` | **automatic** session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded) |
|
||||
| `reflectToolTimeoutMs` | `330000` | timeout for the agent-invoked `hindsight_reflect` tool — a call the agent waits on, whose high-budget synthesis on a populated bank runs for minutes. Defaults above the server's own reflect wall timeout (`HINDSIGHT_API_REFLECT_WALL_TIMEOUT`, 300s) so the server decides when to give up. Unset, it inherits an explicitly raised `reflectTimeoutMs`, but a short one never lowers it |
|
||||
@@ -643,6 +643,36 @@ scope per bank, which is what a bank already is — one project's memory. Set th
|
||||
}
|
||||
```
|
||||
|
||||
### Splitting code from conversation — `per_source`
|
||||
|
||||
`shared` puts every document a repo produces into one belief set. `"per_source"` keeps that set and
|
||||
adds one per origin, so "what the commits say" and "what was decided in conversation" can be asked
|
||||
apart:
|
||||
|
||||
```jsonc
|
||||
{ "observationScopes": "per_source" }
|
||||
```
|
||||
|
||||
Each document consolidates into the global scope **plus** one named for each `source:` tag it
|
||||
carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git"]]` for a commit
|
||||
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
|
||||
`tags: [], tags_match: "exact"`.
|
||||
|
||||
A document carrying two `source:` tags gets a scope for each, and that is deliberate rather than
|
||||
duplication. The commit-message seed is tagged `source:git` and `source:git-log`, so
|
||||
`source:git-log` is fed only by the seed — what the commit _messages_ say — while `source:git` also
|
||||
collects every per-commit diff under `gitIngest: "full"`. Two questions, two answers, each
|
||||
deduplicated within itself by consolidation. A fact belonging to more than one axis is the point.
|
||||
|
||||
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
|
||||
unconditional — it is not filtered against the memory's own tags — so a configured
|
||||
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
|
||||
scope fills with beliefs built from chat transcripts. Only a per-document decision separates them.
|
||||
|
||||
It costs one extra consolidation pass per document, and it reads only `source:`, so a volatile
|
||||
provenance tag never becomes a scope. The global scope is still written first and unchanged, so the
|
||||
untagged observations knowledge pages read are unaffected.
|
||||
|
||||
`"per_tag"` and `"all_combinations"` split further still, and an explicit `[["project:demo"], …]`
|
||||
declares the scopes literally. `HINDSIGHT_OBSERVATION_SCOPES` sets the scalar modes; a scope list is
|
||||
file-only. Changing this does not rewrite observations already consolidated under the old scoping —
|
||||
|
||||
@@ -202,7 +202,7 @@ hook by Codex...), so one shared config serves several agents side by side:
|
||||
| `retainTags` | — | extra tags on every document written by the integration, e.g. `["project:{gitProject}"]` — see **Recording where a memory came from** below |
|
||||
| `retainMetadata` | — | extra metadata on every document written by the integration, e.g. `{"repo": "{gitProject}"}` |
|
||||
| `manageBankConfig` | `true` | let the plugin shape the bank's own configuration — the retain strategies it writes under, the `knowledge` entity-label group, and, on a bank that has none, the missions. Writing is strictly **additive**: it adds what the bank does not define and never overwrites what is there, so your control-plane edits survive. Set `false` to keep it out of the bank config entirely — see **A bank you shape yourself** below |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]` |
|
||||
| `observationScopes` | `"shared"` | how consolidation groups observations: `"shared"` (default) = ONE global scope per bank, so every agent on a repo builds one set of beliefs; also `"combined"` (the server default), `"per_tag"`, `"all_combinations"`, `[["t"]]`; `"per_source"` adds a scope per `source:` kind alongside the global one, so commit knowledge and conversation knowledge consolidate apart |
|
||||
| `disabled` | `false` | hard off-switch (inert plugin/hook — a no-memory baseline) |
|
||||
| `reflectTimeoutMs` | `120000` | **automatic** session-reflect timeout (hook harnesses additionally cap it at 25s to fit the host's hook window); on timeout the session runs without reflect (recorded) |
|
||||
| `reflectToolTimeoutMs` | `330000` | timeout for the agent-invoked `hindsight_reflect` tool — a call the agent waits on, whose high-budget synthesis on a populated bank runs for minutes. Defaults above the server's own reflect wall timeout (`HINDSIGHT_API_REFLECT_WALL_TIMEOUT`, 300s) so the server decides when to give up. Unset, it inherits an explicitly raised `reflectTimeoutMs`, but a short one never lowers it |
|
||||
@@ -388,6 +388,36 @@ scope per bank, which is what a bank already is — one project's memory. Set th
|
||||
}
|
||||
```
|
||||
|
||||
### Splitting code from conversation — `per_source`
|
||||
|
||||
`shared` puts every document a repo produces into one belief set. `"per_source"` keeps that set and
|
||||
adds one per origin, so "what the commits say" and "what was decided in conversation" can be asked
|
||||
apart:
|
||||
|
||||
```jsonc
|
||||
{ "observationScopes": "per_source" }
|
||||
```
|
||||
|
||||
Each document consolidates into the global scope **plus** one named for each `source:` tag it
|
||||
carries — `[[], ["source:chat"]]` for a session transcript, `[[], ["source:git"]]` for a commit
|
||||
diff. Read an axis back with `tags: ["source:git"], tags_match: "exact"`, and the merged view with
|
||||
`tags: [], tags_match: "exact"`.
|
||||
|
||||
A document carrying two `source:` tags gets a scope for each, and that is deliberate rather than
|
||||
duplication. The commit-message seed is tagged `source:git` and `source:git-log`, so
|
||||
`source:git-log` is fed only by the seed — what the commit _messages_ say — while `source:git` also
|
||||
collects every per-commit diff under `gitIngest: "full"`. Two questions, two answers, each
|
||||
deduplicated within itself by consolidation. A fact belonging to more than one axis is the point.
|
||||
|
||||
This cannot be expressed as a scope list. The server treats an explicit `list[list[str]]` as
|
||||
unconditional — it is not filtered against the memory's own tags — so a configured
|
||||
`[[], ["source:git"], ["source:chat"]]` writes every document into all three, and the `source:git`
|
||||
scope fills with beliefs built from chat transcripts. Only a per-document decision separates them.
|
||||
|
||||
It costs one extra consolidation pass per document, and it reads only `source:`, so a volatile
|
||||
provenance tag never becomes a scope. The global scope is still written first and unchanged, so the
|
||||
untagged observations knowledge pages read are unaffected.
|
||||
|
||||
`"per_tag"` and `"all_combinations"` split further still, and an explicit `[["project:demo"], …]`
|
||||
declares the scopes literally. `HINDSIGHT_OBSERVATION_SCOPES` sets the scalar modes; a scope list is
|
||||
file-only. Changing this does not rewrite observations already consolidated under the old scoping —
|
||||
|
||||
@@ -370,6 +370,11 @@ describe("observationScopes", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("takes per_source, the one scoping an explicit list cannot express", () => {
|
||||
writeJson(globalCfg, { observationScopes: "per_source" });
|
||||
expect(loadConfig({ path: globalCfg }).observationScopes).toBe("per_source");
|
||||
});
|
||||
|
||||
it("takes an explicit scope list, dropping non-string entries", () => {
|
||||
expect(
|
||||
resolveConfig({ observationScopes: [["project:demo"], ["team:eng", "x"]] }).observationScopes
|
||||
|
||||
@@ -265,7 +265,15 @@ function resolveReflectBudget(raw: RawConfig): "low" | "mid" | "high" {
|
||||
}
|
||||
|
||||
/** The server's scalar scoping modes; anything else in this field has to be an explicit scope list. */
|
||||
const OBSERVATION_SCOPE_MODES = ["shared", "combined", "per_tag", "all_combinations"] as const;
|
||||
// `per_source` is this plugin's own mode, resolved per document in `resolveRetainScopes` and never
|
||||
// sent to the server; the rest are the server's.
|
||||
const OBSERVATION_SCOPE_MODES = [
|
||||
"shared",
|
||||
"combined",
|
||||
"per_tag",
|
||||
"all_combinations",
|
||||
"per_source",
|
||||
] as const;
|
||||
|
||||
/**
|
||||
* Validate `observationScopes`, falling back to the default on anything unrecognized.
|
||||
|
||||
@@ -218,7 +218,10 @@ describe("retryAfterMs", () => {
|
||||
});
|
||||
|
||||
describe("HindsightClient.retain — observation scoping", () => {
|
||||
async function retainItem(client: HindsightClient): Promise<Record<string, unknown>> {
|
||||
async function retainItem(
|
||||
client: HindsightClient,
|
||||
tags: string[] = ["source:chat", "harness:claude-code"]
|
||||
): Promise<Record<string, unknown>> {
|
||||
let sent: string | undefined;
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
@@ -227,13 +230,7 @@ describe("HindsightClient.retain — observation scoping", () => {
|
||||
return jsonResponse(200, { operation_id: "op-1" });
|
||||
})
|
||||
);
|
||||
await client.retain(
|
||||
"c",
|
||||
"ctx",
|
||||
"doc-1",
|
||||
["source:chat", "harness:claude-code"],
|
||||
"conversation"
|
||||
);
|
||||
await client.retain("c", "ctx", "doc-1", tags, "conversation");
|
||||
const body = JSON.parse(String(sent)) as { items: Record<string, unknown>[] };
|
||||
return body.items[0];
|
||||
}
|
||||
@@ -258,6 +255,78 @@ describe("HindsightClient.retain — observation scoping", () => {
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* `per_source` is the one scoping a static config cannot express. The server treats an explicit
|
||||
* scope list as UNCONDITIONAL — `_resolve_obs_tags_list` returns it verbatim without filtering
|
||||
* against the memory's own tags — so configuring `[[], ["source:git"], ["source:chat"]]` writes
|
||||
* every document into all three, and the `source:git` scope fills with beliefs built from chat
|
||||
* transcripts. Deriving the scope per document from its own `source:` tag is the only way to get
|
||||
* "what the commits say" apart from "what was discussed" while keeping the merged global set.
|
||||
*
|
||||
* It reads ONLY `source:`, so volatile provenance tags (a session id in `retainTags`) can never
|
||||
* become a scope — the failure mode `per_tag` would reintroduce.
|
||||
*/
|
||||
describe("HindsightClient.retain — per_source scoping", () => {
|
||||
async function retainItem(
|
||||
client: HindsightClient,
|
||||
tags: string[]
|
||||
): Promise<Record<string, unknown>> {
|
||||
let sent: string | undefined;
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn(async (_url: string, init: RequestInit) => {
|
||||
sent = String(init.body);
|
||||
return jsonResponse(200, { operation_id: "op-1" });
|
||||
})
|
||||
);
|
||||
await client.retain("c", "ctx", "doc-1", tags, "conversation");
|
||||
const body = JSON.parse(String(sent)) as { items: Record<string, unknown>[] };
|
||||
return body.items[0];
|
||||
}
|
||||
|
||||
const perSource = () =>
|
||||
new HindsightClient({ apiUrl: "http://x", bank: "b", observationScopes: "per_source" });
|
||||
|
||||
it("keeps the global scope and adds the document's own source scope", async () => {
|
||||
const item = await retainItem(perSource(), ["source:chat", "harness:claude-code"]);
|
||||
expect(item.observation_scopes).toEqual([[], ["source:chat"]]);
|
||||
});
|
||||
|
||||
it("scopes a git document apart from a chat one", async () => {
|
||||
const item = await retainItem(perSource(), ["source:git", "harness:claude-code"]);
|
||||
expect(item.observation_scopes).toEqual([[], ["source:git"]]);
|
||||
});
|
||||
|
||||
it("falls back to the global scope alone when a document carries no source tag", async () => {
|
||||
const item = await retainItem(perSource(), ["knowledge:convention"]);
|
||||
expect(item.observation_scopes).toEqual([[]]);
|
||||
});
|
||||
|
||||
// The commit-message seed carries `source:git` AND `source:git-log` (git.ts keeps
|
||||
// both so the cold-repo check can find it), so it writes to both scopes. That is
|
||||
// not duplication: `source:git-log` is fed only by the seed — what the commit
|
||||
// MESSAGES say — while `source:git` also collects every per-commit diff under
|
||||
// gitIngest: "full". Two questions, two answers, each deduplicated within itself.
|
||||
// A fact belonging to more than one axis is the design working, not a leak.
|
||||
it("gives a document carrying two source tags a scope for each", async () => {
|
||||
const item = await retainItem(perSource(), ["source:git", "source:git-log", "gitlog-head:abc"]);
|
||||
expect(item.observation_scopes).toEqual([[], ["source:git"], ["source:git-log"]]);
|
||||
});
|
||||
|
||||
it("orders the scopes independently of the order the tags arrive in", async () => {
|
||||
const item = await retainItem(perSource(), ["source:git-log", "source:git"]);
|
||||
expect(item.observation_scopes).toEqual([[], ["source:git"], ["source:git-log"]]);
|
||||
});
|
||||
|
||||
it("never lets a volatile provenance tag become a scope", async () => {
|
||||
const item = await retainItem(perSource(), [
|
||||
"source:chat",
|
||||
"hermes-session:20260829_101500_abc",
|
||||
]);
|
||||
expect(item.observation_scopes).toEqual([[], ["source:chat"]]);
|
||||
});
|
||||
});
|
||||
|
||||
/**
|
||||
* The scoping default lives in the client, so an entrypoint that forgets to forward the config
|
||||
* fails SOFTLY — it keeps writing correct memories and just ignores the user's `observationScopes`.
|
||||
@@ -420,7 +489,9 @@ describe("every memory write goes through the one call site that scopes it", ()
|
||||
// Everything between retain()'s signature and the POST is the body it builds; the scoping
|
||||
// has to be set in there, not left to whatever the server defaults to.
|
||||
const body = src.slice(src.indexOf("async retain("), src.indexOf('bankUrl("/memories")'));
|
||||
expect(body).toContain("observation_scopes: this.observationScopes");
|
||||
// The scoping may be derived per document (see `per_source`), but it must still be set on the
|
||||
// item here and still come from the configured value — not from a server default.
|
||||
expect(body).toMatch(/observation_scopes: .*this\.observationScopes/);
|
||||
});
|
||||
});
|
||||
|
||||
|
||||
@@ -33,7 +33,40 @@ export interface KnowledgeNode {
|
||||
* How consolidation scopes the observations a retained memory feeds (`observation_scopes` on the
|
||||
* retain API). The scalar modes are the server's; a `string[][]` declares the scopes explicitly.
|
||||
*/
|
||||
export type ObservationScopes = "shared" | "combined" | "per_tag" | "all_combinations" | string[][];
|
||||
export type ObservationScopes =
|
||||
"shared" | "combined" | "per_tag" | "all_combinations" | "per_source" | string[][];
|
||||
|
||||
/**
|
||||
* `per_source` is resolved HERE, per document, and never reaches the server: it expands to the
|
||||
* global scope plus one named for that document's own `source:` tag.
|
||||
*
|
||||
* It cannot be expressed as configuration. The server treats an explicit scope list as
|
||||
* unconditional — consolidation returns it verbatim without filtering it against the memory's own
|
||||
* tags — so a configured `[[], ["source:git"], ["source:chat"]]` writes EVERY document into all
|
||||
* three, and the `source:git` scope fills up with beliefs built from chat transcripts. Only a
|
||||
* per-document decision separates "what the commits say" from "what was discussed".
|
||||
*
|
||||
* `per_tag` would split on the right axis but also on every other one: it reinstates the per-agent
|
||||
* `harness:` fork that `shared` exists to prevent, and any volatile tag (a session id in
|
||||
* `retainTags`) becomes its own scope, which is the fragmentation bug itself. Reading only
|
||||
* `source:` is what keeps this safe.
|
||||
*
|
||||
* A document may carry more than one source tag — the commit-message seed is both `source:git` and
|
||||
* `source:git-log`, because the cold-repo check filters on `source:git` — so every distinct one
|
||||
* gets a scope. Taking all of them, sorted, is the only rule that needs no arbitrary tie-break and
|
||||
* does not silently depend on the order the caller assembled its tags in.
|
||||
*
|
||||
* The empty scope is always first and always present, so the untagged observations that knowledge
|
||||
* pages read (they match with `tags_match: "all"`) keep being written exactly as under `shared`.
|
||||
*/
|
||||
export function resolveRetainScopes(
|
||||
tags: string[] | undefined,
|
||||
configured: ObservationScopes
|
||||
): ObservationScopes {
|
||||
if (configured !== "per_source") return configured;
|
||||
const sources = [...new Set((tags ?? []).filter((t) => t.startsWith("source:")))].sort();
|
||||
return [[], ...sources.map((s) => [s])];
|
||||
}
|
||||
|
||||
/**
|
||||
* One global scope for everything this plugin writes.
|
||||
@@ -271,7 +304,7 @@ export class HindsightClient {
|
||||
// Sent on EVERY retain, including the server default `combined`, so the scoping a bank's
|
||||
// observations were built under is a property of the write rather than of whichever server
|
||||
// version happened to process it. Servers older than 0.4.15 ignore the field.
|
||||
observation_scopes: this.observationScopes,
|
||||
observation_scopes: resolveRetainScopes(tags, this.observationScopes),
|
||||
};
|
||||
if (opts.timestamp) item.timestamp = opts.timestamp;
|
||||
if (opts.metadata) item.metadata = opts.metadata;
|
||||
|
||||
Reference in New Issue
Block a user
