mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
f747d96c38
* feat(recall): fuzzy tag matching on tag_groups leaves (#4026) Tags increasingly hold user-facing names, and tag filtering is exact array containment. A caller filtering by what a query mentioned passes `typsecript`, and the memory tagged `typescript` is dropped before ranking runs — so the recall returns empty even though ranking would have found it. Better ranking cannot fix that; the match itself has to tolerate the misspelling. A `tag_groups` leaf gains one optional field, `resolve`, defaulting to `exact` (today's behaviour). Set to `fuzzy`, its tags are matched against the bank's tags by similarity instead of literally. That is the whole API change: no new config, no new response field, no new TagsMatch values. Matching is trigram similarity at 0.45 via `entity_resolver._trigram_similarity`, already verified byte-identical to Postgres `similarity()` (#3107), so Postgres, Oracle and store-owned backends behave the same. Resolves: typescropt/typescript 0.57, kubernets/kubernetes 0.62, user:alcie/user:alice 0.47. Does not: mango/mongo 0.33, k9s/k8s 0.14. Known limit, pinned by a test: similarity is length-sensitive. A short tag has few trigrams and one edit destroys three of them, so kakfa/kafka scores 0.20 and does not resolve. Fuzzy matching is effective on descriptive tags and close to inert on very short ones — and that same property is what keeps different short words apart. Resolution runs above the SQL layer, rewriting the leaf into ordinary exact leaves so only those reach the query builders. The ~20 SQL call sites, the Python mirrors used on the graph path, the GIN(tags) index, the store protocol and the Oracle dialect are untouched. Per mode, for tokens t1..tn resolving to E1..En: any/any_strict becomes one leaf over the union; all/all_strict becomes an AND of one OR-leaf per token, so a memory must carry some spelling of each; exact becomes an OR over the cross product, one tag per token, bounded at 32 branches and checked before enumeration, with combinations carrying fewer distinct tags than tokens dropped. Failing closed: a tag that resolves to nothing stays in the filter as itself, leaving the leaf unsatisfiable. Returning an empty list would read as "no tag filtering" in the builders and hand back the whole bank. The vocabulary comes from the existing `list_tags` store method, so there is no schema change. A bank holding more than 5000 distinct tags is rejected with a 422 rather than resolved against a truncated vocabulary. Claude-Session: https://claude.ai/code/session_011KDT484YujNcBxHzbfzNbk * fix(clients): make tag_groups reachable through the wrapper SDKs `Hindsight.recall(tag_groups=...)` and `.reflect(tag_groups=...)` raised ModuleNotFoundError for every caller. The wrapper imported `hindsight_client_api.models.recall_request_tag_groups_inner`, which the generator does not emit: it produces one union model per tag_groups shape and names it after the first schema that used it, so the class is `MentalModelTriggerInputTagGroupsInner`. Nothing caught it because the wrapper's tests never passed tag_groups and the import sits inside the `if tag_groups is not None` branch, so it only fires when the feature is used. Fixed at both call sites, with mirrored regression tests on the Python and TypeScript wrappers asserting a tag group reaches the request body with the leaf's `resolve` intact — the pair the review checklist asks for, since a capability that exists in one wrapper and not the other is invisible to client-coverage-check (it validates request-body fields, not wrapper surface). Also thread tag_groups through the control-plane recall and reflect proxy routes and their client types. Both accepted every other tag filter and silently dropped this one, so no control-plane caller could use compound tag filtering at all — fuzzy or exact. Two follow-ups from reviewing #4026: - Reject `resolve="fuzzy"` in a mental-model trigger's tag_groups. A trigger's scope is read by two paths that resolve differently: the refresh runs through reflect, which resolves fuzzy leaves, while the staleness check and the scope watermark build SQL straight from the stored groups and do not. A stored fuzzy leaf would build content from the resolved tags while never being marked stale by them, and would drift as the bank's tag vocabulary changes. - Promote `entity_resolver._trigram_similarity` to `trigram_similarity`. Two subsystems now share it — entity resolution and fuzzy tag matching — so the leading underscore misrepresented a real contract between them. Claude-Session: https://claude.ai/code/session_011KDT484YujNcBxHzbfzNbk