mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
92695c6e5e
* docs: explain how entity resolution decides to merge a name Entity resolution's thresholds were documented one env var at a time, with nothing describing how the pieces fit together — so the obvious reading is that pg_trgm similarity is what decides a match. It isn't: the trigram threshold only admits candidates, and the merge decision is a separate weighted score (sequence-ratio name similarity 0.5 + co-occurrence 0.3 + recency 0.2, merged above 0.6) with no minimum name similarity of its own. The two measures disagree most on short names, which is how a new person's facts can land on an unrelated existing entity. - configuration.md: new "How entity resolution decides" section covering both stages, the weights, and what to change in either direction. - configuration.md: RETAIN_ENTITY_LOOKUP said `full` was "exact match". It is exact-or-substring, and both strategies feed the same scoring pass, so the choice changes which names can merge — not just how fast the lookup is. - retain.md: short note that resolution can absorb a new name into an existing entity, pointing at the reference section. * fix(entities): don't merge a name onto an existing entity on history alone (#3751) A retain that correctly extracted a new person, Tigran, stored the fact on a pre-existing country entity, Iran. Three defects combined: 1. The metric that admits candidates and the metric that decides the merge were different, and they rank differently. pg_trgm scores tigran/iran at 0.20 — above the 0.15 that admits it, below anything that should merge. difflib's SequenceMatcher, which actually decides, scores it 0.80: higher than alice/alice chen (0.67), a merge the resolver exists to make. With the non-name signals worth 0.5 of the 0.6 threshold and no floor of their own, history alone could carry it. Gate candidates on trigram similarity before scoring (HINDSIGHT_API_ENTITY_MERGE_MIN_SIMILARITY, 0.3). A gate, not a replacement: SequenceMatcher stays load-bearing for typo variants arriving with no co-occurrence context ("Dr Waler" -> "Dr Wall"), which is why the metric is not simply swapped. This gives the existing-entity side the same three-tier story the in-batch side already had — 0.15 to be considered, 0.3 to be merged onto, 0.5 to be folded in-batch. 2. The co-occurrence term counted every shared entity equally, so `user` — which co-occurs with nearly everything in a mature bank — was worth as much as a selective partner, and was what pushed Iran over. Weight each shared partner by 1/sqrt(degree). Degrees come from the co-occurrence rows already fetched, so no extra query; with no degrees known the formula is arithmetically the old overlap fraction, so nothing changes for callers that pass none. 3. A score *of* 0.6 landed on either side of the strict `>` depending only on which signals produced it: 0.4 + 0.2 is 0.6000000000000001 and merged, 0.3 + 0.3 is 0.6 and did not. Round, and compare inclusively — which is the verdict the common composition already got, so no merge that happens today stops. Also folds the three byte-identical co-occurrence-map builders into one indexer (it now computes degrees too), and shares the trigram Jaccard so the mention's trigram set is built once per mention rather than once per candidate — the scoring loop runs up to max_candidates times per mention (GH-3211). * test(entities): dataset-driven eval of the whole resolution pipeline The unit tests around entity resolution pin individual pieces — a similarity number, one scoring branch, one strategy's SQL. #3751 was not any of those: each piece behaved as designed and the *combination* attributed a new person's facts to an unrelated country. Nothing was positioned to catch that. 25 cases, each seeding real entities and entity_cooccurrences into a bank and running the real resolve_entities_batch over them — pg_trgm probe, the partial index that excludes labels, the candidate cap, scoring, insert. Covers identity (case, Turkish İ, emoji), variants that must merge, coincidental names that must not, co-occurrence quality, the recency window, labels, caller-authored names, in-batch clustering and containment. Every case runs under both lookup strategies and must agree, unless it declares otherwise — which turns "full vs trigram is not just a performance choice" from a claim in the docs into six cases that say exactly where they diverge. Time is fixed (last_seen is seeded relative to a constant EVENT_DATE), and the resolver is built from the shipped config rather than constructor defaults, so the dataset measures a real deployment and moving a threshold shows up here. Verified it bites: with HINDSIGHT_API_ENTITY_MERGE_MIN_SIMILARITY=0.01 the three collision cases fail exactly as reported, and only under trigram. Two cases are xfailed as known limitations rather than quietly given the answer the code produces: - Iran/Iraq clears the floor at 0.43, so a genuinely selective shared partner still merges two distinct entities. Telling "same entity, different spelling" from "different entity, similar spelling" needs the extractor's type or an adjudication step, not another threshold. - A decorated form of a stored name ("Wren 🎵" vs "Wren") has *identical* trigram sets, but the score reads the sequence ratio (0.80), so it forks into a second entity once the stored one is a day stale — while the same two forms DO unify when both are new, since the in-batch pass merges at 0.5 trigram. #3107 fixed that half only. * test(entities): move the resolution eval to JSONL and grow it to 61 cases The dataset was Python literals inside the test module, which put the cases and the harness in one file and made adding a case a code change. It now lives in tests/entity_resolution_cases.jsonl, one object per line, and the module is only a loader plus the runner. 61 passing cases (up from 23), 3 recorded as known limitations. New coverage: diacritics across four scripts, punctuation as a pg_trgm separator (hyphen, apostrophe, trailing period), spelling variants, typos, abbreviations, middle initials, corporate suffixes, near-miss place and person names, the recency window from both sides, the floor's inclusive boundary (Nick/Nicolas is exactly 0.30), label keys and unseen values, three-way in-batch clustering, containment in both directions, a crowded candidate field, and bank isolation. Two things the dataset now guards that it did not before: - A well-formedness test. A case that co-occurs with a name it never seeded, or expects the wrong number of answers, would pass while measuring nothing — how the first draft of this eval nearly shipped blind. It also fails if the file drops below 50 passing cases. - The bank-isolation case seeds the other bank with exactly the entity the mention would merge onto, so a leak changes the answer instead of being masked by an identical name. Verified the dataset discriminates in both directions: with ENTITY_MERGE_MIN_SIMILARITY=0.01 five cases fail (the collisions, one of them under `full` too, since "Hannah" contains "anna"), and at 0.6 twenty fail as legitimate variants stop merging. So it brackets the setting rather than only pinning today's value. The third known limitation is unchanged in kind but worse than recorded: with the stored entity seen the same day, Iran/Iraq merges on an *indiscriminate* partner too, not only a selective one — 0.75 by sequence ratio is 0.375 and recency alone adds 0.2. The damping case now holds the stored entity outside the recency window so it pins the damping rather than the clock. * fix(entities): close the two defects the resolution eval surfaced Both came out of the dataset in the previous commit, where they were recorded as known limitations rather than given whatever answer the code produced. **Different given names sharing a surname merged.** "John Smith" and "Jane Smith" are 0.47 by trigram and 0.80 by sequence ratio, so two people who share a surname and a workplace scored as one entity — the shared long word drowns out a completely different short one. A candidate must now agree word by word: every word of the shorter name has to find a counterpart in the longer one, by equality, by abbreviation (corp/corporation, via prefix) or by spelling (são/sao 0.67, waler/wall 0.67, arbor/arbour 0.91). The cutoff sits at 0.6, between john/jane at 0.50 and every legitimate word difference above it. Single-word names are exempt on purpose: with one token the whole-name floor already IS the token check, and applying this on top would reject real variants that have no long shared word to hide behind ("Nick"/"Nicolas" is 0.55). **A decorated form of a stored name forked off it.** "Wren 🎵" against a stored "Wren" has identical trigram sets — pg_trgm builds trigrams per word, so only the separators differ — yet the score reads the sequence ratio (0.80 -> 0.40) and needed history to reach 0.6. The in-batch pass already unifies such names on the name alone and at a *lower* bar (0.5), so whether two forms were one entity depended only on whether they arrived in the same retain; #3107 fixed that half only. Identical trigram sets now reuse the candidate outright. The candidate-cap test counted SequenceMatcher calls as a proxy for "candidates scored"; the word-level check calls it too, and a candidate the trigram gate rejects never reaches the name score, so it now counts the word-level check — once per scored candidate, used nowhere else. The cap itself is unchanged. Left as the one known limitation, now with the proof: Iran/Iraq is a single character apart, and so are Alice/Alise (0.33 trigram) and Sara/Sarah (0.57), which are real merges scoring below and around it. No character-level threshold can order "different entity, similar spelling" above "same entity, different spelling", and the word rule cannot help — both names are one word. That one needs the extractor's type or an adjudication step. * docs(entities): correct three comments the resolution change made stale From code review of the branch, no behaviour change: - _tokens_are_compatible lowercases its own input now. _trigram_set does, so a mixed-case caller would have silently disagreed with the gate beside it. - The resolve=True contract said similar entities "are scored"; there is a floor in front of the score now, so say which. - The max_candidates note said scoring is one SequenceMatcher call per candidate. It is a trigram set per candidate, plus a word-level check and a SequenceMatcher pass for those clearing the floor — which is also why the cap's own test no longer counts SequenceMatcher.
65 lines
24 KiB
JSON
65 lines
24 KiB
JSON
{"id": "exact-name-is-reused", "pins": "Nothing may get between a name and its own entity.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 400}], "mentions": [{"text": "Alice Chen"}], "expect": ["Alice Chen"]}
|
|
{"id": "case-differences-are-the-same-entity", "pins": "Canonical names match case-insensitively.", "existing": [{"name": "Alice Chen"}], "mentions": [{"text": "alice chen"}], "expect": ["Alice Chen"]}
|
|
{"id": "turkish-dotted-i-is-the-same-entity", "pins": "Postgres and Python disagree on lowercasing İ; resolution must not split on it.", "existing": [{"name": "İstanbul"}], "mentions": [{"text": "istanbul"}], "expect": ["İstanbul"]}
|
|
{"id": "the-same-name-twice-in-one-fact-is-one-entity", "pins": "A fact naming someone twice must not create two rows or two different answers.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 0}], "mentions": [{"text": "Alice Chen"}, {"text": "Alice Chen"}], "expect": ["Alice Chen", "Alice Chen"]}
|
|
{"id": "another-banks-entity-is-never-a-candidate", "pins": "Bank isolation is a hard invariant. The other bank holds exactly the entity this mention would merge onto in its own bank, so a leak changes the answer rather than being masked by an identical name.", "existing": [], "mentions": [{"text": "Dr Waler"}], "expect": ["Dr Waler"], "other_bank": [{"name": "Dr Wall", "last_seen_days_ago": 0}]}
|
|
{"id": "an-empty-fact-resolves-nothing", "pins": "A batch with no mentions must not touch the bank.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 0}], "mentions": [], "expect": []}
|
|
{"id": "diacritic-umlaut-merges", "pins": "A diacritic is a spelling of the same name, not a different city.", "existing": [{"name": "Zurich", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Zürich"}, {"text": "Bletchley Park"}], "expect": ["Zurich", "Bletchley Park"], "expect_full": ["Zürich", "Bletchley Park"]}
|
|
{"id": "diacritic-acute-merges", "pins": "Same, with an acute accent.", "existing": [{"name": "Bogota", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Bogotá"}, {"text": "Bletchley Park"}], "expect": ["Bogota", "Bletchley Park"], "expect_full": ["Bogotá", "Bletchley Park"]}
|
|
{"id": "diacritic-tilde-merges", "pins": "Same, across a two-word name.", "existing": [{"name": "Sao Paulo", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "São Paulo"}, {"text": "Bletchley Park"}], "expect": ["Sao Paulo", "Bletchley Park"], "expect_full": ["São Paulo", "Bletchley Park"]}
|
|
{"id": "diacritic-in-a-person-name-merges", "pins": "Person names carry the same accent variation.", "existing": [{"name": "Renee", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Renée"}, {"text": "Bletchley Park"}], "expect": ["Renee", "Bletchley Park"], "expect_full": ["Renée", "Bletchley Park"]}
|
|
{"id": "diacritic-spanish-merges", "pins": "0.43 by trigram — the floor is set below ordinary accent loss.", "existing": [{"name": "Jose", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "José"}, {"text": "Bletchley Park"}], "expect": ["Jose", "Bletchley Park"], "expect_full": ["José", "Bletchley Park"]}
|
|
{"id": "hyphen-is-a-separator-so-forms-merge", "pins": "pg_trgm splits on punctuation, so hyphen and space forms share every trigram (1.00).", "existing": [{"name": "Jean Luc", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Jean-Luc"}, {"text": "Bletchley Park"}], "expect": ["Jean Luc", "Bletchley Park"], "expect_full": ["Jean-Luc", "Bletchley Park"]}
|
|
{"id": "apostrophe-form-merges", "pins": "An apostrophe is a separator; the trigram sets still overlap at 0.50.", "existing": [{"name": "OBrien", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "O'Brien"}, {"text": "Bletchley Park"}], "expect": ["OBrien", "Bletchley Park"], "expect_full": ["O'Brien", "Bletchley Park"]}
|
|
{"id": "hyphenated-product-version-merges", "pins": "Digits and hyphens: identical trigram sets.", "existing": [{"name": "GPT 4", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "GPT-4"}, {"text": "Bletchley Park"}], "expect": ["GPT 4", "Bletchley Park"], "expect_full": ["GPT-4", "Bletchley Park"]}
|
|
{"id": "case-and-trailing-punctuation-merge", "pins": "Case plus a trailing period is the same surface form (1.00).", "existing": [{"name": "ACME Corp.", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "acme corp"}, {"text": "Bletchley Park"}], "expect": ["ACME Corp.", "Bletchley Park"]}
|
|
{"id": "a-typo-merges", "pins": "A doubled letter is 0.70 by trigram.", "existing": [{"name": "Beijjing", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Beijing"}, {"text": "Bletchley Park"}], "expect": ["Beijjing", "Bletchley Park"], "expect_full": ["Beijing", "Bletchley Park"]}
|
|
{"id": "regional-spelling-merges", "pins": "British/American spelling of one place.", "existing": [{"name": "Ann Arbour", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Ann Arbor"}, {"text": "Bletchley Park"}], "expect": ["Ann Arbour", "Bletchley Park"], "expect_full": ["Ann Arbor", "Bletchley Park"]}
|
|
{"id": "localised-place-name-merges", "pins": "Milan is a substring of Milano, so both strategies see it.", "existing": [{"name": "Milano", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Milan"}, {"text": "Bletchley Park"}], "expect": ["Milano", "Bletchley Park"]}
|
|
{"id": "abbreviated-product-name-merges", "pins": "Postgres is contained in PostgreSQL.", "existing": [{"name": "Postgres", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "PostgreSQL"}, {"text": "Bletchley Park"}], "expect": ["Postgres", "Bletchley Park"]}
|
|
{"id": "spelling-variant-of-a-given-name-merges", "pins": "0.54 by trigram; one name, two spellings.", "existing": [{"name": "Catherine", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Katherine"}, {"text": "Bletchley Park"}], "expect": ["Catherine", "Bletchley Park"], "expect_full": ["Katherine", "Bletchley Park"]}
|
|
{"id": "second-spelling-variant-merges", "pins": "0.36 by trigram — deliberately above the floor.", "existing": [{"name": "Stephen", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Steven"}, {"text": "Bletchley Park"}], "expect": ["Stephen", "Bletchley Park"], "expect_full": ["Steven", "Bletchley Park"]}
|
|
{"id": "short-form-of-a-name-merges", "pins": "Sara is contained in Sarah.", "existing": [{"name": "Sarah", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Sara"}, {"text": "Bletchley Park"}], "expect": ["Sarah", "Bletchley Park"]}
|
|
{"id": "corporate-suffix-merges", "pins": "A truncated legal suffix.", "existing": [{"name": "Microsoft Corporation", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Microsoft Corp"}, {"text": "Bletchley Park"}], "expect": ["Microsoft Corporation", "Bletchley Park"]}
|
|
{"id": "middle-initial-merges", "pins": "0.85 by trigram; an initial is not a different person.", "existing": [{"name": "John Smith", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "John A Smith"}, {"text": "Bletchley Park"}], "expect": ["John Smith", "Bletchley Park"], "expect_full": ["John A Smith", "Bletchley Park"]}
|
|
{"id": "added-surname-merges", "pins": "The canonical merge this resolver exists for.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Alice"}, {"text": "Bletchley Park"}], "expect": ["Alice Chen", "Bletchley Park"]}
|
|
{"id": "typo-in-a-title-merges", "pins": "The shape #3479 pinned, from the other side.", "existing": [{"name": "Dr Wall", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Dr. Waller"}, {"text": "Bletchley Park"}], "expect": ["Dr Wall", "Bletchley Park"], "expect_full": ["Dr. Waller", "Bletchley Park"]}
|
|
{"id": "a-transposed-name-merges", "pins": "0.33 by trigram — just above the floor, and one person.", "existing": [{"name": "Alise", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Alice"}, {"text": "Bletchley Park"}], "expect": ["Alise", "Bletchley Park"], "expect_full": ["Alice", "Bletchley Park"]}
|
|
{"id": "a-typo-merges-on-name-and-recency-alone", "pins": "A typo variant arriving with no co-occurrence context at all (#3479).", "existing": [{"name": "Dr Wall", "last_seen_days_ago": 0}], "mentions": [{"text": "Dr Waler"}], "expect": ["Dr Wall"], "expect_full": ["Dr Waler"]}
|
|
{"id": "new-person-is-not-absorbed-by-a-similar-country", "pins": "#3751 exactly: 0.20 by trigram, 0.80 by sequence ratio, every other signal maxed.", "existing": [{"name": "Iran", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Tigran"}, {"text": "user"}], "expect": ["Tigran", "user"]}
|
|
{"id": "cyrillic-form-of-the-same-collision", "pins": "The reporting bank was Cyrillic; pg_trgm scores it identically to the transliteration.", "existing": [{"name": "Иран", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Тигран"}, {"text": "user"}], "expect": ["Тигран", "user"]}
|
|
{"id": "a-city-is-not-absorbed-by-its-country", "pins": "Tehran/Iran is 0.20 — a shared suffix between two genuinely different entities.", "existing": [{"name": "Iran", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Tehran"}, {"text": "user"}], "expect": ["Tehran", "user"]}
|
|
{"id": "an-unrelated-short-name-is-not-absorbed", "pins": "Ivan/Iran is 0.25: above the probe's 0.15, below the merge floor.", "existing": [{"name": "Iran", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Ivan"}, {"text": "user"}], "expect": ["Ivan", "user"]}
|
|
{"id": "different-given-names-stay-separate", "pins": "Ahmed/Mohammed is 0.15 by trigram but 0.62 by sequence ratio.", "existing": [{"name": "Mohammed", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Ahmed"}, {"text": "user"}], "expect": ["Ahmed", "user"]}
|
|
{"id": "neighbouring-states-stay-separate", "pins": "New York/New Jersey shares a whole word and is still 0.25 — two places.", "existing": [{"name": "New Jersey", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "New York"}, {"text": "user"}], "expect": ["New York", "user"]}
|
|
{"id": "similar-looking-cities-stay-separate", "pins": "Berlin/Berkeley is 0.23; a shared prefix is not an identity.", "existing": [{"name": "Berkeley", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Berlin"}, {"text": "user"}], "expect": ["Berlin", "user"]}
|
|
{"id": "names-sharing-only-letters-stay-separate", "pins": "Anna/Hannah is 0.80 by sequence ratio and 0.20 by trigram — the #3751 shape again.", "existing": [{"name": "Hannah", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Anna"}, {"text": "user"}], "expect": ["Anna", "user"]}
|
|
{"id": "unrelated-cities-stay-separate", "pins": "Tokyo/Kyoto is an anagram-ish 0.09; the sequence ratio still reads 0.60.", "existing": [{"name": "Kyoto", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Tokyo"}, {"text": "user"}], "expect": ["Tokyo", "user"]}
|
|
{"id": "two-countries-sharing-a-word-stay-separate", "pins": "United States/United Kingdom is 0.32 — over the floor, under the threshold on merit.", "existing": [{"name": "United Kingdom", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "United States"}, {"text": "user"}], "expect": ["United States", "user"]}
|
|
{"id": "an-exonym-is-not-unified-with-its-endonym", "pins": "Wien/Vienna is one city under two names and 0.09 by trigram. Resolution keys off surface form, so it cannot know — documented behaviour, not a bug to route around here.", "existing": [{"name": "Vienna", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Wien"}, {"text": "user"}], "expect": ["Wien", "user"]}
|
|
{"id": "an-indiscriminate-partner-cannot-carry-a-merge", "pins": "Iran/Iraq clears the floor at 0.43, so the shared `user` is the only thing that could carry it: the stored entity is held outside the recency window on purpose, or 0.2 of recency would decide the case instead of the damping it exists to pin. Unweighted, the hub alone would score 0.675 and merge.", "existing": [{"name": "Iraq", "last_seen_days_ago": 30, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Iran"}, {"text": "user"}], "expect": ["Iran", "user"]}
|
|
{"id": "a-plausible-variant-also-waits-for-better-evidence", "pins": "The damping is not aimed at bad names: Alice/Alice Chen is a real variant, and on a hub alone it still falls short. Evidence quality gates both directions.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Alice"}, {"text": "user"}], "expect": ["Alice", "user"]}
|
|
{"id": "a-selective-partner-is-real-evidence", "pins": "The same weak name merges when the shared entity is one that actually means something.", "existing": [{"name": "Alise", "last_seen_days_ago": 30, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Alice"}, {"text": "Bletchley Park"}], "expect": ["Alise", "Bletchley Park"], "expect_full": ["Alice", "Bletchley Park"]}
|
|
{"id": "a-stale-entity-does-not-win-on-name-alone", "pins": "Outside the 7-day window the recency term is zero, so the name must stand alone.", "existing": [{"name": "Dr Wall", "last_seen_days_ago": 30}], "mentions": [{"text": "Dr Waler"}], "expect": ["Dr Waler"], "expect_full": ["Dr Waler"]}
|
|
{"id": "the-recency-window-is-closed-at-seven-days", "pins": "Exactly 7 days old scores zero recency: Nick/Nicolas then falls short of the threshold.", "existing": [{"name": "Nicolas", "last_seen_days_ago": 7, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Nick"}, {"text": "Bletchley Park"}], "expect": ["Nick", "Bletchley Park"], "expect_full": ["Nick", "Bletchley Park"]}
|
|
{"id": "inside-the-recency-window-the-same-pair-merges", "pins": "The same names, seen today, clear it — and Nick/Nicolas is exactly 0.30, so this doubles as the floor's inclusive boundary.", "existing": [{"name": "Nicolas", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Nick"}, {"text": "Bletchley Park"}], "expect": ["Nicolas", "Bletchley Park"], "expect_full": ["Nick", "Bletchley Park"]}
|
|
{"id": "a-label-value-is-reused-exactly", "pins": "Labels resolve by exact match; one value, one entity.", "existing": [{"name": "topic:finance", "last_seen_days_ago": 3650, "kind": "label"}], "mentions": [{"text": "topic:finance"}], "expect": ["topic:finance"], "labels": [{"key": "topic", "type": "text"}]}
|
|
{"id": "near-identical-label-values-stay-distinct", "pins": "A controlled vocabulary must not collapse; topic:finance/topic:finances is 0.78.", "existing": [{"name": "topic:finance", "last_seen_days_ago": 0, "kind": "label"}], "mentions": [{"text": "topic:finances"}], "expect": ["topic:finances"], "labels": [{"key": "topic", "type": "text"}]}
|
|
{"id": "regular-text-never-merges-into-a-label-row", "pins": "Label rows are excluded from fuzzy matching in SQL and again while scoring (#1558).", "existing": [{"name": "topic:empathy", "last_seen_days_ago": 0, "kind": "label"}], "mentions": [{"text": "topic empathy"}], "expect": ["topic empathy"], "labels": [{"key": "topic", "type": "text"}]}
|
|
{"id": "a-label-and-a-regular-entity-can-share-a-fact", "pins": "The common retain shape: one label plus one person, resolved in the same batch.", "existing": [{"name": "topic:finance", "last_seen_days_ago": 0, "kind": "label"}, {"name": "Alice Chen", "last_seen_days_ago": 0, "cooccurs_with": ["topic:finance"]}], "mentions": [{"text": "Alice Chen"}, {"text": "topic:finance"}], "expect": ["Alice Chen", "topic:finance"], "labels": [{"key": "topic", "type": "text"}]}
|
|
{"id": "an-unseen-label-value-is-created", "pins": "A new value of a known key becomes its own label entity rather than resolving to a sibling.", "existing": [{"name": "topic:finance", "last_seen_days_ago": 0, "kind": "label"}], "mentions": [{"text": "topic:health"}], "expect": ["topic:health"], "labels": [{"key": "topic", "type": "text"}]}
|
|
{"id": "a-literal-name-is-not-re-resolved", "pins": "resolve=False means the caller authored it; the graph must not overrule them (#3479).", "existing": [{"name": "Dr Wall", "last_seen_days_ago": 0}], "mentions": [{"text": "Dr. Waller", "resolve": false}], "expect": ["Dr. Waller"]}
|
|
{"id": "a-literal-name-still-reuses-an-exact-match", "pins": "Taking a name literally means no fuzzy matching — not refusing to reuse its own entity.", "existing": [{"name": "Dr. Waller", "last_seen_days_ago": 0}], "mentions": [{"text": "dr. waller", "resolve": false}], "expect": ["Dr. Waller"]}
|
|
{"id": "opting-out-is-per-mention", "pins": "One retain carries the caller's names and the extractor's; only the caller's are literal.", "existing": [{"name": "Dr Wall", "last_seen_days_ago": 0}], "mentions": [{"text": "Dr. Waller", "resolve": false}, {"text": "Dr Waler"}], "expect": ["Dr. Waller", "Dr Wall"], "expect_full": ["Dr. Waller", "Dr Waler"]}
|
|
{"id": "same-batch-variants-collapse-to-one-entity", "pins": "Both names are new, so only the in-batch pass can unify them (#3107).", "existing": [], "mentions": [{"text": "Wren"}, {"text": "Wren 🎵"}], "expect": ["Wren", "Wren"]}
|
|
{"id": "same-batch-literal-names-stay-apart", "pins": "Two names the caller wrote as two must stay two, however similar (#3479).", "existing": [], "mentions": [{"text": "Alice", "resolve": false}, {"text": "Alice Smith", "resolve": false}], "expect": ["Alice", "Alice Smith"]}
|
|
{"id": "same-batch-unrelated-names-stay-apart", "pins": "The in-batch cutoff is 0.5; unrelated names are nowhere near it.", "existing": [], "mentions": [{"text": "Alice"}, {"text": "Bogotá"}], "expect": ["Alice", "Bogotá"]}
|
|
{"id": "same-batch-clusters-can-hold-three-forms", "pins": "Union-find, not pairwise: three spellings of one new name collapse to a single entity.", "existing": [], "mentions": [{"text": "ACME Corp"}, {"text": "acme corp."}, {"text": "ACME Corp"}], "expect": ["ACME Corp", "ACME Corp", "ACME Corp"]}
|
|
{"id": "an-existing-match-and-a-new-name-in-one-fact", "pins": "The two paths run in one batch: one mention reuses, the other inserts.", "existing": [{"name": "Alice Chen", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Alice"}, {"text": "Bletchley Park"}, {"text": "Tigran"}], "expect": ["Alice Chen", "Bletchley Park", "Tigran"], "expect_full": ["Alice Chen", "Bletchley Park", "Tigran"]}
|
|
{"id": "a-short-name-is-not-swallowed-by-a-long-one", "pins": "`full` admits any substring match, so containment alone must not merge.", "existing": [{"name": "Alice Smith Holdings Ltd", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Alice"}, {"text": "user"}], "expect": ["Alice", "user"]}
|
|
{"id": "a-long-name-is-not-swallowed-by-a-short-one", "pins": "The same in reverse: the stored name being contained in the mention is not identity either.", "existing": [{"name": "Ltd", "last_seen_days_ago": 0, "cooccurs_with": ["user"]}, {"name": "user", "last_seen_days_ago": 3650, "hub_degree": 60}], "mentions": [{"text": "Alice Smith Holdings Ltd"}, {"text": "user"}], "expect": ["Alice Smith Holdings Ltd", "user"]}
|
|
{"id": "the-right-candidate-wins-a-crowded-field", "pins": "Twenty near-identical stored names: the candidate cap ranks by real similarity, so the exact one must still win rather than whichever the planner returned first.", "existing": [{"name": "Alicia Chen 00", "last_seen_days_ago": 100}, {"name": "Alicia Chen 01", "last_seen_days_ago": 100}, {"name": "Alicia Chen 02", "last_seen_days_ago": 100}, {"name": "Alicia Chen 03", "last_seen_days_ago": 100}, {"name": "Alicia Chen 04", "last_seen_days_ago": 100}, {"name": "Alicia Chen 05", "last_seen_days_ago": 100}, {"name": "Alicia Chen 06", "last_seen_days_ago": 100}, {"name": "Alicia Chen 07", "last_seen_days_ago": 100}, {"name": "Alicia Chen 08", "last_seen_days_ago": 100}, {"name": "Alicia Chen 09", "last_seen_days_ago": 100}, {"name": "Alicia Chen 10", "last_seen_days_ago": 100}, {"name": "Alicia Chen 11", "last_seen_days_ago": 100}, {"name": "Alicia Chen 12", "last_seen_days_ago": 100}, {"name": "Alicia Chen 13", "last_seen_days_ago": 100}, {"name": "Alicia Chen 14", "last_seen_days_ago": 100}, {"name": "Alicia Chen 15", "last_seen_days_ago": 100}, {"name": "Alicia Chen 16", "last_seen_days_ago": 100}, {"name": "Alicia Chen 17", "last_seen_days_ago": 100}, {"name": "Alicia Chen 18", "last_seen_days_ago": 100}, {"name": "Alicia Chen 19", "last_seen_days_ago": 100}, {"name": "Alice Chen", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "Alice Chen"}, {"text": "Bletchley Park"}], "expect": ["Alice Chen", "Bletchley Park"]}
|
|
{"id": "selective-partner-still-merges-two-real-countries", "pins": "Iran/Iraq is one character apart, and so are Alice/Alise (0.33 trigram) and Sara/Sarah (0.57) — both real merges that score BELOW or around it. No character-level threshold can order 'different entity, similar spelling' above 'same entity, different spelling', and the word-level rule cannot help either: both names are a single word. Separating these needs the extractor's type or an adjudication step. Weaker than it looks, too: with the stored entity seen the same day, even an indiscriminate partner carries the pair over.", "existing": [{"name": "Iraq", "last_seen_days_ago": 1, "cooccurs_with": ["Basra"]}, {"name": "Basra"}], "mentions": [{"text": "Iran"}, {"text": "Basra"}], "expect": ["Iran", "Basra"], "known_limitation": "resolution merges Iran onto Iraq"}
|
|
{"id": "different-given-names-sharing-a-surname-stay-separate", "pins": "John Smith/Jane Smith is 0.47 by trigram and 0.80 by sequence ratio: the shared surname drowns out a completely different given name. Word-level agreement is what separates them — john/jane is 0.50, below every legitimate word difference.", "existing": [{"name": "Jane Smith", "last_seen_days_ago": 0, "cooccurs_with": ["Bletchley Park"]}, {"name": "Bletchley Park"}], "mentions": [{"text": "John Smith"}, {"text": "Bletchley Park"}], "expect": ["John Smith", "Bletchley Park"]}
|
|
{"id": "emoji-decoration-is-the-same-entity", "pins": "A decorated form of a stored name has identical trigram sets to it — pg_trgm builds trigrams per word, so only the separators differ. The in-batch pass already unifies such names on the name alone at a lower bar (0.5), so requiring history here made two forms one entity or two depending only on whether they arrived in the same retain.", "existing": [{"name": "Wren", "last_seen_days_ago": 2}], "mentions": [{"text": "Wren 🎵"}], "expect": ["Wren"]}
|