mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
* fix(recall): boost the prioritised arm in rank space, not score space (#3956) `RECALL_STRATEGY_BOOSTS=graph:high` could exclude an entire retrieval arm from the cross-encoder rather than merely deprioritising it: on a ~15k-fact bank a reporter measured recall@20 falling 0.9667 -> 0.4000, with zero semantic-only candidates surviving the reranker cap on all 30 test queries. The stage-1 boost multiplied the arm's `1/(k+rank)` RRF contribution by a weight `w`, and that sort key feeds the hard RERANKER_MAX_CANDIDATES cut. RRF with k=60 is deliberately flat: across the 300-candidate cap window the score spans only 1/61 -> 1/360, a factor of 5.9. `high` used w=7, above that spread, so the sort degenerated into a lexicographic one -- boosted arm first, rank merely a tiebreaker -- and the boosted arm took every slot. The culprit is the `k` term: in score space the displacement reach is `r_max = w*(k+s) - k`, so at the head of the ranking the constant `w*k` dominates and the boosted arm's ~366th hit outranked the other arm's first. The levels were tuned against a bank with 336 merged candidates against a 300 cap (89% survival), where the cut could evict at most 36 candidates and the boost really was a reordering; nothing in the formula carried a pool-size term, so the calibration stopped holding as pools grew. Boost the rank instead -- `1/(k + rank/divisor)` -- which cancels `k`: the boosted arm's rank `r` beats another arm's rank `s` iff `r < divisor * s`. Displacement becomes proportional rather than an absolute offset, so it can never invert the head of another arm, and it no longer depends on the merged pool size or on `k`. Levels become divisors: low=2, medium=4, high=8. Replaying the reported shape (3563 merged candidates, 300 cap, 8.4% survival), old formula vs new on an identical pool: semantic-only kept top-20 semantic kept no boost 48 20/20 old graph:high 0 3/20 new graph:high 10 19/20 The boost still does its job: `high` protects the boosted arm to rank 267 against an unboosted baseline of 150. Also surface the cut in the recall trace as a `rerank_prefilter` phase (kept/dropped, the cap in force, active boosts, per-arm composition of the survivors). The boosts previously reached only the server log, so a trace -- where you look when ranking seems wrong -- gave no hint a boost was applied. Note the cap is now caller-supplied and budget-resolved, so it can be below 300 on a low budget, which made the old behaviour strictly worse than the figures above. Claude-Session: https://claude.ai/code/session_018HDqrzHgqZqsGc7EDqoTEu * chore(docs): regenerate docs skill for the --shm-size=1g install snippet Pre-existing drift, not introduced here. `hindsight-docs/docs/developer/` gained `--shm-size=1g` on the `docker run` snippet, but the generated `skills/hindsight-docs/references/` copies were never regenerated. `verify-generated-files` does not run on main pushes, so the drift stayed invisible until a PR touching hindsight-docs/** made the job run and regenerate everything. Committing the generated output unblocks CI. Claude-Session: https://claude.ai/code/session_018HDqrzHgqZqsGc7EDqoTEu
This commit is contained in:
Nicolò Boschi
@@ -7234,14 +7234,36 @@ class MemoryEngine(MemoryEngineInterface):
|
||||
)
|
||||
if len(merged_candidates) > max_candidates:
|
||||
# Sort by RRF score (boosted per-strategy if configured) and take top
|
||||
# candidates. The weighted-RRF boost keeps boosted-arm candidates from
|
||||
# being trimmed out of the reranker's global budget.
|
||||
# candidates. The rank-space boost reaches deeper into a boosted arm
|
||||
# before the cut without displacing the head of the other arms (#3956).
|
||||
from .search.recall_boost import boosted_rrf_score
|
||||
|
||||
strategy_boosts = get_config().recall_strategy_boosts
|
||||
merged_candidates.sort(key=lambda mc: boosted_rrf_score(mc, strategy_boosts), reverse=True)
|
||||
pre_filtered_count = len(merged_candidates) - max_candidates
|
||||
merged_candidates = merged_candidates[:max_candidates]
|
||||
if tracer:
|
||||
# Surface the cut in the trace: which arms actually made it into
|
||||
# the reranker's budget, and whether a boost shaped that. Ranking
|
||||
# complaints land on the trace first, and without this the boost
|
||||
# is only visible in server logs (issue #3956). Cheap: source_ranks
|
||||
# is already in memory and payloads are not materialized until below.
|
||||
arm_composition: dict[str, int] = {}
|
||||
for mc in merged_candidates:
|
||||
for key in mc.source_ranks:
|
||||
arm = key.removesuffix("_rank")
|
||||
arm_composition[arm] = arm_composition.get(arm, 0) + 1
|
||||
tracer.add_phase_metric(
|
||||
"rerank_prefilter",
|
||||
0.0,
|
||||
{
|
||||
"kept": len(merged_candidates),
|
||||
"dropped": pre_filtered_count,
|
||||
"max_candidates": max_candidates,
|
||||
"strategy_boosts": dict(strategy_boosts) if strategy_boosts else None,
|
||||
"arm_composition": arm_composition,
|
||||
},
|
||||
)
|
||||
|
||||
# Materialize the payload for the candidates that survived fusion, for a store
|
||||
# that returned scores rather than payloads. THIS is why ranking can be cheap: the
|
||||
|
||||
@@ -10,8 +10,9 @@ structurally different places that live on different score scales, so a single
|
||||
number could not mean the same thing in both. The level maps to a tuned
|
||||
:class:`BoostWeights` pair:
|
||||
|
||||
1. **Before the reranker cap** — :func:`boosted_rrf_score` uses ``BoostWeights.rrf``
|
||||
as a weighted-RRF multiplier on the boosted arm's rank contribution, so its
|
||||
1. **Before the reranker cap** — :func:`boosted_rrf_score` promotes the boosted
|
||||
arm in *rank space*: the arm's RRF contribution is recomputed as if the
|
||||
candidate had placed ``rank / rank_divisor`` instead of ``rank``, so its
|
||||
candidates survive the global reranker candidate budget instead of being
|
||||
trimmed by raw RRF score. Rank-aware: a candidate ranked #1 in the boosted
|
||||
arm is protected more than one ranked #200.
|
||||
@@ -22,6 +23,28 @@ number could not mean the same thing in both. The level maps to a tuned
|
||||
boosted arm's candidates up the final ordering.
|
||||
|
||||
Both functions are no-ops when ``boosts`` is empty, preserving current behaviour.
|
||||
|
||||
Why stage 1 boosts the rank and not the score
|
||||
---------------------------------------------
|
||||
The original implementation multiplied the arm's ``1/(k+rank)`` contribution by
|
||||
a weight ``w``. That is standard weighted RRF, but it interacts badly with the
|
||||
hard ``RERANKER_MAX_CANDIDATES`` cut that immediately follows it (issue #3956).
|
||||
|
||||
RRF with ``k=60`` is deliberately flat: across the whole 300-candidate cap window
|
||||
the score only spans ``1/61 -> 1/360``, a factor of 5.9. Any ``w`` above that
|
||||
spread exceeds the entire dynamic range of the rank term, so the sort degenerates
|
||||
into a *lexicographic* one — boosted arm first, rank merely a tiebreaker. ``high``
|
||||
was ``w=7``, over the line, and on a bank whose merged pool is far larger than the
|
||||
cap the boosted arm then filled all 300 slots and no semantic-only candidate ever
|
||||
reached the cross-encoder (measured: recall@20 0.97 -> 0.40).
|
||||
|
||||
The culprit is the ``k`` term. In score space the displacement reach is
|
||||
``r_max = w*(k+s) - k``, so at the head of the ranking the constant ``w*k``
|
||||
dominates and the boosted arm's ~360th hit outranks the other arm's *first*.
|
||||
Boosting the rank instead — ``1/(k + rank/w)`` — cancels ``k``: the boosted arm's
|
||||
rank ``r`` beats another arm's rank ``s`` iff ``r < w*s``. Displacement becomes
|
||||
strictly proportional rather than an absolute offset, so it can never invert the
|
||||
head of the ranking, and the behaviour no longer depends on the pool size.
|
||||
"""
|
||||
|
||||
from dataclasses import dataclass
|
||||
@@ -34,27 +57,33 @@ class BoostWeights:
|
||||
"""Per-stage boost magnitudes for one priority level.
|
||||
|
||||
The two fields live on different scales on purpose (see module docstring):
|
||||
``rrf`` multiplies an arm's ``1/(k+rank)`` RRF contribution; ``additive`` is
|
||||
added directly to the post-rerank weight in ~[0, 1].
|
||||
``rank_divisor`` divides an arm's rank before the ``1/(k+rank)`` RRF
|
||||
contribution is computed; ``additive`` is added directly to the post-rerank
|
||||
weight in ~[0, 1].
|
||||
"""
|
||||
|
||||
rrf: float
|
||||
rank_divisor: float
|
||||
additive: float
|
||||
|
||||
|
||||
# Priority level -> per-stage boost magnitudes. Tuned against real recall traces
|
||||
# (LoCoMo bank, 336 merged candidates → 300-cap, local ms-marco cross-encoder):
|
||||
# Priority level -> per-stage boost magnitudes.
|
||||
#
|
||||
# Stage 1 (rrf, weighted-RRF multiplier on the arm's 1/(k+rank) contribution).
|
||||
# The observed 300-cap boundary RRF score was ~0.0055; a graph-only candidate
|
||||
# falls below it past graph-rank ~120. The multipliers map to that boundary:
|
||||
# low=1.0 doubles the arm's vote — rescues at-risk candidates from the cut
|
||||
# (graph-rank 150: 0.0048 → 0.0095) without reshuffling much.
|
||||
# medium=3.0 promotes them into the middle of the pool (~rank 60).
|
||||
# high=6.0 makes the boosted arm dominate the top of the candidate pool.
|
||||
# Stage 1 (rank_divisor, applied in rank space: the arm's contribution becomes
|
||||
# 1/(k + rank/divisor)). A boosted candidate at arm-rank r outranks an unboosted
|
||||
# candidate at arm-rank s exactly when r < divisor * s, independent of k, of the
|
||||
# cap, and of the merged pool size. Simulated against 1000-deep arms and the
|
||||
# default 300-cap: the share of the reranker budget left to unboosted-only
|
||||
# candidates, and how deep into the boosted arm the cut still reaches:
|
||||
# (unboosted baseline: 150 slots each, boosted arm protected to rank 150)
|
||||
# low=2.0 100 slots left to other arms; boosted arm protected to rank 200.
|
||||
# medium=4.0 60 slots left; protected to rank 240.
|
||||
# high=8.0 33 slots left; protected to rank 267.
|
||||
# Every level keeps the *head* of every other arm — the top-ranked semantic hit
|
||||
# is only ever displaced by boosted hits from the arm's own top `divisor` ranks —
|
||||
# which is the property the score-space form could not offer at `high`.
|
||||
#
|
||||
# Stage 2 (additive, flat bump to the post-rerank weight in [0, 1]). The local
|
||||
# cross-encoder is sharply bimodal: strong direct matches score 0.5–0.999, while
|
||||
# cross-encoder is sharply bimodal: strong direct matches score 0.5-0.999, while
|
||||
# everything else — including graph hits the CE undervalues, which is exactly
|
||||
# what we boost — collapses near 0. So the additive lifts a ~0 candidate up the
|
||||
# weight scale. Levels are calibrated as relevance thresholds it can outrank:
|
||||
@@ -66,18 +95,18 @@ class BoostWeights:
|
||||
# The keys are the user-facing contract; config.py validates env input against
|
||||
# them (kept in sync by a guard test).
|
||||
BOOST_LEVELS: dict[str, BoostWeights] = {
|
||||
"low": BoostWeights(rrf=1.0, additive=0.05),
|
||||
"medium": BoostWeights(rrf=3.0, additive=0.2),
|
||||
"high": BoostWeights(rrf=6.0, additive=0.5),
|
||||
"low": BoostWeights(rank_divisor=2.0, additive=0.05),
|
||||
"medium": BoostWeights(rank_divisor=4.0, additive=0.2),
|
||||
"high": BoostWeights(rank_divisor=8.0, additive=0.5),
|
||||
}
|
||||
|
||||
|
||||
def boosted_rrf_score(candidate: MergedCandidate, boosts: dict[str, str], k: int = 60) -> float:
|
||||
"""Return ``candidate``'s RRF score plus a weighted-RRF boost delta.
|
||||
"""Return ``candidate``'s RRF score with boosted arms promoted in rank space.
|
||||
|
||||
For each boosted arm the candidate appeared in, adds ``level.rrf * 1/(k+rank)``
|
||||
— i.e. scales that arm's RRF contribution by the level's multiplier. Staying
|
||||
in RRF units keeps the boost comparable to the base score and rank-aware.
|
||||
For each boosted arm the candidate appeared in, replaces that arm's
|
||||
``1/(k+rank)`` contribution with ``1/(k + rank/divisor)`` — expressed as a
|
||||
delta so ``rrf_score`` stays authoritative and unboosted arms are untouched.
|
||||
|
||||
Args:
|
||||
candidate: Merged candidate carrying ``rrf_score`` and ``source_ranks``.
|
||||
@@ -94,7 +123,8 @@ def boosted_rrf_score(candidate: MergedCandidate, boosts: dict[str, str], k: int
|
||||
for strategy, level in boosts.items():
|
||||
rank = candidate.source_ranks.get(f"{strategy}_rank")
|
||||
if rank is not None:
|
||||
delta += BOOST_LEVELS[level].rrf * (1.0 / (k + rank))
|
||||
divisor = BOOST_LEVELS[level].rank_divisor
|
||||
delta += 1.0 / (k + rank / divisor) - 1.0 / (k + rank)
|
||||
return candidate.rrf_score + delta
|
||||
|
||||
|
||||
|
||||
@@ -27,7 +27,7 @@ def test_config_levels_match_boost_table():
|
||||
def test_levels_are_monotonic():
|
||||
"""Higher levels must boost more in both stages, or the names lie."""
|
||||
low, medium, high = (BOOST_LEVELS[lvl] for lvl in ("low", "medium", "high"))
|
||||
assert low.rrf < medium.rrf < high.rrf
|
||||
assert low.rank_divisor < medium.rank_divisor < high.rank_divisor
|
||||
assert low.additive < medium.additive < high.additive
|
||||
|
||||
|
||||
@@ -78,10 +78,13 @@ def test_boosted_rrf_noop_when_no_boosts():
|
||||
assert boosted_rrf_score(cand, {}) == 0.5
|
||||
|
||||
|
||||
def test_boosted_rrf_adds_weighted_contribution():
|
||||
cand = _candidate(0.5, {"graph_rank": 1})
|
||||
expected = 0.5 + BOOST_LEVELS["high"].rrf * (1.0 / 61)
|
||||
assert boosted_rrf_score(cand, {"graph": "high"}, k=60) == expected
|
||||
def test_boosted_rrf_promotes_the_arm_in_rank_space():
|
||||
"""The boosted arm contributes as if it had placed rank/divisor."""
|
||||
cand = _candidate(0.5, {"graph_rank": 8})
|
||||
divisor = BOOST_LEVELS["high"].rank_divisor
|
||||
# rank 8 at divisor 8 contributes as rank 1, replacing its rank-8 contribution.
|
||||
expected = 0.5 + (1.0 / (60 + 8 / divisor)) - (1.0 / 68)
|
||||
assert boosted_rrf_score(cand, {"graph": "high"}, k=60) == pytest.approx(expected)
|
||||
|
||||
|
||||
def test_boosted_rrf_higher_level_boosts_more():
|
||||
@@ -92,12 +95,27 @@ def test_boosted_rrf_higher_level_boosts_more():
|
||||
|
||||
|
||||
def test_boosted_rrf_is_rank_aware():
|
||||
"""A better rank in the boosted arm yields a larger boost."""
|
||||
top = _candidate(0.5, {"graph_rank": 1})
|
||||
deep = _candidate(0.5, {"graph_rank": 200})
|
||||
"""Boosting preserves the boosted arm's internal order.
|
||||
|
||||
Note the boost *delta* is deliberately largest deep in the arm, where the
|
||||
reranker cut bites — a rank-1 candidate needs no rescuing. So this asserts
|
||||
the invariant that matters, final-score monotonicity, using base scores
|
||||
consistent with the ranks (as fusion produces them) rather than a flat stub.
|
||||
"""
|
||||
top = _candidate(1.0 / 61, {"graph_rank": 1})
|
||||
deep = _candidate(1.0 / 260, {"graph_rank": 200})
|
||||
assert boosted_rrf_score(top, {"graph": "high"}) > boosted_rrf_score(deep, {"graph": "high"})
|
||||
|
||||
|
||||
def test_boost_delta_is_largest_where_the_cut_bites():
|
||||
"""The rescue is aimed at candidates near the cut, not at the arm's head."""
|
||||
top = _candidate(1.0 / 61, {"graph_rank": 1})
|
||||
deep = _candidate(1.0 / 260, {"graph_rank": 200})
|
||||
top_delta = boosted_rrf_score(top, {"graph": "high"}) - top.rrf_score
|
||||
deep_delta = boosted_rrf_score(deep, {"graph": "high"}) - deep.rrf_score
|
||||
assert deep_delta > top_delta
|
||||
|
||||
|
||||
def test_boosted_rrf_ignores_non_matching_arm():
|
||||
# Candidate only came from semantic; a graph boost must not touch it.
|
||||
cand = _candidate(0.5, {"semantic_rank": 3})
|
||||
@@ -124,3 +142,77 @@ def test_additive_sums_matched_arms():
|
||||
|
||||
def test_additive_ignores_unmatched_arm():
|
||||
assert additive_strategy_boost({"semantic_rank": 1}, {"graph": "high"}) == 0.0
|
||||
|
||||
|
||||
# --- #3956: the boost must not monopolise the reranker cap --------------------
|
||||
|
||||
|
||||
def _cut(level: str | None, cap: int = 300, arm_depth: int = 1000, k: int = 60) -> list[tuple[str, int]]:
|
||||
"""Merge a boosted arm and an unboosted arm, sort as recall does, take top ``cap``.
|
||||
|
||||
Mirrors ``memory_engine`` step 4's pre-filter: build the merged pool, sort by
|
||||
``boosted_rrf_score``, slice to the reranker candidate budget. Returns the
|
||||
surviving ``(arm, rank)`` pairs.
|
||||
"""
|
||||
boosts = {"graph": level} if level else {}
|
||||
pool = [_candidate(1.0 / (k + r), {"graph_rank": r}) for r in range(1, arm_depth + 1)]
|
||||
pool += [_candidate(1.0 / (k + s), {"semantic_rank": s}) for s in range(1, arm_depth + 1)]
|
||||
pool.sort(key=lambda mc: boosted_rrf_score(mc, boosts, k=k), reverse=True)
|
||||
survivors = []
|
||||
for mc in pool[:cap]:
|
||||
arm, rank = next(iter(mc.source_ranks.items()))
|
||||
survivors.append((arm.removesuffix("_rank"), rank))
|
||||
return survivors
|
||||
|
||||
|
||||
@pytest.mark.parametrize("level", ["low", "medium", "high"])
|
||||
def test_boost_never_starves_the_other_arm_at_the_cap(level):
|
||||
"""Regression for #3956: `graph:high` left zero semantic-only survivors.
|
||||
|
||||
The score-space form multiplied the arm's contribution by a weight larger
|
||||
than RRF's whole dynamic range over the cap window, so the sort degenerated
|
||||
to "boosted arm first" and the cut kept 300/300 graph candidates.
|
||||
"""
|
||||
semantic = [rank for arm, rank in _cut(level) if arm == "semantic"]
|
||||
assert semantic, f"{level} starved the unboosted arm out of the reranker budget"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("level", ["low", "medium", "high"])
|
||||
def test_boost_never_displaces_the_head_of_the_other_arm(level):
|
||||
"""No level may push the *top* unboosted hit out of the reranker budget.
|
||||
|
||||
This is the property that makes the boost safe on banks whose merged pool is
|
||||
far larger than ``RERANKER_MAX_CANDIDATES``: displacement is proportional to
|
||||
rank, so the head of every arm is preserved whatever the pool size.
|
||||
"""
|
||||
assert ("semantic", 1) in _cut(level)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("level", ["low", "medium", "high"])
|
||||
def test_boost_still_protects_the_arm_from_the_cut(level):
|
||||
"""The feature must still do its job: reach deeper into the boosted arm."""
|
||||
unboosted_depth = max(rank for arm, rank in _cut(None) if arm == "graph")
|
||||
boosted_depth = max(rank for arm, rank in _cut(level) if arm == "graph")
|
||||
assert boosted_depth > unboosted_depth
|
||||
|
||||
|
||||
def test_higher_levels_protect_the_arm_more_deeply():
|
||||
depths = [max(rank for arm, rank in _cut(lvl) if arm == "graph") for lvl in ("low", "medium", "high")]
|
||||
assert depths[0] < depths[1] < depths[2]
|
||||
|
||||
|
||||
def test_rank_boost_crossover_is_independent_of_k():
|
||||
"""`r < divisor * s` must hold whatever RRF constant fusion was run with.
|
||||
|
||||
The score-space form's crossover carried a `w*k` term, which is why the
|
||||
damage scaled with k and surprised on real banks; the rank-space form must
|
||||
not depend on k at all.
|
||||
"""
|
||||
divisor = BOOST_LEVELS["medium"].rank_divisor
|
||||
for k in (10, 60, 200):
|
||||
boosted_wins = _candidate(1.0 / (k + 39), {"graph_rank": 39})
|
||||
boosted_loses = _candidate(1.0 / (k + 41), {"graph_rank": 41})
|
||||
rival = _candidate(1.0 / (k + 10), {"semantic_rank": 10}) # crossover at r = 4*10 = 40
|
||||
assert boosted_rrf_score(boosted_wins, {"graph": "medium"}, k=k) > boosted_rrf_score(rival, {}, k=k)
|
||||
assert boosted_rrf_score(boosted_loses, {"graph": "medium"}, k=k) < boosted_rrf_score(rival, {}, k=k)
|
||||
assert divisor == 4.0
|
||||
|
||||
@@ -1346,7 +1346,7 @@ For advanced authentication (JWT, OAuth, multi-tenant schemas), implement a cust
|
||||
| `HINDSIGHT_API_SEMANTIC_LINK_MIN_SIMILARITY` | Minimum cosine similarity for creating semantic links during normal retain, streaming retain, and graph-maintenance relinking. This directly controls semantic graph density. Must be between `0` and `1`. | `0.7` |
|
||||
| `HINDSIGHT_API_BM25_MIN_SCORE` | Minimum BM25 score a row must exceed to enter fusion. Gates out zero-score, non-matching rows on backends (notably `vchord`) whose operator ranks every document instead of pre-filtering to query-term matches. `0` keeps only genuine term matches; raise it to require stronger matches. | `0` |
|
||||
| `HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE` | Cap on candidates each retrieval source (semantic, BM25, graph, temporal) contributes to RRF, applied before the global reranker cap. Prevents one over-expanding backend from filling the reranker budget on its own. `0` disables the cap. | `0` |
|
||||
| `HINDSIGHT_API_RECALL_STRATEGY_BOOSTS` | Prioritise one or more retrieval sources over the others on recall, as a comma-separated `strategy:level` list (e.g. `graph:high` to strongly favour graph hits, or `graph:high,bm25:low`). Strategies: `semantic`, `bm25`, `graph`, `temporal`. Levels: `low` (gentle — mainly protects the source's candidates from being dropped before reranking), `medium` (moderate preference), `high` (strong — the source dominates the candidate pool and outranks most other matches, only a strong direct match still wins). The boost is applied in two places: before the reranker cap (so favoured candidates survive the `HINDSIGHT_API_RERANKER_MAX_CANDIDATES` budget) and after reranking (to nudge them up the final order); a named level is used because those two stages live on different score scales. Only the strategies you list are boosted — any you omit keep their normal weight (no implicit boost). A strategy written without a level (`graph` or `graph:`) defaults to `medium`. Empty disables the feature. | _(empty)_ |
|
||||
| `HINDSIGHT_API_RECALL_STRATEGY_BOOSTS` | Prioritise one or more retrieval sources over the others on recall, as a comma-separated `strategy:level` list (e.g. `graph:high` to strongly favour graph hits, or `graph:high,bm25:low`). Strategies: `semantic`, `bm25`, `graph`, `temporal`. Levels: `low` (gentle — mainly protects the source's candidates from being dropped before reranking), `medium` (moderate preference), `high` (strong — the source takes the large majority of the reranker's candidate budget and outranks most other matches, only a strong direct match still wins). The pre-cap boost works in rank space: a boosted candidate at rank `r` outranks another arm's candidate at rank `s` when `r < divisor * s` (divisors: `low` 2, `medium` 4, `high` 8). Displacement is therefore proportional to rank — the top-ranked hits of the other arms always survive the cut — and does not change with the size of your bank's merged candidate pool. The boost is applied in two places: before the reranker cap (so favoured candidates survive the `HINDSIGHT_API_RERANKER_MAX_CANDIDATES` budget) and after reranking (to nudge them up the final order); a named level is used because those two stages live on different score scales. Only the strategies you list are boosted — any you omit keep their normal weight (no implicit boost). A strategy written without a level (`graph` or `graph:`) defaults to `medium`. Empty disables the feature. | _(empty)_ |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_FUNCTION` | Shape of the recency boost applied during reranking — how a memory's age is turned into a small freshness adjustment to its final rank. `linear` (default) decays in a straight line from full freshness (today) to a floor reached at `HINDSIGHT_API_RECENCY_DECAY_LINEAR_WINDOW_DAYS`. `exponential` decays by half-life: a memory is treated as neutral (no boost or penalty) at `HINDSIGHT_API_RECENCY_DECAY_HALFLIFE_DAYS`, younger memories are boosted and older ones penalised, with a smooth fade rather than a hard cutoff. `none` disables recency entirely (age never affects ranking). | `linear` |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_LINEAR_WINDOW_DAYS` | For the `linear` decay function: the number of days over which a memory fades from full freshness to the minimum. Only used when `HINDSIGHT_API_RECENCY_DECAY_FUNCTION=linear`. | `365` |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_HALFLIFE_DAYS` | For the `exponential` decay function: the age (in days) at which a memory is considered neutral — younger memories get a recency boost, older ones a penalty. Smaller values favour very recent memories more aggressively. Only used when `HINDSIGHT_API_RECENCY_DECAY_FUNCTION=exponential`. | `90` |
|
||||
|
||||
@@ -267,7 +267,7 @@ The first memory ranks higher because it has **consensus** across strategies.
|
||||
|
||||
RRF gives a good initial ranking, but it's based on positions, not on deep query-document understanding. The cross-encoder evaluates each candidate against the query as a pair, producing a relevance score.
|
||||
|
||||
**Pre-filtering:** Before reranking, candidates are trimmed to the top **300** (by RRF score) to limit computational cost. This is configurable via `HINDSIGHT_API_RERANKER_MAX_CANDIDATES`. If [`HINDSIGHT_API_RECALL_STRATEGY_BOOSTS`](./configuration) is set, the boost is applied to the RRF scores before this cut, so candidates from a favoured source are more likely to survive it.
|
||||
**Pre-filtering:** Before reranking, candidates are trimmed to the top **300** (by RRF score) to limit computational cost. This is configurable via `HINDSIGHT_API_RERANKER_MAX_CANDIDATES`. If [`HINDSIGHT_API_RECALL_STRATEGY_BOOSTS`](./configuration) is set, the boost is applied before this cut, so candidates from a favoured source are more likely to survive it. The boost promotes the favoured arm in *rank* space (its rank is divided by the level's divisor before the RRF contribution is computed) rather than scaling its score, so it reaches deeper into that arm without evicting the top-ranked hits of the others — including on banks whose merged pool is many times the cap. When `trace: true` is requested, the `rerank_prefilter` phase reports how many candidates were kept and dropped, the cap in force, the active boosts, and the per-arm composition of the survivors.
|
||||
|
||||
**Why rerank after RRF?** RRF is position-based — it knows a memory ranked well across strategies, but it never actually reads the query and the memory together. The cross-encoder does: it takes the query and each candidate as a pair and produces a relevance score based on their full interaction. This catches nuances that position-based fusion misses, like a memory that ranked #1 in keyword search because it matched a common term but is actually irrelevant to the query's intent.
|
||||
|
||||
|
||||
@@ -1346,7 +1346,7 @@ For advanced authentication (JWT, OAuth, multi-tenant schemas), implement a cust
|
||||
| `HINDSIGHT_API_SEMANTIC_LINK_MIN_SIMILARITY` | Minimum cosine similarity for creating semantic links during normal retain, streaming retain, and graph-maintenance relinking. This directly controls semantic graph density. Must be between `0` and `1`. | `0.7` |
|
||||
| `HINDSIGHT_API_BM25_MIN_SCORE` | Minimum BM25 score a row must exceed to enter fusion. Gates out zero-score, non-matching rows on backends (notably `vchord`) whose operator ranks every document instead of pre-filtering to query-term matches. `0` keeps only genuine term matches; raise it to require stronger matches. | `0` |
|
||||
| `HINDSIGHT_API_RECALL_MAX_CANDIDATES_PER_SOURCE` | Cap on candidates each retrieval source (semantic, BM25, graph, temporal) contributes to RRF, applied before the global reranker cap. Prevents one over-expanding backend from filling the reranker budget on its own. `0` disables the cap. | `0` |
|
||||
| `HINDSIGHT_API_RECALL_STRATEGY_BOOSTS` | Prioritise one or more retrieval sources over the others on recall, as a comma-separated `strategy:level` list (e.g. `graph:high` to strongly favour graph hits, or `graph:high,bm25:low`). Strategies: `semantic`, `bm25`, `graph`, `temporal`. Levels: `low` (gentle — mainly protects the source's candidates from being dropped before reranking), `medium` (moderate preference), `high` (strong — the source dominates the candidate pool and outranks most other matches, only a strong direct match still wins). The boost is applied in two places: before the reranker cap (so favoured candidates survive the `HINDSIGHT_API_RERANKER_MAX_CANDIDATES` budget) and after reranking (to nudge them up the final order); a named level is used because those two stages live on different score scales. Only the strategies you list are boosted — any you omit keep their normal weight (no implicit boost). A strategy written without a level (`graph` or `graph:`) defaults to `medium`. Empty disables the feature. | _(empty)_ |
|
||||
| `HINDSIGHT_API_RECALL_STRATEGY_BOOSTS` | Prioritise one or more retrieval sources over the others on recall, as a comma-separated `strategy:level` list (e.g. `graph:high` to strongly favour graph hits, or `graph:high,bm25:low`). Strategies: `semantic`, `bm25`, `graph`, `temporal`. Levels: `low` (gentle — mainly protects the source's candidates from being dropped before reranking), `medium` (moderate preference), `high` (strong — the source takes the large majority of the reranker's candidate budget and outranks most other matches, only a strong direct match still wins). The pre-cap boost works in rank space: a boosted candidate at rank `r` outranks another arm's candidate at rank `s` when `r < divisor * s` (divisors: `low` 2, `medium` 4, `high` 8). Displacement is therefore proportional to rank — the top-ranked hits of the other arms always survive the cut — and does not change with the size of your bank's merged candidate pool. The boost is applied in two places: before the reranker cap (so favoured candidates survive the `HINDSIGHT_API_RERANKER_MAX_CANDIDATES` budget) and after reranking (to nudge them up the final order); a named level is used because those two stages live on different score scales. Only the strategies you list are boosted — any you omit keep their normal weight (no implicit boost). A strategy written without a level (`graph` or `graph:`) defaults to `medium`. Empty disables the feature. | _(empty)_ |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_FUNCTION` | Shape of the recency boost applied during reranking — how a memory's age is turned into a small freshness adjustment to its final rank. `linear` (default) decays in a straight line from full freshness (today) to a floor reached at `HINDSIGHT_API_RECENCY_DECAY_LINEAR_WINDOW_DAYS`. `exponential` decays by half-life: a memory is treated as neutral (no boost or penalty) at `HINDSIGHT_API_RECENCY_DECAY_HALFLIFE_DAYS`, younger memories are boosted and older ones penalised, with a smooth fade rather than a hard cutoff. `none` disables recency entirely (age never affects ranking). | `linear` |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_LINEAR_WINDOW_DAYS` | For the `linear` decay function: the number of days over which a memory fades from full freshness to the minimum. Only used when `HINDSIGHT_API_RECENCY_DECAY_FUNCTION=linear`. | `365` |
|
||||
| `HINDSIGHT_API_RECENCY_DECAY_HALFLIFE_DAYS` | For the `exponential` decay function: the age (in days) at which a memory is considered neutral — younger memories get a recency boost, older ones a penalty. Smaller values favour very recent memories more aggressively. Only used when `HINDSIGHT_API_RECENCY_DECAY_FUNCTION=exponential`. | `90` |
|
||||
|
||||
@@ -267,7 +267,7 @@ The first memory ranks higher because it has **consensus** across strategies.
|
||||
|
||||
RRF gives a good initial ranking, but it's based on positions, not on deep query-document understanding. The cross-encoder evaluates each candidate against the query as a pair, producing a relevance score.
|
||||
|
||||
**Pre-filtering:** Before reranking, candidates are trimmed to the top **300** (by RRF score) to limit computational cost. This is configurable via `HINDSIGHT_API_RERANKER_MAX_CANDIDATES`. If [`HINDSIGHT_API_RECALL_STRATEGY_BOOSTS`](./configuration) is set, the boost is applied to the RRF scores before this cut, so candidates from a favoured source are more likely to survive it.
|
||||
**Pre-filtering:** Before reranking, candidates are trimmed to the top **300** (by RRF score) to limit computational cost. This is configurable via `HINDSIGHT_API_RERANKER_MAX_CANDIDATES`. If [`HINDSIGHT_API_RECALL_STRATEGY_BOOSTS`](./configuration) is set, the boost is applied before this cut, so candidates from a favoured source are more likely to survive it. The boost promotes the favoured arm in *rank* space (its rank is divided by the level's divisor before the RRF contribution is computed) rather than scaling its score, so it reaches deeper into that arm without evicting the top-ranked hits of the others — including on banks whose merged pool is many times the cap. When `trace: true` is requested, the `rerank_prefilter` phase reports how many candidates were kept and dropped, the cap in force, the active boosts, and the per-arm composition of the survivors.
|
||||
|
||||
**Why rerank after RRF?** RRF is position-based — it knows a memory ranked well across strategies, but it never actually reads the query and the memory together. The cross-encoder does: it takes the query and each candidate as a pair and produces a relevance score based on their full interaction. This catches nuances that position-based fusion misses, like a memory that ranked #1 in keyword search because it matched a common term but is actually irrelevant to the query's intent.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user