mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Nicolò Boschi
5be9ad9156
* diag: measure event-loop lag, to tell a slow await from a busy loop
Every per-phase timer in recall measures its own await, so a request that is runnable but not
running is invisible to all of them: the phases stay fast while the total inflates. Measured on
this API under load, the instrumented phases covered 10% of a recall's wall time (store hop 32ms,
embedding 5ms of a 356ms mean) and no candidate I/O accounted for the other 90% — CPU, Postgres,
the embedding service, the store, auth and the recall semaphore were each measured and excluded.
Those two explanations need different fixes and look identical from the phase timings, so this
measures the difference directly. The probe sleeps a known interval and reports the overshoot,
which is time the loop spent elsewhere while the probe was ready. Lag near zero means a real await
is missing a timer; lag tracking request latency means the loop is oversubscribed and tuning I/O
will not help.
Off unless HINDSIGHT_API_LOOP_LAG is set, and started from the lifespan hook so it runs on the
loop that serves requests.
* diag(recall): time the tenant auth await, and stop post_recall being zero by construction
Two blind spots on the recall path, found while chasing 90% of a recall's wall time that no phase
timer accounted for (store hop 32ms, embedding 5ms of a 356ms mean, event-loop lag 0.2ms — so a
real await, not a busy loop).
`recall_duration` was measured at the END of the handler, so it already contained response
building, and `post_recall = handler_duration - pre_recall - recall_duration` was ~0 by
construction. The line reported pre=0 post=0 for every request and charged everything to
`recall=`, whichever layer actually spent it. It now ends where the engine call returns.
`recall_async` awaits `_authenticate_tenant` before any phase timer starts, so that call was
invisible to both the phase metrics and the handler's split. It is timed and logged when it
exceeds 25ms.
Diagnostics only; no behaviour change.
* diag(recall): time the four untimed awaits in recall_async
`recall_async` awaits four things that no phase timer covers: the operation-validator pre-hook
(`validate_recall`), fuzzy tag-group resolution, the bank-config read, and the validator post-hook
(`on_recall_complete`). Together with the already-timed embedding and store hop they are the whole
request, so with them dark the phase metrics accounted for only ~10% of a recall's wall time and
the remaining 90% looked like it belonged to some I/O nobody had instrumented.
Each is timed and logged over 25ms. Diagnostics only; no behaviour change.
Note for whoever reads the numbers: measured through a port-forward these are dominated by round
trip latency, so the useful output is WHICH await dominates in-cluster, not the millisecond values
from a laptop.
* diag(recall): trace the whole request, not the parts that already had timers
The recall phase metrics accounted for ~10% of a request's wall time on a loaded fleet — store hop
32ms and embedding 5ms of a 356ms mean — and the missing 90% survived every candidate: CPU at 4%,
both databases idle with sub-millisecond queries, TEI at 9%, the store at 0.6%, the recall
semaphore at ~0, connection pinning ruled out by a keep-alive A/B, and event-loop lag flat at
0.2ms (so a real await, not a busy loop). Each candidate was excluded one build at a time, which
is slow and only ever rules things out.
This times the whole path instead:
* `http_to_handler` — ASGI entry to the endpoint body. `handler_start` is set INSIDE the endpoint,
so routing, body parsing and dependency resolution (auth among them) sat outside every existing
timer and read as unattributed time.
* `engine_call` — the whole engine call, marked diagnostic because it contains the store and
embedding phases and would otherwise double-count.
* `post_engine` — from the engine returning to the response being built.
* `validate_pre`, `bank_config`, `fuzzy_tags`, `validate_post` — the four awaits in `recall_async`
that no phase covered.
They are metrics, not logs over a threshold: a `>25ms` line shows only the tail and cannot
distinguish "small and constant" from "small and rare", which is how a 30ms phase on 2% of
requests briefly looked like an explanation for 319ms on all of them.
`recall_duration` also now ends where the engine call returns rather than at the end of the
handler, so `post_recall` — computed as the remainder — stops being ~0 by construction.
Diagnostics only; no behaviour change.
* diag(recall): time the two awaits in the precheck dependency
`http_to_handler` — ASGI entry to the endpoint body — turned out to be a third of a recall, and
nothing inside it was timed. It contains the `precheck_for` dependency, which the docstring says
"authenticates the tenant" before the body is read, so the FIRST and uncached `_authenticate_tenant`
call happens here. The one inside `recall_async` is a cached re-resolution, which is why auth kept
measuring cheap and kept being cleared as a suspect.
Adds `dep_auth` and `dep_precheck`. Measured locally through a port-forward, `dep_auth` is 76% of
the pre-handler time and the largest single phase of the request — larger than the whole engine
call. Round-trip latency inflates the absolute value; the ranking is the finding.
Diagnostics only; no behaviour change.
* diag(recall): close the accounting — the missing time is the metering post-hook
The recall waterfall had ~950ms (45% of the engine call) that no phase covered, with the API at
1% of one core and every await, async-with and async-for in `recall_async` already timed. The
cause was an instrumentation bug of my own: `on_recall_complete` has THREE call sites and only the
first was wrapped, so `validate_post` recorded 0.0ms while the path actually taken went through
one of the other two.
With all three timed the residual is 1ms, and the breakdown is:
validate_post 968 ms 45% metering post-hook
search_with_retries 529 ms 24% the store + embedding
validate_pre 404 ms 19% metering pre-hook
bank_config 265 ms 12%
engine_auth, fuzzy_tags, semaphore_acquire, validate_post ~ 0
Measured through a port-forward, so the absolute numbers are round-trip dominated; the shares are
the finding. Metering is 63% of the engine call and is two synchronous control-plane hops per
recall — a credits read before the work and a credits UPDATE plus `usage_records` and
`crm_milestones` inserts after it.
Also splits `http_to_handler` into `mw_and_routing` / `deps_total` / `body_parse`, which closes
that region to a 0ms residual, and adds `recall_async_body` to separate "inside the coroutine"
from "between the handler's timer and the body running" (the latter is 0).
Diagnostics only; no behaviour change.
* diag(recall): time the hop minus the store's own reported stages
A store-answered recall reports the store's per-stage timings and `full_recall`, the whole
hop. The gap between them is ours -- the client, the serialization either side, and any
time the request sat in the channel -- and there was no way to read it.
It cannot be derived after the fact: percentiles of separate phases are not additive, so
subtracting one phase's p99 from another's says nothing. Recorded per request instead.
On a measured window it is 4.8 ms mean and 10 ms at p99, which is what rules the client
out as the source of a 1.9 s request tail.
* perf(metrics): record a phase once, not into a histogram and a counter
`record_recall_phase` and `record_validator_phase` each wrote the same measurement twice:
into a duration histogram, and into a parallel counter of invocations. The histogram
already carries `_count` for the identical attribute set, so the counter was a second copy
of a number that was never missing.
It is not free. A py-spy profile of the API under recall load put OpenTelemetry's
`consume_measurement` at 8.9% of the process's busy CPU, and `record_recall_phase` alone
at 5.35% -- the aggregation path, not the record call, is the cost, and it ran twice.
`hindsight.recall.phase.calls` and `hindsight.validator.phase.calls` are removed rather
than left emitting: a metric that exists and is never written is worse than one that is
gone. Use `hindsight_recall_phase_duration_seconds_count`, which has the same value.
Also adds `HINDSIGHT_API_RECALL_DIAGNOSTIC_PHASES=false` to drop the subset phases, which
are the bulk of the instruments on a busy path and are only wanted while diagnosing.
test_profiling sliced the histogram's argument block to the counter that follows it; the
delimiter moves to the next instrument.
* perf(api): a tunable gzip floor, and no TLS setup for a plaintext embedder
Two things a CPU profile of a recall-heavy API found, neither of which buys anything on a
deployment that is CPU-bound rather than bandwidth-bound:
GZipMiddleware compressed every response over 1 KB, which a recall always is. That was
~5% of the request's CPU. `HINDSIGHT_API_GZIP_MIN_SIZE` raises the floor, and a negative
value drops the middleware. Default is unchanged at 1024.
`httpx.Client()` builds an SSLContext and loads the system CA bundle whatever the scheme,
and an in-cluster TEI is plain http://. `ssl.load_default_certs` showed up in the profile
for exactly that, paid again for each thread the pool retires and recreates. Skipped when
the base URL is http://; an https:// TEI verifies exactly as before.
* style: ruff format the recall phase timers
verify-generated-files runs the lint hook and then fails on any resulting diff; these four
files were committed unformatted. Formatting only, no behaviour change.
* review(recall-perf): route the new knobs through config, tidy the timers, add tests
- HINDSIGHT_API_GZIP_MIN_SIZE / RECALL_DIAGNOSTIC_PHASES / LOOP_LAG_REPORT_SECONDS
(renamed from LOOP_LAG) are HindsightConfig fields now, documented and in .env.example,
instead of ad-hoc os.environ reads.
- get_request_context: the timestamp line sat above the docstring, which demoted it
to a no-op string.
- _bind_bank_id decides once per function whether to time the recall body, and no
longer swallows exceptions from the metrics call.
- Reuse semaphore_wait_start / backend_acquire_start instead of parallel timers.
- loop_lag: keep a strong reference to the probe task, drop the noqa lambda.
- Tests: diagnostic-phase flag, TEI verify for http vs https, loop-lag probe.
- ruff format (the verify-generated-files failure).
633 lines
41 KiB
Bash
633 lines
41 KiB
Bash
# Hindsight Environment Variables
|
|
# Copy this file to .env and fill in your values
|
|
|
|
# LLM Configuration (Required)
|
|
# Supported providers: openai, openai-responses, groq, ollama, gemini, anthropic, lmstudio, vertexai, minimax, deepseek, zai, atlas, meta, volcano, openai-codex, claude-code, github-copilot
|
|
HINDSIGHT_API_LLM_PROVIDER=openai
|
|
HINDSIGHT_API_LLM_API_KEY=your-api-key-here
|
|
HINDSIGHT_API_LLM_MODEL=gpt-4o-mini
|
|
# HINDSIGHT_API_LLM_BASE_URL=https://api.openai.com/v1
|
|
|
|
# Vision slot — the model used for retain chunks that carry an inline attachment.
|
|
# Only those chunks use it; every text-only chunk stays on the retain LLM, so a
|
|
# bank ingesting the occasional screenshot need not run (or pay for) a vision
|
|
# model on all of its text. Unset, attachments go to the retain LLM as before.
|
|
# HINDSIGHT_API_VLM_PROVIDER=gemini
|
|
# HINDSIGHT_API_VLM_API_KEY=your-vision-api-key
|
|
# HINDSIGHT_API_VLM_MODEL=gemini-2.5-flash
|
|
# HINDSIGHT_API_VLM_BASE_URL=
|
|
# Reasoning effort for providers/models that support it. Examples: none, low, medium, high, xhigh.
|
|
# Set it and the value is sent as given, whatever the model is called — use `none` to stop a
|
|
# self-hosted reasoning model (vLLM, Ollama, llama.cpp, TGI) emitting thinking blocks. Unset,
|
|
# no reasoning parameter is sent at all and each model runs at its own default effort.
|
|
# HINDSIGHT_API_LLM_REASONING_EFFORT=low
|
|
|
|
# Sampling temperature for internal LLM calls. Set a number in [0.0, 2.0], or `none`
|
|
# to omit the temperature parameter entirely (required for models that reject explicit
|
|
# temperatures, e.g. Azure gpt-5.5). The global override below applies to every operation;
|
|
# per-operation overrides (defaults: verification=0.0, retain=0.1, reflect=0.9,
|
|
# consolidation=0.0) take precedence.
|
|
# HINDSIGHT_API_LLM_TEMPERATURE=none
|
|
# HINDSIGHT_API_LLM_TEMPERATURE_VERIFICATION=0.0
|
|
# HINDSIGHT_API_LLM_TEMPERATURE_RETAIN=0.1
|
|
# HINDSIGHT_API_LLM_TEMPERATURE_REFLECT=0.9
|
|
# HINDSIGHT_API_LLM_TEMPERATURE_CONSOLIDATION=0.0
|
|
|
|
# Grammar-enforce structured output (json_schema strict) instead of the soft
|
|
# schema-in-prompt path. Helps weaker self-hosted models that emit prose preambles
|
|
# or invalid JSON. The global override below applies to every operation;
|
|
# per-operation overrides take precedence, in both directions -- set one to false
|
|
# to opt that operation out while the global flag is on.
|
|
# HINDSIGHT_API_LLM_STRICT_SCHEMA=false
|
|
# HINDSIGHT_API_LLM_STRICT_SCHEMA_RETAIN=true
|
|
# HINDSIGHT_API_LLM_STRICT_SCHEMA_REFLECT=true
|
|
# HINDSIGHT_API_LLM_STRICT_SCHEMA_CONSOLIDATION=true
|
|
|
|
# Ceiling on the connect phase (TCP + TLS handshake) of an LLM request, in seconds.
|
|
# Read, write and pool keep the full HINDSIGHT_API_LLM_TIMEOUT budget, so an unreachable
|
|
# endpoint fails fast instead of burning the whole request timeout. 0 disables the cap.
|
|
# HINDSIGHT_API_LLM_CONNECT_TIMEOUT=10
|
|
|
|
# Log level for the httpx/httpcore loggers. DEBUG traces every LLM request through its
|
|
# transport phases (connect_tcp, send_request_headers, receive_response_headers), which
|
|
# is how you tell a request that stalled before being sent from one never answered.
|
|
# HINDSIGHT_API_LLM_HTTP_LOG_LEVEL=WARNING
|
|
|
|
# Some backends, including Bedrock Converse, reject JSON Schema maxItems.
|
|
# Disable it only for those backends; consolidation still enforces the cap.
|
|
# HINDSIGHT_API_LLM_SUPPORTS_MAX_ITEMS=true
|
|
|
|
# Constrain retain's occurred_start/occurred_end to an ISO timestamp with a JSON
|
|
# Schema pattern. Stops grammar-constrained models reasoning inside the timestamp
|
|
# string and burning the whole completion budget. Off by default: backends that
|
|
# validate schemas against an allowlist (Bedrock) reject the keyword with a 400.
|
|
# HINDSIGHT_API_LLM_SUPPORTS_STRING_PATTERN=false
|
|
|
|
# Pin a conversation to one backend prompt cache (OpenAI-compatible providers only).
|
|
# Server-side prompt caches are per backend server, so the same conversation has to
|
|
# reach the same one to hit. Values: auto (default), xai_conv_id (sends xAI's
|
|
# x-grok-conv-id header), openai_prompt_cache_key (sends OpenAI's prompt_cache_key
|
|
# field), none (sends nothing). "auto" picks from the base URL host and is an
|
|
# allowlist: x.ai / grok.com get the header, native OpenAI / openai.com / Azure
|
|
# OpenAI get the field, and every other backend gets nothing. Per-operation
|
|
# overrides take precedence. Set to none to disable entirely.
|
|
# HINDSIGHT_API_LLM_CACHE_AFFINITY=auto
|
|
# HINDSIGHT_API_RETAIN_LLM_CACHE_AFFINITY=none
|
|
# HINDSIGHT_API_REFLECT_LLM_CACHE_AFFINITY=xai_conv_id
|
|
# HINDSIGHT_API_CONSOLIDATION_LLM_CACHE_AFFINITY=none
|
|
|
|
# Ask litellm/litellmrouter/bedrock for structured output via a forced tool call
|
|
# instead of response_format. Enable it for backends that reject response_format
|
|
# outright -- e.g. Bedrock Claude in ap-southeast-2 ("Extra inputs are not permitted");
|
|
# the same model in us-east-1 accepts response_format and needs nothing here.
|
|
# HINDSIGHT_API_LLM_STRUCTURED_OUTPUT_FORCED_TOOL=false
|
|
|
|
# Transport-level output cap for reflect's final synthesis call. Unset = uncapped:
|
|
# the model runs to a natural stop and the reflect/mental-model max_tokens governs
|
|
# visible length via a prompt directive + a post-hoc rewrite (not by truncating the
|
|
# provider call, which on thinking models is eaten by reasoning tokens and cuts pages
|
|
# off mid-word). Set an integer only to enforce a hard cost ceiling on the call.
|
|
# HINDSIGHT_API_REFLECT_MAX_COMPLETION_TOKENS=16000
|
|
|
|
# Diagnostic: on any LLM 4xx, log the exact assembled request ([LLM_4XX_DUMP]) --
|
|
# serialized request config (message bodies stripped) + capped per-message previews.
|
|
# For debugging otherwise-unreproducible rejected calls. Off by default.
|
|
# HINDSIGHT_API_LLM_DEBUG_DUMP_4XX=false
|
|
|
|
# Example: Anthropic Claude configuration
|
|
# HINDSIGHT_API_LLM_PROVIDER=anthropic
|
|
# HINDSIGHT_API_LLM_API_KEY=your-anthropic-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=claude-sonnet-4-20250514
|
|
|
|
# Example: GitHub Copilot subscription via the official Copilot SDK
|
|
# Sign in with Copilot CLI first; no HINDSIGHT_API_LLM_API_KEY is needed.
|
|
# HINDSIGHT_API_LLM_PROVIDER=github-copilot
|
|
# HINDSIGHT_API_LLM_MODEL=gpt-5.6-terra
|
|
|
|
# Example: Google Vertex AI configuration
|
|
# HINDSIGHT_API_LLM_PROVIDER=vertexai
|
|
# HINDSIGHT_API_LLM_MODEL=google/gemini-2.0-flash-001
|
|
# HINDSIGHT_API_LLM_VERTEXAI_PROJECT_ID=your-gcp-project-id
|
|
# HINDSIGHT_API_LLM_VERTEXAI_REGION=us-central1
|
|
# HINDSIGHT_API_LLM_VERTEXAI_SERVICE_ACCOUNT_KEY=/path/to/service-account-key.json # Optional, uses ADC if not set
|
|
|
|
# Example: MiniMax configuration (1M context window)
|
|
# HINDSIGHT_API_LLM_PROVIDER=minimax
|
|
# HINDSIGHT_API_LLM_API_KEY=your-minimax-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=MiniMax-M3 # or MiniMax-M2.7 for the previous generation
|
|
|
|
# Example: OpenAI Responses API (/v1/responses) — reasoning + function tools together
|
|
# HINDSIGHT_API_LLM_PROVIDER=openai-responses
|
|
# HINDSIGHT_API_LLM_API_KEY=your-openai-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=gpt-5.6 # reasoning model (gpt-5.x / o-series); e.g. gpt-5.6-terra
|
|
# HINDSIGHT_API_LLM_REASONING_EFFORT=high # sent alongside tools, unlike chat/completions
|
|
|
|
# Example: DeepSeek configuration (https://api.deepseek.com)
|
|
# HINDSIGHT_API_LLM_PROVIDER=deepseek
|
|
# HINDSIGHT_API_LLM_API_KEY=your-deepseek-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=deepseek-v4-flash # or deepseek-v4-pro / deepseek-chat / deepseek-reasoner
|
|
|
|
# Example: z.ai configuration (Zhipu GLM series, https://z.ai)
|
|
# HINDSIGHT_API_LLM_PROVIDER=zai
|
|
# HINDSIGHT_API_LLM_API_KEY=your-zai-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=glm-4.5-flash # or glm-4.5-air for the paid tier
|
|
|
|
# Example: Atlas Cloud configuration (OpenAI-compatible, https://www.atlascloud.ai)
|
|
# HINDSIGHT_API_LLM_PROVIDER=atlas
|
|
# HINDSIGHT_API_LLM_API_KEY=your-atlascloud-api-key
|
|
# HINDSIGHT_API_LLM_MODEL=deepseek-ai/deepseek-v4-pro # reasoning model; also Qwen / GLM / Kimi / MiniMax, etc.
|
|
|
|
# Example: Meta Model API configuration (Muse Spark, OpenAI-compatible, https://ai.developer.meta.com)
|
|
# HINDSIGHT_API_LLM_PROVIDER=meta
|
|
# HINDSIGHT_API_LLM_API_KEY=your-meta-model-api-key # base_url defaults to https://api.meta.ai/v1
|
|
# HINDSIGHT_API_LLM_MODEL=muse-spark-1.3 # always-reasoning model; reasoning_effort "none" is rejected
|
|
|
|
# Example: LM Studio local configuration (Qwen 2.5 32B recommended)
|
|
# HINDSIGHT_API_LLM_PROVIDER=lmstudio
|
|
# HINDSIGHT_API_LLM_API_KEY=lmstudio
|
|
# HINDSIGHT_API_LLM_BASE_URL=http://localhost:1234/v1
|
|
# HINDSIGHT_API_LLM_MODEL=qwen2.5-32b-instruct
|
|
|
|
# Example: Ollama local configuration (native provider)
|
|
# HINDSIGHT_API_LLM_PROVIDER=ollama
|
|
# HINDSIGHT_API_LLM_BASE_URL=http://localhost:11434/v1
|
|
# HINDSIGHT_API_LLM_MODEL=gemma3:12b
|
|
# Native Ollama context-window override (num_ctx). Leave unset to let Ollama use
|
|
# the model Modelfile / server default; set a positive integer only to force a
|
|
# specific context size (e.g. 16384 to keep the previous request behavior).
|
|
# Setting it also routes free-form calls through Ollama's native /api/chat API,
|
|
# the only endpoint that can carry a context size.
|
|
# HINDSIGHT_API_LLM_OLLAMA_NUM_CTX=16384
|
|
|
|
# Example: OpenAI Codex (ChatGPT Plus/Pro OAuth, no API key)
|
|
# HINDSIGHT_API_LLM_PROVIDER=openai-codex
|
|
# Credentials directory for this provider (the dir holding auth.json). Overrides the
|
|
# process-wide CODEX_HOME. Set it, plus the indexed HINDSIGHT_API_LLM_<n>_CODEX_HOME
|
|
# below, to run two independently authorized ChatGPT profiles in one process.
|
|
# HINDSIGHT_API_LLM_CODEX_HOME=/var/lib/hindsight/codex-a
|
|
|
|
# Multi-LLM strategies: configure extra LLMs by index alongside the primary above,
|
|
# then pick a routing strategy. Unset = single primary LLM (default). Members are
|
|
# numbered from 1; indices must be contiguous. Each operation can override with a
|
|
# RETAIN_/REFLECT_/CONSOLIDATION_ prefix (e.g. HINDSIGHT_API_RETAIN_LLM_1_PROVIDER).
|
|
# HINDSIGHT_API_LLM_1_PROVIDER=groq
|
|
# HINDSIGHT_API_LLM_1_API_KEY=your-groq-api-key
|
|
# HINDSIGHT_API_LLM_1_MODEL=openai/gpt-oss-120b
|
|
# HINDSIGHT_API_LLM_2_PROVIDER=anthropic
|
|
# HINDSIGHT_API_LLM_2_API_KEY=your-anthropic-api-key
|
|
# An openai-codex member can point at its own credentials directory, so a chain of
|
|
# two Codex members fails over between two ChatGPT accounts instead of retrying one.
|
|
# HINDSIGHT_API_LLM_1_PROVIDER=openai-codex
|
|
# HINDSIGHT_API_LLM_1_CODEX_HOME=/var/lib/hindsight/codex-b
|
|
# Strategy JSON: {"mode": "failover"}, {"mode": "round-robin"}, or
|
|
# {"mode": "metadata", "routes": [{"key": "classification", "value": "sensitive", "member": 1}]}.
|
|
# Round-robin accepts optional positive-int "weights" (one per member, primary first).
|
|
# Metadata mode is RETAIN ONLY: each retained item picks a member from its own
|
|
# metadata (first matching route wins; no match uses the primary). It chooses which
|
|
# model extracts a document, not where that document's data can end up -- recall,
|
|
# reflect, consolidation and mental models all keep using the primary.
|
|
# HINDSIGHT_API_LLM_STRATEGY={"mode": "failover"}
|
|
|
|
# API Configuration (Optional)
|
|
HINDSIGHT_API_HOST=0.0.0.0
|
|
HINDSIGHT_API_PORT=8888
|
|
HINDSIGHT_API_LOG_LEVEL=info
|
|
# Min response bytes to gzip; negative disables compression (saves CPU on recall).
|
|
# HINDSIGHT_API_GZIP_MIN_SIZE=1024
|
|
# Diagnostic: log event-loop lag percentiles every N seconds. 0 disables.
|
|
# HINDSIGHT_API_LOOP_LAG_REPORT_SECONDS=0
|
|
# Record the diagnostic (subset) recall phase metrics.
|
|
# HINDSIGHT_API_RECALL_DIAGNOSTIC_PHASES=true
|
|
# Vocabulary used for every token count and chunk boundary (recall budgets, chunk
|
|
# sizes, prompt fitting, embedding truncation). o200k_base matches current OpenAI
|
|
# models; set cl100k_base to reproduce the counts Hindsight produced before this
|
|
# default changed. Also bundled: o200k_harmony.
|
|
# HINDSIGHT_API_TOKENIZER_ENCODING=o200k_base
|
|
# Optional retain chunking override for structured logs/transcripts.
|
|
# Unset uses HINDSIGHT_API_RETAIN_CHUNK_SIZE as the structured-chunk limit.
|
|
# HINDSIGHT_API_RETAIN_STRUCTURED_CHUNK_SIZE=
|
|
|
|
# Inline attachments in retain content (content as a list of text/image/file blocks).
|
|
# Whether the configured LLM can read images/files. Unset lets each provider answer for
|
|
# itself; a retain carrying images is refused when the answer is no or unknown.
|
|
# Set true for a vision model behind a gateway Hindsight cannot identify
|
|
# (litellm, ollama, lmstudio, an OpenAI-compatible proxy).
|
|
# HINDSIGHT_API_LLM_VISION=
|
|
# Max decoded size of a single inline attachment, in MB. Default 20.
|
|
# HINDSIGHT_API_RETAIN_ATTACHMENT_MAX_SIZE_MB=20
|
|
# Max inline attachments in one retain item. Default 50.
|
|
# HINDSIGHT_API_RETAIN_ATTACHMENT_MAX_COUNT=50
|
|
# Max attachments in one extraction chunk. retain_chunk_size budgets text only;
|
|
# this is what bounds attachments, matching a provider's per-request limit.
|
|
# Default 8.
|
|
# HINDSIGHT_API_RETAIN_MAX_ATTACHMENTS_PER_CHUNK=8
|
|
|
|
# When true, a retain operation that hit any fact-extraction errors is marked
|
|
# 'failed' (not 'completed'), surfacing silently-dropped facts. Default false.
|
|
# HINDSIGHT_API_FAIL_ON_EXTRACTION_ERRORS=false
|
|
|
|
# Wall-clock ceiling (seconds) for one retain task in the worker. A retain that
|
|
# blocks indefinitely is cancelled and marked 'failed' — and so becomes
|
|
# retryable — instead of holding its worker slot until the process restarts.
|
|
# Set well above your slowest healthy retain; 0 disables. Default 3600.
|
|
# HINDSIGHT_API_RETAIN_WALL_TIMEOUT=3600
|
|
|
|
# Ceiling (seconds) on how long one consolidation task in the worker may run
|
|
# WITHOUT making progress. Every batch that commits restarts the clock, so a large
|
|
# backlog is never cut short — only a stalled job is. A stalled consolidation is
|
|
# cancelled and marked 'failed' — and so becomes retryable, and the reconcile sweep
|
|
# can re-schedule the bank — instead of holding its reserved worker slot until the
|
|
# process restarts. 0 disables. Default 7200.
|
|
# HINDSIGHT_API_CONSOLIDATION_WALL_TIMEOUT=7200
|
|
|
|
# Megabytes of extracted-but-unwritten state ONE retain operation may hold. The chunk
|
|
# batch size bounds how many chunks are in flight, not what they weigh, so this is the
|
|
# figure to size a worker against: peak per retain is roughly this, whatever the document.
|
|
# Budget for HINDSIGHT_API_WORKER_MAX_SLOTS concurrent retains. Over budget, extraction
|
|
# waits for the write path instead of growing. 0 disables. Default 128.
|
|
# HINDSIGHT_API_RETAIN_MEMORY_BUDGET_MB=128
|
|
|
|
# Dry-run extraction preview endpoint (POST /memories/dry-run-extract). Enabled by default; it makes
|
|
# a real LLM call but stores nothing. Set to false to remove the endpoint (returns 404).
|
|
# HINDSIGHT_API_ENABLE_DRY_RUN_EXTRACT=true
|
|
|
|
# Base Path / Reverse Proxy Support (Optional)
|
|
# Set these when deploying behind a reverse proxy with path-based routing
|
|
# Example: To deploy at example.com/hindsight/, set both to "/hindsight"
|
|
# HINDSIGHT_API_BASE_PATH=/hindsight
|
|
# NEXT_PUBLIC_BASE_PATH=/hindsight
|
|
|
|
# Database (Optional - uses embedded pg0 by default)
|
|
# HINDSIGHT_API_DATABASE_URL=postgresql://user:pass@host:5432/db
|
|
# HINDSIGHT_API_READ_DATABASE_URL= # Optional read-replica URL. When set, recall queries (semantic, BM25, graph, temporal) flow through a separate pool against this URL, offloading the primary. Typically points to a read-only endpoint (CNPG's <cluster>-ro service or Aurora reader endpoint).
|
|
# HINDSIGHT_API_MIGRATION_DATABASE_URL= # Direct PostgreSQL URL for migrations (bypasses PgBouncer). Falls back to DATABASE_URL.
|
|
# HINDSIGHT_API_DATABASE_SCHEMA=public # PostgreSQL schema name (default: public)
|
|
# HINDSIGHT_API_DB_MAX_PARALLEL_WORKERS_PER_GATHER= # Optional cap on Postgres planner parallelism for this process's pool connections. Unset leaves the server default; 0 makes background/bulk queries run serially (useful on worker processes sharing a primary with latency-sensitive traffic).
|
|
# HINDSIGHT_API_DB_SESSION_SETUP_ON_ACQUIRE=true # Re-apply the per-connection session settings (statement_timeout, planner parallelism, trigram threshold, vector-search tuning, and the vchord search path) every time a connection is taken from the pool, not just when it is opened. Releasing a connection resets it to the server defaults, so turn this off only when the same settings are pinned on the role/database (ALTER ROLE ... SET) — then it is a pure round trip per acquire, worth reclaiming behind a transaction-mode pooler. On the vchord text-search backend the search path is in that set and losing it fails recall outright, so pin it too. application_name is always re-applied regardless.
|
|
# HINDSIGHT_API_ENTITY_TRGM_SIMILARITY_THRESHOLD=0.15 # Postgres pg_trgm.similarity_threshold applied on every pool connection, used by entity resolution's % trigram match. Must be in (0, 1]. Lower catches more substring-ish matches at higher CPU cost on large entity sets; higher is stricter and cheaper.
|
|
# HINDSIGHT_API_ENTITY_INTRABATCH_MERGE_SIMILARITY=0.5 # Trigram similarity (pg_trgm-equivalent, computed in-memory) at/above which two new names created by the SAME retain are merged into one entity (in-batch dedup of surface-form variants). Must be in (0, 1]. A merge cutoff, stricter than the recall threshold above; raise toward 1.0 to merge only near-identical forms.
|
|
# HINDSIGHT_API_ENTITY_MERGE_MIN_SIMILARITY=0.3 # Minimum trigram similarity a name must have with an EXISTING entity before that entity can be reused for it, whatever the other resolution signals (co-occurrence, recency) say. Must be in (0, 1]. Sits between the recall threshold (0.15) and the in-batch cutoff (0.5). Lower it for corpora of very short names; raise it to merge only clear surface variants.
|
|
# HINDSIGHT_API_RETAIN_ENTITY_RESOLUTION_MAX_CANDIDATES=200 # Max candidates scored per entity mention during retain. The fuzzy lookup keeps only this many best matches per name (ranked by trigram/Jaro-Winkler similarity) before scoring them one by one. On banks holding thousands of near-identical names an uncapped set turns one retain into minutes of CPU that stall the worker's health checks. Raise only if entities that should merge are being duplicated.
|
|
# HINDSIGHT_API_EXTERNALLY_OWNED_ROUTINES= # Comma-separated maintenance discovery routines this deployment installs itself, e.g. mental_models_with_cron,banks_needing_consolidation,schemas_with_expired_rows,schemas_with_expired_operations. Migrations skip anything named here and leave your CREATE OR REPLACE in place; without it the next migration that reinstalls the routine silently overwrites it. Empty (the default) installs every routine as usual. Naming a routine you have not installed leaves it missing, and the maintenance loop then fails loudly on it.
|
|
# HINDSIGHT_API_MIGRATION_CONCURRENCY=1 # Tenant schemas to migrate concurrently (PG only, each in its own process; per-schema work stays sequential). Each worker has ~1-2s startup cost + uses ~3 DB connections, so it only pays off with many schemas (tens+) or slow migrations; keep concurrency*3 <= spare max_connections. Default: 1 (sequential).
|
|
# HINDSIGHT_API_MIGRATION_ISOLATION=false # Run migrations in a subprocess instead of in the calling process: true | false. true keeps alembic's import graph and its psycopg2 sync engine out of a long-lived server process. Default: false.
|
|
# HINDSIGHT_API_OPERATION_RETENTION_DAYS=30 # Prune terminal operation rows, payloads, and metadata after this many days; 0 (the default) keeps them forever.
|
|
# HINDSIGHT_API_OPERATION_CLEANUP_BATCH_SIZE=1000 # Maximum expired terminal rows deleted per tenant schema in each cleanup cycle; must be positive.
|
|
|
|
# Background maintenance cadences (Optional)
|
|
# Each sweep begins with one cross-tenant discovery call that probes every schema holding the relevant
|
|
# table, in every API/worker process — so its cost scales with tenant count while the work it finds does
|
|
# not. On deployments with thousands of tenants these intervals are the knob to raise.
|
|
# HINDSIGHT_API_RETENTION_SWEEP_INTERVAL_SECONDS=3600 # How often expired audit_log / llm_requests rows are deleted across all tenant schemas. Retention is counted in days, so this only sets how promptly they disappear; 0 disables the sweeps.
|
|
# HINDSIGHT_API_OPERATION_CLEANUP_INTERVAL_SECONDS=900 # How often expired terminal operation rows are pruned; with the batch size above this sets the drain rate for a backlog. 0 disables the job.
|
|
# HINDSIGHT_API_MAINTENANCE_START_JITTER_SECONDS=60 # Upper bound on a random delay before a process runs its FIRST maintenance tick. Every job is due on that tick, so without an offset a fleet started together runs every sweep in every process at once. 0 disables the jitter.
|
|
|
|
# Vector Extension (Optional - uses pgvector by default)
|
|
# Options: "pgvector" (default), "vchord", "pgvectorscale" (DiskANN)
|
|
# HINDSIGHT_API_VECTOR_EXTENSION=pgvector
|
|
|
|
# Let a vector index scan resume until the query's LIMIT is satisfied, instead of
|
|
# stopping when its first candidate list drains (pgvector: hnsw.ef_search, 200) — with
|
|
# it off, a larger recall budget cannot retrieve more rows. Needs pgvector 0.8.0+;
|
|
# older servers reject it and it is dropped automatically. Set false and restart as a
|
|
# quick revert to the previous retrieval depth, with no code change.
|
|
# HINDSIGHT_API_ANN_ITERATIVE_SCAN=true
|
|
# Ceiling on tuples one resumed scan may visit. Bounds the CPU and memory a selective
|
|
# query can spend resuming (filters are applied after the scan, so it resumes often).
|
|
# Lower it to trade depth back for latency. pgvector's own default is 20000.
|
|
# HINDSIGHT_API_ANN_MAX_SCAN_TUPLES=4000
|
|
# For Azure PostgreSQL with DiskANN:
|
|
# HINDSIGHT_API_VECTOR_EXTENSION=pgvectorscale # Auto-detects pg_diskann on Azure
|
|
|
|
# Per-bank vector indexes (pgvector / pgvectorscale / vchord only; ScaNN and Oracle use one global index)
|
|
# HINDSIGHT_API_VECTOR_INDEX_MIN_ROWS=0 # Memories a bank needs in one fact type before that fact type gets its own vector index. 0 (default) turns the threshold OFF: every bank is indexed when it is created, and no background maintenance runs. Set ~10000 on deployments with thousands of banks: every index lives on the shared memory_units table and is planned against by every OTHER bank's queries, so unconditional per-bank indexes put a ceiling on bank count. Smaller banks then use exact search, which is faster AND exact.
|
|
# HINDSIGHT_API_VECTOR_INDEX_MAINTENANCE_MIN_INTERVAL_SECONDS=900 # Shortest gap between two index-maintenance runs for one bank, so a bank hovering at the threshold cannot build and drop the same index repeatedly. Unused while the threshold is off.
|
|
|
|
# Text Search Extension (Optional - uses native PostgreSQL full-text search by default)
|
|
# Unused by a bank with HINDSIGHT_API_ENABLE_TEXT_SEARCH=false (see Recall pipeline stages).
|
|
# Backend options: "native" (default), "vchord", "pg_textsearch", "pgroonga", "pg_search"
|
|
# HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native
|
|
# Native backend dictionary (only used by HINDSIGHT_API_TEXT_SEARCH_EXTENSION=native)
|
|
# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_NATIVE_LANGUAGE=english
|
|
# ParadeDB pg_search tokenizer (only used when creating pg_search BM25 indexes).
|
|
# Empty uses ParadeDB's default tokenizer: unicode_words.
|
|
# Supported values: unicode_words, simple, whitespace, literal, literal_normalized,
|
|
# chinese_compatible, icu, jieba, source_code,
|
|
# chinese_lindera/lindera(chinese), japanese_lindera/lindera(japanese),
|
|
# korean_lindera/lindera(korean), ngram(min,max), edge_ngram(min,max)
|
|
# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_TOKENIZER=
|
|
# ParadeDB pg_search function schema (default: paradedb).
|
|
# Certain managed PostgreSQL distributions install pg_search functions under pgsearch.
|
|
# HINDSIGHT_API_TEXT_SEARCH_EXTENSION_PG_SEARCH_FUNCTION_SCHEMA=paradedb
|
|
# Cap on the number of terms in the native PostgreSQL BM25 tsquery. Long queries
|
|
# OR-join every normalized token, and native ranking (no IDF, re-ranks every
|
|
# match) can then scan a large fraction of the bank and time out. Over the cap,
|
|
# the most selective terms are kept — lowest tenant-wide document frequency, read
|
|
# for free from pg_stats (no reindex). 0 restores the uncapped behavior; the cap
|
|
# bounds only the native backend (other BM25 backends get the raw query).
|
|
# HINDSIGHT_API_BM25_MAX_QUERY_TERMS=16
|
|
# When the cap above trims a query, keep the most selective terms (lowest
|
|
# document frequency, from pg_stats) instead of the first N. true is strictly
|
|
# better for recall at no extra cost when stats exist; set false to opt out of
|
|
# the catalog read and cap by position. Ignored when the cap is 0.
|
|
# HINDSIGHT_API_BM25_SELECTIVE_TERMS=true
|
|
|
|
# File Parser (Optional - uses markitdown by default)
|
|
# HINDSIGHT_API_FILE_PARSER=markitdown
|
|
# Enable image OCR for MarkItDown using an OpenAI-compatible OCR/vision endpoint.
|
|
# These OCR settings are independent from HINDSIGHT_API_LLM_* because MarkItDown
|
|
# uses the OpenAI SDK directly and requires Chat Completions image input support.
|
|
# When OCR is enabled, API_KEY, BASE_URL, and MODEL are required.
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_ENABLED=false
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_API_KEY=
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_BASE_URL=
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_MODEL=
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_PROMPT=
|
|
# Optional JSON dict of custom headers for the OCR OpenAI client (e.g. proxies / request tracing).
|
|
# HINDSIGHT_API_FILE_PARSER_MARKITDOWN_OCR_DEFAULT_HEADERS=
|
|
|
|
# Embeddings Configuration (Optional - uses local by default)
|
|
# Provider: "local" (default), "onnx", "tei", "openai", "cohere", "google", "openrouter", "zeroentropy", "litellm", or "litellm-sdk"
|
|
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=local
|
|
# For local provider:
|
|
# HINDSIGHT_API_EMBEDDINGS_LOCAL_MODEL=BAAI/bge-small-en-v1.5
|
|
# Force CPU if local embeddings hit MPS/XPC instability on macOS:
|
|
# HINDSIGHT_API_EMBEDDINGS_LOCAL_FORCE_CPU=false
|
|
# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
|
|
# variable-length workloads). CUDA/XPU still auto-select regardless:
|
|
# HINDSIGHT_API_EMBEDDINGS_LOCAL_ALLOW_MPS=false
|
|
# For ONNX provider (local CPU embeddings without an Ollama/TEI sidecar):
|
|
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=onnx
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_ID=intfloat/multilingual-e5-small
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_FILE=onnx/model.onnx
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_DIMENSIONS=384
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_MAX_TOKENS=512
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_POOLING=mean
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_NORMALIZE=true
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_QUERY_PREFIX="query: "
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_PASSAGE_PREFIX="passage: "
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_BATCH_SIZE=32 # Texts per forward pass; bounds peak memory
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_CPU_MEM_ARENA=false # ONNX CPU memory arena; true lets RSS ratchet up
|
|
# Optional for local model paths or pre-downloaded artifacts:
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_MODEL_PATH=/models/multilingual-e5-small/onnx/model.onnx
|
|
# HINDSIGHT_API_EMBEDDINGS_ONNX_TOKENIZER_NAME_OR_PATH=/models/multilingual-e5-small
|
|
# Optional for China network / restricted HF access:
|
|
# HF_ENDPOINT=https://hf-mirror.com
|
|
# Applies to any provider: cap each input at this many tokens before embedding, so
|
|
# oversized content is truncated instead of failing the embed call permanently.
|
|
# Defaults to 8192, the input limit of essentially every remote embedding model
|
|
# (OpenAI text-embedding-3-*, Bedrock Titan V2, Cohere v3, a stock llama.cpp
|
|
# context); raise or lower it to match your model, or set 0 to send text uncapped.
|
|
# (Deprecated alias: HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MAX_INPUT_TOKENS)
|
|
# HINDSIGHT_API_EMBEDDINGS_MAX_INPUT_TOKENS=8192
|
|
# Asymmetric models (E5, google/embeddinggemma-300m, ...) expect a different instruction
|
|
# in front of a search than in front of stored text. Providers that only accept plain text
|
|
# (tei, openai-compatible, litellm) need it applied client-side; local/zeroentropy handle it
|
|
# themselves and ignore these. Unset = text sent as-is.
|
|
# HINDSIGHT_API_EMBEDDINGS_QUERY_PREFIX="task: search result | query: "
|
|
# HINDSIGHT_API_EMBEDDINGS_PASSAGE_PREFIX="title: none | text: "
|
|
# Embedding requests a remote provider keeps in flight for one encode() call. Concurrency,
|
|
# not bigger requests, is what saturates an embedding service (applies to every remote
|
|
# provider; the in-process local/onnx backends are unaffected).
|
|
# HINDSIGHT_API_EMBEDDINGS_MAX_CONCURRENT_REQUESTS=8
|
|
# For TEI provider:
|
|
# HINDSIGHT_API_EMBEDDINGS_TEI_URL=http://localhost:8080
|
|
# Max texts per TEI /embed request, and the unit the client fans out over (see the
|
|
# concurrency setting above). TEI's --max-client-batch-size (32 by default) rejects
|
|
# anything larger outright rather than clamping it.
|
|
# HINDSIGHT_API_EMBEDDINGS_TEI_BATCH_SIZE=32
|
|
# For Gemini/Vertex AI embeddings:
|
|
# Max texts per embed_content request, and the unit the client fans out over (see
|
|
# the concurrency setting above).
|
|
# HINDSIGHT_API_EMBEDDINGS_GEMINI_BATCH_SIZE=100
|
|
# For OpenAI-compatible embeddings:
|
|
# HINDSIGHT_API_EMBEDDINGS_OPENAI_API_KEY=sk-xxxx
|
|
# HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL=text-embedding-3-small
|
|
# HINDSIGHT_API_EMBEDDINGS_OPENAI_BASE_URL=https://api.openai.com/v1
|
|
# For LiteLLM proxy embeddings:
|
|
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_API_BASE=http://localhost:4000
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_API_KEY=your-litellm-key
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_MODEL=text-embedding-3-small
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_DIMENSIONS=1536 # declare the vector width to skip the startup probe
|
|
# For LiteLLM SDK embeddings (no proxy; provider credentials read from the environment):
|
|
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=litellm-sdk
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MODEL=bedrock/amazon.titan-embed-text-v2:0
|
|
# Bedrock only: invoke this target instead of the model above (e.g. an application
|
|
# inference profile ARN, when a Service Control Policy denies the bare model id).
|
|
# LiteLLM picks the Bedrock payload shape from _MODEL, which must stay a recognizable
|
|
# id, so the opaque ARN goes here.
|
|
# HINDSIGHT_API_EMBEDDINGS_LITELLM_SDK_MODEL_ID=arn:aws:bedrock:eu-west-1:123456789012:application-inference-profile/abc123
|
|
# For ZeroEntropy zembed-1:
|
|
# HINDSIGHT_API_EMBEDDINGS_PROVIDER=zeroentropy
|
|
# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_API_KEY=ze-xxxx
|
|
# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_MODEL=zembed-1
|
|
# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_DIMENSIONS=1280
|
|
# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_ENCODING_FORMAT=float
|
|
# HINDSIGHT_API_EMBEDDINGS_ZEROENTROPY_LATENCY=fast
|
|
#
|
|
# IMPORTANT: Embedding keys require provider-specific names:
|
|
# HINDSIGHT_API_EMBEDDINGS_{PROVIDER}_{PARAMETER}
|
|
# (for example, HINDSIGHT_API_EMBEDDINGS_OPENAI_MODEL).
|
|
#
|
|
# DeepSeek note: DeepSeek is supported for LLM calls, but not for embeddings.
|
|
# If using DeepSeek as LLM provider, keep embeddings on local/openai/cohere/google/etc.
|
|
|
|
# Retry policy for remote embedding calls (litellm / litellm-sdk providers).
|
|
# Transient upstream failures (5xx, timeouts, connection errors) are retried with
|
|
# jittered exponential backoff; 4xx auth/validation errors are never retried.
|
|
# HINDSIGHT_API_EMBEDDINGS_MAX_RETRIES=4 # retries after the first attempt; 0 disables
|
|
# HINDSIGHT_API_EMBEDDINGS_INITIAL_BACKOFF=0.5 # seconds; doubles per attempt, with jitter
|
|
# HINDSIGHT_API_EMBEDDINGS_MAX_BACKOFF=4.0 # cap on per-retry backoff, seconds
|
|
# HINDSIGHT_API_EMBEDDINGS_RETRY_BUDGET=15.0 # wall-clock ceiling per encode() spent retrying
|
|
|
|
# Embedding similarity thresholds. These defaults preserve the behavior calibrated
|
|
# for BAAI/bge-small-en-v1.5. Recalibrate each threshold independently when changing
|
|
# embedding models because cosine-similarity distributions are model-dependent.
|
|
# HINDSIGHT_API_SEMANTIC_MIN_SIMILARITY=0.3
|
|
# HINDSIGHT_API_GRAPH_SEED_MIN_SIMILARITY=0.3
|
|
# HINDSIGHT_API_TEMPORAL_SEMANTIC_MIN_SIMILARITY=0.1
|
|
# HINDSIGHT_API_SEMANTIC_LINK_MIN_SIMILARITY=0.7
|
|
# HINDSIGHT_API_CONSOLIDATION_DEDUP_THRESHOLD=0.97
|
|
|
|
# Floor on how often an automatic mental-model refresh (after-consolidation or cron)
|
|
# may run, in seconds. A trigger that fires sooner is queued and parked until the
|
|
# window closes, and further triggers fold into it, so a burst of small retains costs
|
|
# one refresh instead of one per retain. 0 = no floor (the historical behaviour).
|
|
# Hierarchical, and overridable per model via the trigger's min_refresh_interval_seconds.
|
|
# Explicit refreshes always run immediately.
|
|
# HINDSIGHT_API_MENTAL_MODEL_MIN_REFRESH_INTERVAL_SECONDS=0
|
|
|
|
# Recall pipeline stages (all on by default). Each is hierarchical, so a single
|
|
# bank can switch a stage off via the config API without changing the server
|
|
# default. Turning all four off reduces recall to a single vector query, the
|
|
# lowest-latency recall path.
|
|
# Keyword (BM25) arm. false leaves pure vector search: the arm is left out of the
|
|
# query entirely rather than filtered to nothing, so its SQL, its query tokenization
|
|
# and its pg_stats term-selection lookup are all skipped. Also drops the keyword arm
|
|
# from knowledge-page search:
|
|
# HINDSIGHT_API_ENABLE_TEXT_SEARCH=true
|
|
# Temporal retrieval arm, plus the date-aware query analysis that feeds it:
|
|
# HINDSIGHT_API_ENABLE_TEMPORAL_RETRIEVAL=true
|
|
# Entity/link graph traversal arm:
|
|
# HINDSIGHT_API_ENABLE_GRAPH_RETRIEVAL=true
|
|
# Cross-encoder rerank of the fused candidates (false = use the RRF order):
|
|
# HINDSIGHT_API_ENABLE_RERANKING=true
|
|
|
|
# Reranker Configuration (Optional - uses local by default)
|
|
# Provider: "local" (default) or "tei" (HuggingFace Text Embeddings Inference)
|
|
# HINDSIGHT_API_RERANKER_PROVIDER=local
|
|
# Trusted gateway attribution (disabled by default). When enabled, remote
|
|
# reranker requests include X-Hindsight-Bank-Id with the current bank ID.
|
|
# HINDSIGHT_API_RERANKER_SEND_BANK_AS_HEADER=false
|
|
# Transient upstream failures (5xx, timeouts, connection errors, 429 quota) are
|
|
# retried with jittered exponential backoff; 4xx auth/validation errors are never
|
|
# retried. Applies to every remote provider except "tei", which retries on its own.
|
|
# HINDSIGHT_API_RERANKER_MAX_RETRIES=3 # retries after the first attempt; 0 disables
|
|
# HINDSIGHT_API_RERANKER_INITIAL_BACKOFF=0.5 # seconds; doubles per attempt, with jitter
|
|
# HINDSIGHT_API_RERANKER_MAX_BACKOFF=4.0 # cap on per-retry backoff, seconds
|
|
# HINDSIGHT_API_RERANKER_RETRY_BUDGET=10.0 # wall-clock ceiling per rerank spent retrying
|
|
# For local provider:
|
|
# HINDSIGHT_API_RERANKER_LOCAL_MODEL=cross-encoder/ms-marco-MiniLM-L-6-v2
|
|
# Force CPU if the local reranker hits MPS/XPC instability on macOS:
|
|
# HINDSIGHT_API_RERANKER_LOCAL_FORCE_CPU=false
|
|
# Opt in to the Apple Silicon MPS GPU (off by default: MPS leaks memory under
|
|
# variable-length workloads). CUDA/XPU still auto-select regardless:
|
|
# HINDSIGHT_API_RERANKER_LOCAL_ALLOW_MPS=false
|
|
# For TEI provider:
|
|
# HINDSIGHT_API_RERANKER_TEI_URL=http://localhost:8081
|
|
# For flashrank provider: passages scored per ONNX forward pass. Each pass
|
|
# allocates attention tensors sized batch * heads * seq^2, so raising this
|
|
# raises peak memory quadratically in passage length:
|
|
# HINDSIGHT_API_RERANKER_FLASHRANK_BATCH_SIZE=32
|
|
# Max candidates the cross-encoder reranks per recall (RRF pre-filters the rest):
|
|
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES=300
|
|
# Optionally scale that cap by the recall budget level (the cross-encoder dominates
|
|
# a large recall's latency). 0 = fall back to the flat cap above; fully backwards-compatible.
|
|
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_LOW=0
|
|
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_MID=0
|
|
# HINDSIGHT_API_RERANKER_MAX_CANDIDATES_HIGH=0
|
|
|
|
# Reranker failover chain: extra rerankers tried, in order, when the one above
|
|
# fails. Members are numbered from 1 (indices must be contiguous) and every
|
|
# setting of member n carries the same index. A member inherits nothing from the
|
|
# primary, so spell out everything it needs. Unset = no fallback (default): a
|
|
# failing reranker fails the recall. End the chain with "rrf" to fail open and
|
|
# keep the retrieval order instead.
|
|
# HINDSIGHT_API_RERANKER_1_PROVIDER=cohere
|
|
# HINDSIGHT_API_RERANKER_1_COHERE_API_KEY=your-cohere-api-key
|
|
# HINDSIGHT_API_RERANKER_2_PROVIDER=rrf
|
|
|
|
# Observability & Tracing (Optional - disabled by default)
|
|
# Enable OpenTelemetry tracing for LLM calls (GenAI semantic conventions)
|
|
# HINDSIGHT_API_OTEL_TRACES_ENABLED=true
|
|
#
|
|
# Local development with Grafana LGTM stack (recommended - see scripts/dev/grafana/README.md)
|
|
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
|
#
|
|
# Cloud backends (Grafana Cloud, Langfuse, DataDog, etc.)
|
|
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_ENDPOINT=https://your-backend-url
|
|
# HINDSIGHT_API_OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"
|
|
#
|
|
# Custom service name and environment (optional, defaults: hindsight-api, development)
|
|
# HINDSIGHT_API_OTEL_SERVICE_NAME=hindsight-production
|
|
# HINDSIGHT_API_OTEL_DEPLOYMENT_ENVIRONMENT=production
|
|
#
|
|
# Expose async-operation queue + consolidation-backlog gauges on /metrics.
|
|
# Runs periodic per-schema COUNT queries on a background task (disabled by default).
|
|
# HINDSIGHT_API_METRICS_BACKLOG_ENABLED=true
|
|
#
|
|
# Runtime-stall observability (enabled by default). When a liveness probe fails,
|
|
# these tell you WHY: a blocked event loop vs DB connection-pool exhaustion.
|
|
# The loop watchdog logs the offending stack when the loop is unresponsive; the
|
|
# DB-pool acquire timing logs (and exposes hindsight.db.pool.waiting) when
|
|
# callers queue for a connection. Both are cheap; tune or disable if needed.
|
|
# HINDSIGHT_API_LOOP_WATCHDOG_ENABLED=false
|
|
# HINDSIGHT_API_LOOP_WATCHDOG_STALL_THRESHOLD_MS=1000
|
|
# HINDSIGHT_API_LOOP_WATCHDOG_POLL_INTERVAL_MS=250
|
|
# HINDSIGHT_API_DB_ACQUIRE_WARN_THRESHOLD_MS=1000
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Operations (Optional)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Where `--daemon` redirects the server's stdout/stderr. Set this per profile when
|
|
# several daemons share a machine, so their output does not interleave.
|
|
# HINDSIGHT_API_DAEMON_LOG=~/.hindsight/daemon.log
|
|
|
|
# Periodic CPU profile, as JSON. Absent = profiling off.
|
|
# HINDSIGHT_API_PROFILE={"every": 60, "top": 20, "mode": "cprofile"}
|
|
|
|
# How long a task the store shed for backpressure is held before being retried.
|
|
# Deferrals do not count against HINDSIGHT_API_WORKER_MAX_RETRIES.
|
|
# HINDSIGHT_API_BACKPRESSURE_DEFER_SECONDS=120
|
|
|
|
# Legacy MCP bearer token, checked before the TenantExtension's own auth.
|
|
# HINDSIGHT_API_MCP_AUTH_TOKEN=your-mcp-token
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# SuperGrok OAuth provider (Optional; HINDSIGHT_API_LLM_PROVIDER=xai-oauth)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# All optional — the defaults match the vendor's own client. Log in once with
|
|
# `python -m hindsight_api.engine.providers.xai_oauth_auth login`.
|
|
# HINDSIGHT_API_XAI_OAUTH_TOKEN_PATH=~/.hindsight/xai_oauth.json
|
|
# HINDSIGHT_API_XAI_OAUTH_BASE_URL=https://api.x.ai/v1
|
|
# HINDSIGHT_API_XAI_OAUTH_CLIENT_ID=your-oauth-client-id
|
|
# HINDSIGHT_API_XAI_OAUTH_SCOPE=openid profile email offline_access
|
|
# HINDSIGHT_API_XAI_OAUTH_REFRESH_SKEW_SECONDS=60
|
|
# HINDSIGHT_API_XAI_OAUTH_REFRESH_TIMEOUT_SECONDS=20
|
|
# Debug-only: logs an allowlist of response headers on a non-2xx reply.
|
|
# HINDSIGHT_API_XAI_OAUTH_DEBUG_HEADERS=false
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Extensions (Optional)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Your own FileStorage implementation, used instead of the built-in backends.
|
|
# Every other HINDSIGHT_API_FILE_STORAGE_* variable is passed to it as config.
|
|
# HINDSIGHT_API_FILE_STORAGE_EXTENSION=my_package.storage:MyStorage
|
|
|
|
# Request headers copied into RequestContext.extra_headers so a custom
|
|
# TenantExtension / OperationValidatorExtension can read them. Comma-separated,
|
|
# matched case-insensitively. Unset by default: extensions see only the
|
|
# Authorization header. Use this when the bearer token identifies a proxy rather
|
|
# than the caller, and per-caller identity arrives in a separate header. A listed
|
|
# header that arrives more than once is dropped, so only list headers the proxy
|
|
# in front of Hindsight sets itself (stripping any client-supplied copy).
|
|
# HINDSIGHT_API_EXTENSION_PASSTHROUGH_HEADERS=x-user-assertion
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Webhooks (Optional)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Outbound webhook delivery targets caller-supplied URLs. To prevent SSRF, the
|
|
# delivery worker blocks private, loopback, and link-local destinations
|
|
# (including the cloud metadata address 169.254.169.254) by default. List hosts
|
|
# or IP/CIDR ranges here (comma-separated) to re-permit specific internal
|
|
# destinations — e.g. 127.0.0.1 for local testing, or an internal receiver.
|
|
# HINDSIGHT_API_WEBHOOK_ALLOWED_HOSTS=127.0.0.1,internal-receiver.svc,10.0.0.0/8
|
|
|
|
# Whether the webhook delivery-history API returns the raw upstream response
|
|
# body. Off by default: returning arbitrary response bodies to callers is an
|
|
# information-exfiltration primitive. The delivery status code is always
|
|
# returned regardless. Enable only if you trust your webhook destinations.
|
|
# HINDSIGHT_API_WEBHOOK_EXPOSE_RESPONSE_BODY=false
|
|
|
|
# -----------------------------------------------------------------------------
|
|
# Control Plane (Optional)
|
|
# -----------------------------------------------------------------------------
|
|
|
|
# Dataplane API URL - where the CP proxies requests to
|
|
# HINDSIGHT_CP_DATAPLANE_API_URL=http://localhost:8888
|
|
|
|
# Optional: Bearer token the CP sends as `Authorization: Bearer <key>` to the
|
|
# dataplane API. Required when the API service is auth-protected; omit for a
|
|
# public/unauthenticated API.
|
|
# HINDSIGHT_CP_DATAPLANE_API_KEY=your-dataplane-bearer-token
|
|
|
|
# Optional: Require a shared access key to view the Control Plane UI.
|
|
# When set, visitors see a login page and must enter the key before
|
|
# accessing the dashboard or any /api/* routes (except /api/health).
|
|
# HINDSIGHT_CP_ACCESS_KEY=your-shared-secret-key
|