Nicolò Boschi
b5034cb690
* test(system): stories 02-09 for the recall surface Eight stories over the retrieval pipeline, all driven through the published client against a real server. Part of #4214. - 02 every retrieval arm runs, and fusion records what each one found. Asserted through `trace=True`: a blended final score cannot distinguish "both arms agreed" from "one arm did all the work". Also pins that `graph` returns nothing for a single-document bank, and that temporal is a scoring component rather than a fourth arm despite the docs' "four strategies". - 03 `max_tokens` spends the budget in rank order. The middle case looks wrong and is the point: at a tight budget the *second*-ranked fact comes back, because the top one does not fit and must skip only itself (#3688). - 04 `budget` and `max_tokens` are independent dials — depth asserted via the numeric budget in the trace, since any corpus small enough to read finds everything at every level. - 05 the `assistant` -> `experience` rename between the extraction schema and the read model, which nothing in either schema hints at. - 06 tag scoping, and that fuzzy matching is opt-in via a `tag_groups` leaf rather than the plain `tags` parameter. Includes the short-word cliff: the same single transposition clears the 0.45 trigram floor in `typescript` (0.467) and misses it in `music` (0.333). - 07 the two clocks. `fact_kind="event"` is load-bearing and silent — a date on a `conversation` fact is discarded with no error — and recency decays from the event date, which `query_timestamp` re-anchors. - 08 an absurd date must not deny the whole bank: a future-dated fact makes `days_ago` negative, and the unclamped exponential fails every recall rather than mis-ranking one row. - 09 `min_scores` floors two different things. `reranker`/`final` filter results; `semantic`/`keyword` gate their own retrieval arm, so flooring one alone removes nothing because the other arm supplies the fact back. The no-op is pinned deliberately — the parameter reads like a quality filter and is not. `payloads.Fact` gains `fact_kind` for story 07. Each story declares the consolidate step explicitly rather than sharing a helper: the suite's rule is that no LLM call is answered by default, and hiding one behind a fixture would be the first exception. * test(system): stories 10-15, documents and lifecycle - 10 a retain is a document: id, verbatim text, chunk composite id, provenance - 11 replace, including the pure-deletion case where the revision only *stops* saying something and nothing new arrives to overwrite it - 12 append accumulates instead of re-running the removal diff; re-sending a turn does not duplicate its fact - 13 delete takes only its own facts, and — the #3429 shape — two banks sharing a document_id do not take each other down - 14 reprocess is not a silent no-op: the extraction answer changes between the retain and the reprocess, so the new facts can only appear if the stored text was genuinely re-read - 15 curation: edit reaches the read path and is stamped edited_at; invalidation stops answering recalls while keeping reason and timestamp for an audit Found while writing these, filed rather than worked around: #4218 (list rows are untyped dicts while single-fetch siblings return models) — the reason this file mixes item["id"] with attribute access. * test(system): stories 20-27, observations and consolidation - 20 an observation is written *over* the facts, not instead of them, and its cited evidence resolves to facts that are still present - 22 new evidence for an existing claim merges into it (proof_count grows, one observation) rather than growing a near-duplicate sibling - 23 contradictory evidence supersedes: create + delete, so a recall never hands an agent both sides with equal confidence — while the historical *fact* survives the synthesis that summarised it - 25 clear_observations empties the derived layer and leaves every source fact, and deleting the last document behind an observation retires it too - 26 trigger_consolidation reports its operation and whether it deduplicated, so a double-fired trigger cannot quietly run twice - 27 prefer_observations drops the facts an observation superseded, and include_source_facts returns them as keyed provenance instead Restores `answers_with` to the rulebook, trimmed in #4212 for having no consumer: consolidation replies must cite source_fact_ids the server minted during the retain that triggered them, which a literal payload cannot know. * test(system): story 30, mental models, plus the machinery to drive reflect A mental-model refresh runs the full reflect loop in the worker, so the suite had to learn to drive an agentic conversation: - `tool=` matcher on rules. The reflect loop sends the same system prompt every turn and varies only the tools it offers, so the tool list is the one thing that separates one rung from the next; prompt substrings cannot. - `returns_tool_call` restored (it has consumers now), plus `calls_the_offered_tool`, which climbs whichever rung it is on. A story about what reflect concludes should not have to enumerate the ladder, or break when a rung is added. - `reflect.reflect_loop` wraps both into one call. - Two anchors, not one: the search turns and the answering turn use different system prompts, and anchoring only the first leaves the final turn unmatched. Also fixed two things in the harness that this exposed: - The miss report showed only the user message. A refresh sends the bare source query there, so the report named no anchor and showed nothing useful; it now carries the whole conversation and the tools offered. - `wait_until_settled` waited out the worker's retry backoff on an operation that had already recorded an error, turning a fast loud-miss into a 90s timeout. It now reports the error immediately, and the test server runs with worker retries off — against a deterministic stub a retry cannot change the answer. * test(system): stories 35 and 40, knowledge pages and reflect - 35 a page is a mental model with a place in a tree: both ids resolve, the tree carries the name someone browses by, search finds it, and deleting it leaves the facts. Also pins the defaults that make a page different from a bare model — delta refresh after consolidation, over observations, excluding models — the kind of thing a refactor flattens silently. - 40 reflect answers from what it searched. The second test is the load-bearing one and is written inside out: only the answering turn is scripted, so if the server could ever reach an answer without searching first, no search turn would arrive and the test would pass. It asserts the unscripted search turn did arrive, then clears it — the one place in the suite where an unmatched call is the subject rather than a gap. Quality is deliberately not asserted here: the stub supplies the answer text, so these cover the mechanism. Judging what the model actually says needs a real model and stays with the hs_llm_core judge tests. * test(system): stories 43, 50 and 80 — directives, bank config, isolation - 43 a directive is only real if its text reaches the model. Storing, listing and returning it prove nothing; these assert it arrives in the reflect prompt, under the MANDATORY heading (delivered as a rule, not as context), that a bank without directives ships no such section, and that deleting one stops it being sent. Whether the model obeys stays with the judge tests. - 50 config updates are additive. The clobber failure is invisible — no error, the bank just reverts every other field to default — so the assertions are about the neighbours, not the field that changed. - 80 bank isolation across all four verbs, on two banks that deliberately share a document_id (#3429). Read, count, update and delete are separate statements with separate predicates, so getting three right proves nothing about the fourth; each is checked on its own. Restores LLMStub.calls plus prompts_for(step): prompt assembly is deterministic even where the model's reading of it is not, so 'was the directive sent' can be asserted directly instead of judged. * test(system): stories 70 and 90 — entities/graph and multilingual - 70 entities are what stitch documents together. Two documents sharing only 'Alice' merge into one entity, and a query for a word appearing in just one of them reaches the other purely through that link. This is the traversal story 02 could not exercise: remove it and the fact vanishes while everything else still passes. - 90 non-Latin content round-trips byte-for-byte, entities keep their own names, mixed-script text keeps its embedded Latin names, and a Latin query reaches the Chinese sentence containing one. Emoji cover the astral-plane case. Story 90 first failed on the *stub*, not the product: the lexical embedder tokenised `[a-z0-9]+` only, so Chinese text produced no tokens at all and both query and memory collapsed onto the same fallback vector. A test double that cannot represent CJK cannot test CJK, so the tokeniser now emits one token per CJK codepoint — roughly what a real analyser does at the unigram level. ASCII tokenisation is unchanged, so the pinned scores in stories 01 and 09 still hold. * test(system): story 60, async retain and idempotent operations An async retain returns a receipt and moves the whole 'did it land?' question onto the operation record, so the record has to answer it alone: a terminal status, and result metadata saying what was stored — 'completed' on its own cannot distinguish work done from work skipped. The idempotency tests are the load-bearing ones. A caller retries when a request times out and cannot tell a lost request from a slow one; without a caller-supplied operation_id the safe retry does not exist. Both directions are pinned: the same id replayed after completion stores one memory, not two, and two different ids over identical content stay two operations — deduplicating on content instead of the caller's id would be its own silent data loss. * test(system): code-review fixes in the harness - _slots returned a two-item tuple, which the project bans outright; it is a _Slots dataclass now. Introduced in #4212 and missed by that review. - returns_tool_call has no consumers again — calls_the_offered_tool covers every reflect turn — so it comes back out. Second time this method has been added and removed; the rule holding is that surface ships when something uses it. * test(system): story 21, consolidation failure and recovery Consolidation is the one background job that both reads and deletes, so a failure partway through is the most dangerous moment in the system. The correct behaviour turns out to be boring, which is the point: a model returning nonsense loses the synthesis and not one fact, the round completes rather than wedging the bank, and failed_consolidation surfaces the backlog. Recovery is explicit and worth writing down: trigger_consolidation does NOT pick failed facts back up — they stay claimed, so a scheduled round will not re-feed a poison input forever — and recover_consolidation is the deliberate door. Test-env changes this needed, each because a production default is noise for a single deterministic server: bank stats cached 60s (a test asserting on a counter reads a value from before its own action), and the maintenance start jitter that spreads sweeps over a minute to stop a fleet stampeding one database. Two things checked and NOT filed as bugs, both of which looked like one: failed_consolidation appearing stuck was the 60s stats cache, and a failed refresh sitting at status=pending was the worker's retry backoff. * test(system): story 52, whole-bank export and import Everything in a bank points at everything else by id, and every one of those references is minted by the source instance — so an import has to rewrite them all, in one pass, without missing a layer. Miss one and the import still succeeds: the counts are right and only the provenance is broken. That is the bug that has been fixed twice here, so the assertions chase references rather than counts, and check that an imported observation cites facts that exist in the destination. Also pinned: the derived layers are opt-in both ways, the page's backing model exists in the destination rather than naming one left behind, and the imported bank actually answers recalls (rows arriving is not the same as rebuilt indexes). The first draft failed on a test bug worth recording: the destination's own auto-consolidation runs over the imported facts like any other write, so an observation appearing there proved nothing — it might have been carried or invented locally moments later. Every import now silences the destination's consolidation first, which makes each observation necessarily an imported one. * test(system): stories 31, 32 and 64 — staleness, refresh safety, webhooks - 31 the watermark that makes 'always current' work. Both failure directions matter: a watermark that never advances refreshes forever at full cost, and a staleness check that never fires leaves a model quietly frozen while still answering confidently. Also pins that going stale does not blank the answer. - 32 the two wipe guards. A refresh whose scope matches nothing must keep the answer it has — writing 'nothing retrieved' through as 'no content' destroys work the bank cannot re-derive. And a dry run touches neither content, watermarks, nor history, or it is not a dry run. - 64 webhooks, asserted against a real receiver rather than the server's own delivery log, which only proves it tried. The signature is recomputed the way a receiver would — a signature over the wrong bytes is a header that looks right and verifies nowhere. Both SSRF cases from GHSA-ggrr-69wp-fj54 are pinned as security properties, refused at registration. The stub gains a webhook receiver: it is the only endpoint a hermetic test can offer. The test server allowlists exactly that host, so a webhook aimed at any other private address still fails and the guard stays genuinely under test. * test(system): stories 42, 44, 45 — disposition, structured output, tag groups - 42 disposition and mission reach the reasoning prompt. Whether a trait changes the *answer* is a question for the judge tests; that the knob is connected at all is deterministic, and its failure is silent — the API accepts the setting, returns it on read, and the agent behaves identically. Opposite dispositions are asserted to produce different prompts, which a constant string would fail. - 44 structured output is a second extraction call over the prose answer, not a constraint on it. Pins that the prose is unchanged, that nothing pays for the extra call without a schema, and — per #4230 — that a failed extraction is currently indistinguishable from an empty one. - 45 tag_groups and/or/not and nesting, on a corpus where each wrong operator returns a different non-empty set rather than nothing. Three harness bugs surfaced by giving a bank a reflect mission, all of which would have broken every reflect story the first time anyone set one: - The "reflect" anchor was the default role line, which a mission *replaces*. Re-anchored on the CRITICAL preamble, present on every reflect turn. - The final turn is a free choice among every search tool plus the finish tool, and calls_the_offered_tool always took the first — so it searched, was offered the same choice again, and never terminated. Finishing is now claimed by its own rule registered first, and carries the answer in its argument (calling it bare ends the loop with "the done tool returned no answer"). - The search rules and the prose turn share a system prompt; only the search turns carry tools. Rules can now require tools, so the ladder's rules stop swallowing the turn meant to write the answer. * test(system): stories 24, 54, 92 — observation scopes, templates/preview, chunking - 24 observation_scopes decides which observations a multi-tag memory feeds. The isolation is the point: a lesson tagged for a student and a teacher must not produce one observation belonging to neither, and consolidation's all_strict matching is what keeps one party's observations out of another's. Both documented spellings are pinned — [[]] is one global scope, [] is *zero* and falls back to combined, one character apart with no error either way. - 54 templates carry configuration and not memories, leave untouched fields unset rather than freezing today's defaults, and the prompt preview matches what a real retain actually sends. Also pins that retain_custom_instructions is only consulted in `custom` extraction mode — stored, returned on read, and silently unused otherwise. - 92 chunking: contiguous indexes, reassembly loses nothing at the seams, one extraction call per chunk (fewer means a piece was never read, more means paying twice), and the stored document stays byte-identical because it is what a reprocess re-reads. Fixed a real hole in the harness: the client was a path dependency installed non-editable, so the venv held a *copy* taken whenever it was last built. The suite had been testing a stale snapshot — preview_prompt exists in the repo's client and was simply absent from the installed one. Now editable. Two template tests are skipped rather than rewritten against raw HTTP: the import endpoint cannot be called from any SDK (#4232). Reaching around the client to make them pass would hide exactly the defect a client-driven suite exists to surface. * test(system): make the open defects fail instead of documenting them The suite was green while five filed issues sat unfixed, because it accommodated every one of them: comments saying "this returns a dict, see #4218" and then dict access; a skip on the import round trip; and — worst — a test asserting the *current* wrong answer for #4230, which would have failed the day someone fixed it and taught the next reader to delete it rather than read it. A comment is a code-review note, not a gate. These now assert the contract we want and fail until the product honours it: - #4217 the recall trace must report the caller's query_timestamp, not the moment the trace was built - #4218 list rows must be typed like their single-fetch siblings - #4221 every wrapper convenience method must have an async twin — written over the whole family, so the next one added without a twin also fails - #4230 a failed structured-output extraction must be distinguishable from an empty one - #4232 a bank template must round-trip through the SDK (the two skips removed) Deliberately not xfail: an expected-failure marker keeps the run green, so nothing forces the question, and it outlives the bug by months. They sit in one file because they are temporary — when an issue lands, its test moves into the story it belongs to and the file shrinks. When it is empty, delete it. * test(system): fold the open-defect tests back into their stories All five issues are fixed on main, so the temporary defect file has done its job and is deleted. Each contract moves to where a reader would look for it: - #4217 trace anchor -> story 07 (temporal) - #4218 typed list rows -> story 10, plus ~70 call sites across the suite that had been reading rows as dicts and now use attribute access - #4221 async wrapper parity -> a new story 51, since it is a contract about the published client rather than about the server - #4230 structured-output failure is reported -> story 44 - #4232 template round trip -> story 54 (the two skips were already removed) Each keeps a line naming the issue it came from, so the history stays readable without the file that tracked it. Note for anyone converting dict access after a typing change: only *list rows* became models. Trace payloads, score components and mental-model history entries are still plain dicts, and a blanket regex over `x["field"]` rewrites those too — it did here, and turned ten passing tests red before being walked back. * docs(review): require a system story for new user-facing capabilities The system suite only stays useful if it grows with the product, and nothing was asking for that. Adds step 6b to the code-review skill and a pointer in CLAUDE.md's Testing section so the rule is visible before code is written, not only at review. The trigger is deliberately about *composition*, not size: a change needs a story when it adds a capability someone can name, or when it makes two existing capabilities meet for the first time. That second case is the one the ~500-file api-slim suite structurally cannot cover, and where every bug this suite was built for actually lived. Step 6b also carries the review checks the suite's own conventions depend on, each learned by getting it wrong here: go through the published client (reaching around it hid that import_bank_template was uncallable from every SDK, #4232), declare every LLM call, await background work rather than disabling it, assert the whole deterministic payload, and make an unmet contract *fail* rather than be documented — neither xfail nor a test pinning today's wrong answer. Also drops `extra_env` from start_hindsight_server: a parameter no caller ever passed, found by this review.
What is Hindsight?
Hindsight™ is an agent memory system built to create smarter agents that learn over time. Most agent memory systems focus on recalling conversation history. Hindsight is focused on making agents that learn, not just remember.
It eliminates the shortcomings of alternative techniques such as RAG and knowledge graph and delivers state-of-the-art performance on long term memory tasks.
Contents
- Memory Performance & Accuracy
- Quick Start — server · clients · platforms · embedded
- Adding Hindsight to Your Agent — LLM Wrapper · integrations · coding agents · MCP
- Core Concepts — memory types · retain / recall / reflect · observations · mental models & knowledge pages · banks
- Use Cases
- Running in Production
- Resources
Memory Performance & Accuracy
Hindsight is the most accurate agent memory system ever tested according to benchmark performance. It has achieved state-of-the-art performance on the LongMemEval benchmark, widely used to assess memory system performance across a variety of conversational AI scenarios. The current reported performance of Hindsight and other agent memory solutions as of January 2026 is shown here:
Live, continuously updated results — including per-model accuracy, latency and cost — are published at benchmarks.hindsight.vectorize.io.
The benchmark performance data for Hindsight has been independently reproduced by research collaborators at the Virginia Tech Sanghani Center for Artificial Intelligence and Data Analytics and The Washington Post. Other scores are self-reported by software vendors.
Hindsight is being used in production at Fortune 500 enterprises and by a growing number of AI startups.
🤖 Using a coding agent? Install the Hindsight documentation skill for instant access to docs while you code:
npx skills add https://github.com/vectorize-io/hindsight --skill hindsight-docsWorks with Claude Code, Cursor, and other AI coding assistants.
Quick Start
1. Start a server
Docker (recommended)
export OPENAI_API_KEY=sk-xxx
docker run -it --pull always --name hindsight --restart unless-stopped -p 8888:8888 -p 9999:9999 \
-e HINDSIGHT_API_LLM_API_KEY=$OPENAI_API_KEY \
-v hindsight-data:/home/hindsight/.pg0 \
ghcr.io/vectorize-io/hindsight:latest
Hindsight works with 25+ LLM providers via HINDSIGHT_API_LLM_PROVIDER — hosted (openai, anthropic, gemini, groq, bedrock, vertexai, minimax, deepseek, atlas, meta, …), fully local (ollama, lmstudio, llamacpp), any OpenAI-compatible endpoint, and gateways (litellm, litellmrouter) that reach the rest. Existing subscriptions work too: openai-codex (ChatGPT Plus/Pro), claude-code (Claude Pro/Max) and github-copilot (GitHub Copilot) need no API key. See supported models.
Docker (external PostgreSQL)
export OPENAI_API_KEY=sk-xxx
export HINDSIGHT_DB_PASSWORD=choose-a-password
cd docker/docker-compose
docker compose up
Oracle AI Database is also supported for enterprise deployments with full feature parity. See the storage documentation for details.
Bare metal (pip)
pip install hindsight-api
export HINDSIGHT_API_LLM_API_KEY=sk-xxx
hindsight-api
Kubernetes (Helm)
helm install hindsight oci://ghcr.io/vectorize-io/charts/hindsight \
--set api.llm.provider=openai \
--set api.llm.apiKey=sk-xxx \
--set postgresql.enabled=true
Managed (no server)
Hindsight Cloud is the hosted option: managed infrastructure that scales automatically, plus a dashboard, backups, team collaboration and a 99.9% uptime SLA. Billing is usage-based with free credits to start — no fixed monthly or per-seat fee. Point any client at https://api.hindsight.vectorize.io with your API key and skip the deployment entirely.
Compare self-hosted, Cloud and Enterprise → · Sign up →
All options, including Windows and air-gapped setups, are covered in the installation guide.
2. Connect a client
pip install hindsight-client -U # Python
npm install @vectorize-io/hindsight-client # Node.js / TypeScript
go get github.com/vectorize-io/hindsight/hindsight-clients/go # Go
curl -fsSL https://hindsight.vectorize.io/get-cli | bash # CLI
Python
from hindsight_client import Hindsight
client = Hindsight(base_url="http://localhost:8888")
# Retain: Store information
client.retain(bank_id="my-bank", content="Alice works at Google as a software engineer")
# Recall: Search memories
client.recall(bank_id="my-bank", query="What does Alice do?")
# Reflect: Generate disposition-aware response
client.reflect(bank_id="my-bank", query="Tell me about Alice")
Node.js / TypeScript
const { HindsightClient } = require('@vectorize-io/hindsight-client');
const main = async () => {
const client = new HindsightClient({ baseUrl: 'http://localhost:8888' });
await client.retain('my-bank', 'Alice loves hiking in Yosemite');
const results = await client.recall('my-bank', 'What does Alice like?');
console.log(results);
}
main();
Full reference: Python · Node.js · Go · CLI · REST API
Supported Platforms
| Platform | Docker | Bare Metal (pip) | Embedded DB (pg0) |
|---|---|---|---|
| Linux (x86_64, ARM64) | ✅ | ✅ | ✅ |
| macOS (Apple Silicon / arm64) | ✅ | ✅ | ✅ |
| macOS (Intel / x86_64) | ✅ | ⚠️ | ✅ |
| Windows (x86_64) | ✅ | ✅ | ✅ |
⚠️ Intel Macs: use hindsight-all-slim — see the installation guide for details.
Python Embedded (no server required)
pip install hindsight-all -U
On Intel (x86_64) Macs, install hindsight-all-slim instead — see Supported Platforms.
import os
from hindsight import HindsightServer, HindsightClient
with HindsightServer(
llm_provider="openai",
llm_model="gpt-5-mini",
llm_api_key=os.environ["OPENAI_API_KEY"]
) as server:
client = HindsightClient(base_url=server.url)
client.retain(bank_id="my-bank", content="Alice works at Google")
results = client.recall(bank_id="my-bank", query="Where does Alice work?")
A Node.js equivalent and a daemon CLI are also available.
Adding Hindsight to Your Agent
LLM Wrapper (2 lines of code)
The easiest way to add memory to an existing agent is the LLM Wrapper. Swap your LLM client for a wrapped one — memories are then stored and retrieved automatically on every call, with no other changes to your code.
pip install hindsight-litellm
from openai import OpenAI
from hindsight_litellm import wrap_openai
# Wrap your existing LLM client and you're done.
# Defaults to Hindsight Cloud; pass hindsight_api_url for a self-hosted server.
client = wrap_openai(
OpenAI(),
bank_id="user-123",
hindsight_api_url="http://localhost:8888",
)
# Hindsight recalls relevant memories before the call
# and retains the conversation after it.
response = client.chat.completions.create(
model="gpt-5-mini",
messages=[{"role": "user", "content": "What do you know about me?"}],
)
wrap_anthropic() does the same for the Anthropic SDK, and every setting — bank, recall budget, fact types, reflect instead of recall — can be overridden per call with hindsight_* kwargs. LiteLLM sits underneath, so the same integration covers 100+ models. See the LiteLLM integration.
If you need explicit control over when memories are stored and recalled, use the SDKs or REST API directly instead.
Integrations
60+ integrations — most need no code changes.
| Coding agents | Claude Code · Codex · Cursor · GitHub Copilot · opencode · Cline · Aider · Zed · Continue · Roo Code · OpenHands |
| Agent frameworks | LangGraph / LangChain · LlamaIndex · CrewAI · Pydantic AI · OpenAI Agents SDK · Google ADK · Agno · Strands · AutoGen · Microsoft Agent Framework · Vercel AI SDK · Haystack |
| No-code / low-code | n8n · Zapier · Dify · Flowise |
| Apps & tools | ChatGPT · Perplexity · Obsidian · Pipecat · Vapi |
Coding Agents
One package gives CLI coding agents long-term project memory: a per-repo bank built automatically from git history and past sessions, injected into the agent as it starts working, plus curated knowledge pages covering architecture, conventions and in-flight work.
npx @vectorize-io/hindsight-coding-agents install all # every detected agent, wired natively
npx @vectorize-io/hindsight-coding-agents install claude-code # or just one
Supports Claude Code, Codex CLI, Cursor CLI, GitHub Copilot CLI, opencode, Kilo CLI, Cline CLI, Antigravity CLI, Devin CLI, pi, Prime Agent, Grok Build and DeepSeek Harness. Ingestion is automatic — there is no setup command. See the coding agents integration.
MCP Server
Every server ships a built-in Model Context Protocol endpoint, one per bank, enabled by default:
http://localhost:8888/mcp/{bank_id}/
Point any MCP client at it to expose retain, recall and reflect as tools. See the MCP server docs.
Core Concepts
Memory Types
Most agent memory implementations rely on basic vector search or sometimes use a knowledge graph. Hindsight uses biomimetic data structures to organize agent memories in a way that is more like how human memory works:
- World facts: facts about the world ("The stove gets hot")
- Experiences: the agent's own experiences ("I touched the stove and it really hurt")
- Observations: consolidated, evidence-backed beliefs formed from many memories
- Mental models: learned understanding of the agent's world, synthesized from observations and facts
Memories live in banks. When memories are added, they are pushed into either the world facts or the experiences pathway, then represented as a combination of entities, relationships, and time series with sparse/dense vector representations to aid in later recall.
The Three Operations
Retain
The retain operation is used to push new memories into Hindsight. It tells Hindsight to retain the information you pass in as an input.
client.retain(
bank_id="my-bank",
content="Alice got promoted to senior engineer",
context="career update",
timestamp="2025-06-15T10:00:00Z",
)
Behind the scenes, retain uses an LLM to extract key facts, temporal data, entities, and relationships. It passes these through a normalization process to transform extracted data into canonical entities, time series, and search indexes along with metadata. These representations create the pathways for accurate memory retrieval in the recall and reflect operations.
Recall
The recall operation is used to retrieve memories. These memories can come from any of the memory types (world, experiences, etc.)
client.recall(bank_id="my-bank", query="What does Alice do?")
client.recall(bank_id="my-bank", query="What happened in June?") # temporal
Recall performs 4 retrieval strategies in parallel:
- Semantic: Vector similarity
- Keyword: BM25 exact matching
- Graph: Entity/temporal/causal links
- Temporal: Time range filtering
The individual results are merged, ordered by relevance using reciprocal rank fusion and a cross-encoder reranking model, then trimmed as needed to fit within the token limit.
Reflect
The reflect operation performs a more thorough analysis of existing memories. This allows the agent to form new connections between memories and build a more thorough understanding of its world — or to answer a question that needs deep thinking rather than lookup.
client.reflect(bank_id="my-bank", query="What should I know about Alice?")
For example, reflect supports use cases such as:
- An AI Project Manager reflecting on what risks need to be mitigated on a project.
- A Sales Agent reflecting on why certain outreach messages have gotten responses while others haven't.
- A Support Agent reflecting on opportunities where customers have questions not answered by current product documentation.
Observations
Retained facts don't stay a flat pile. In the background, Hindsight consolidates related facts into observations — deduplicated beliefs the bank has built up over time. Each observation keeps its supporting evidence with exact quotes and a proof count, and is refined rather than overwritten when new evidence arrives, so new information strengthens, weakens or extends an existing belief instead of silently replacing it.
Mental Models & Knowledge Pages
A mental model is a standing answer to a question about a bank ("What are this user's preferences?"). You define the question once; Hindsight writes the answer, stores it, and rewrites it in the background as the bank learns more. Reading one is a database read — no retrieval, no LLM call — so an agent can boot with a page of settled knowledge instead of rediscovering it every session.
Knowledge pages are mental models with the mechanics hidden: living documents a bank writes about itself, organized in folders like a wiki, searchable, and projectable onto disk as ordinary markdown. Supply a name and a question; every other decision is a default you can override.
Mental models → · Knowledge pages →
Memory Banks
A bank is an isolated memory store — one "brain" for one user, agent, or project. Isolation is strict: no cross-bank leakage. Banks carry background context and disposition traits (skepticism, literalism, empathy) that shape how reflect reasons over their memories, and can be created from declarative bank templates.
Two more things worth knowing:
- Multilingual by default. Input language is detected and preserved end to end — facts stay in their original language and entities keep their native script (张伟 stays 张伟, not "Zhang Wei"). Docs →
- Memory Defense. An opt-in, per-bank policy that scans every retain for secrets and PII against 45 patterns and either redacts the match (
[REDACTED:github_token]) or blocks the item before it reaches storage. Docs →
Use Cases
Hindsight is built to support conversational AI agents as well as agents that are intended to perform tasks autonomously. The ideal use case for Hindsight are agents that require a blend of these features such as AI employees that need to handle open-ended tasks, change behavior based on user feedback, and learn to perform complex tasks to automate work at a level that approximates a human work. Hindsight can be used with simple AI workflows like those built with n8n and other similar tools, but may be overkill for such applications.
Per-User Memories and Chat History
One of the simpler use cases you can use Hindsight for is to personalize AI chatbots and other conversational agents by storing and recalling memories associated with individual users.
The requirements for this use case usually look something like this:
Satisfying these requirements in Hindsight is straightforward. When new user inputs and tool calls are ingested into Hindsight using the retain operation, custom metadata can be used to enrich the new memories. Metadata provides a convenient way to isolate memories that need to be restricted to a given user. Once these are fed into the retain operation, any raw memories and mental models that get created can be filtered when retrieving relevant memories.
More patterns in the Cookbook and Best Practices.
Running in Production
| Storage | PostgreSQL + pgvector, or Oracle AI Database 23ai with full feature parity — storage |
| Configuration | Hierarchical: global env vars → per-tenant → per-bank — configuration |
| Monitoring | Prometheus metrics and dashboards for LLM calls, tokens and latency — monitoring |
| Operations | Admin CLI for migrations, bank repair and stuck operations — admin CLI |
| Events | Webhooks for retain, consolidation and refresh lifecycle events — webhooks |
| Extensibility | Tenant, auth and storage extension points — extensions |
| Managed | Skip all of it with Hindsight Cloud — managed, usage-based, 99.9% uptime SLA |
Resources
Documentation:
- Docs · FAQ · Best Practices · Cookbook · Blog
- Paper · Benchmarks · RAG vs Memory
Clients:
Community:
Star History
Contributing
See CONTRIBUTING.md.
License
MIT — see LICENSE
Built by Vectorize.io