Skip to main content

Anamnesis: evidence, not claims

Anamnesis is persistent memory for AI. These pages report what it does, measured with a pre-registered, reproducible benchmark suite: three arms (source present, no memory, memory injected), multi-metric, blind-graded, with honest negatives reported alongside the wins. We publish method and results, not implementation internals.

AnamnesisNaive RAGMem0 · comingZep · coming
Anamnesis vs no memory
94.2%
near-oracle quality (98.3%) at 431 vs 7,820 input tokens
Source present0.98 · 7,820 tok
No memory0.07 · 143 tok
Anamnesis0.94 · 431 tok
Token discount18×

The benchmarks​

  • T1. Fidelity: with the source removed, a memoryless model scores near zero; injected memory restores near-oracle answer quality at a fraction of the tokens.
  • T5. Cost: without memory an assistant pays a search tax (more turns, more tokens, slower) to rediscover a fact it could have recalled.
  • T2. Scale and recovery: where naive vector search breaks under adversarial noise, and how provenance-aware ranking recovers it.
  • T3. Belief revision: when a decision changes, the system serves the current answer and retires the stale one.
  • T4. Cross-model handoff: knowledge captured by one model is recalled and honored by a different model in a fresh session.

External benchmark: LongMemEval​

The suite above is ours. LongMemEval (Wu et al., ICLR 2025) is not, which is why we run it: an external benchmark with a locked holdout is the antidote to grading your own homework. Protocol: a 30-question stratified subsample of LongMemEval-S, seed-locked and pre-registered before any run, zero overlap with every question used during development, one end-to-end shot with no iteration against the result.

Without memory the model scores 0/30. With Anamnesis it scores 23/30 (77%). Handed the original source conversations directly, the same model scores 22/30. The pair worth staring at is the last two. Once injected memory matches what the model scores with the source in front of it, the remaining misses trace to the model's own reasoning, and there is nothing left for a memory system to recover. Two judges from different model families (Gemini and GPT) graded blind, agreed on 58 of 60 items, and scored the memory arm identically.

The limits of this number, stated plainly: a 30-question sample carries roughly ±7 points of sampling noise; the paper's reference judge is GPT-4o, which we substitute per the method above; and results on our subsample are not directly comparable to full-set numbers published elsewhere.

Why not just RAG?​

The most common objection from engineers, and a fair one: memory retrieval sounds like embeddings plus top-k over a folder of notes, an afternoon of work. At demo scale it is. If your corpus is small, static, and used from a single tool, a weekend RAG setup may be all you need.

The suite documents where that stops. Retrieval is the smallest part of the problem. Someone has to write the notes, so the pipeline captures sessions automatically, and it screens what enters, because a store that ingests everything eventually ingests garbage and adversarial text. T2 measures what unscreened contradictions do to answer fidelity at scale (0.94 falling to 0.42 at a thousand records) and shows that the recovery comes from provenance-aware ranking, which plain vector search lacks. Injecting raw chunks burns the context budget that distilled memories preserve (T1). When a decision changes, a RAG store serves the stale version and the current one side by side; T3 shows what it takes to retire a superseded belief instead of injecting the conflict. And in practice the do-it-yourself baseline is an agent searching its own files, which is the search tax T5 measures: around five times the turns and latency to reach the same answer.

Reach is the part no local setup covers. ChatGPT, claude.ai, and Claude Desktop will never run your scripts, but they will connect to a standard MCP connector, so the same memory follows you onto surfaces a local stack never touches.

Architecture​

Validity​

  • Judge audit & threats to validity: five blinded judges across three model families plus a human re-graded the results — every κ ≥ 0.87, and every discrepancy runs conservative. Plus the validity threats that remain open, stated plainly.

Downloads​