augmentationBenchmarks
Measured recall on our evaluation corpora. Updated as arms certify.
Recall
Gold-hit recall: for each evaluation question, does the correct memory node appear in the top-ranked results? Cold start means a fresh substrate with no per-question tuning — the number a new customer gets on day one.
| condition | score | notes |
|---|---|---|
| cold start | 95.8% | 3,834 / 4,000 gold hits |
| warm + alias resolution | 100% | 4,000 / 4,000, idempotent |
| LongMemEval-S @6 (scoped) | 82.25% | certified with falsification guards |
| LongMemEval-S @20 (probe) | 90.8% | field probe, uncertified |
How this is measured
Every score comes from a preregistered evaluation arm: the scoring harness is written and committed before results are seen, with guards for shuffle sensitivity (exact binomial bounds), negative controls, leakage and gold resolution. A/A reruns must be bit-identical before a number is certified.
Why the query path is fast
Retrieval is BM25 + dense fused at query time, scored by the native engine — no LLM call sits in the retrieval path. The 384-dim encoder runs on our substrate host, so your text never leaves for a third-party API.
Field context
Self-reported numbers from other memory vendors, for calibration: agentmemory 95.2 R@5, supermemory 97 R@20 (with LLM aggregation), Mem0 94.4, Zep 71.2 e2e. Benchmarks in this field are largely self-graded; ours carry receipts.
Run it yourself
Export your store any time (/wiki/export.ndjson) and evaluate against your own
questions. A memory you can't audit is a memory you can't trust.