Memex — the extra-context layer for AI agents

A high-performance context hierarchy between an agent’s history and its model window.

memex demo

Memex treats context as a hierarchy, not a notes store: when the native context window fills up, older spans are moved out verbatim into sticky L2 (10M, session-private) / L3 (100M, workspace-shared) context pools; capacity is managed through a lossless cold tier plus bounded batch digests; and selected context pages can be retrieved under per-model context budgets. Retrieval and excerpting do not guarantee every relevant fact is included.

Quick start

git clone https://github.com/QiuXu-Tech/memex.git
cd memex
node scripts/build-bundle.mjs && node scripts/check-bundle.mjs
node --import ./test/register.mjs ./test/run.mjs   # offline harness (165 tests)

Install the built bundle into a DSH profile: install guide.

Documentation

Architecture and behavior — levels, eviction seam, retrieval, compression, API/UI.
Evaluation — recall and scale benchmarks with methodology.
Benchmark report — synthetic long-horizon availability and retrieval, with charts.
Synonym and archive excerpt retest — paired seeded diagnostics and historical replay.
Retrospective pilot follow-up — bounded answer continuation, citation validation, outage evidence and cost gaps.
Follow-up registration and post-outage replication registration — preregistered designs and limitations.
Expanded retrospective validation pilot — bounded tools, AI blind review, autonomous-memory observations and cost gaps.
Expanded pilot registration — frozen pre-outcome design and sample limitations.
Exploratory production-validation pilot — randomized retrospective replay, AI review, observed tools and incomplete cost ledger.
Pilot registration — frozen design, model-review scope and later parsing correction.
Production validation protocol — proposed evidence for correctness, cost and autonomous memory claims.
Pager recall and ranking retest — paired synthetic results, lexical limits and historical-session replay.
Pager retest — pager repair, synthetic multi-seed rule/hybrid comparison, and replay limits.
SQLite and file consistency — lock handling, interrupted-operation recovery and backup scope.
Extra context, not just memory — the concept: a context hierarchy between history and the model window.
Context materialization — the direction: canonical events, pager/materializer + DSH switch, tiny trained transformer encoder, runnable paged-KV/prefix host, structural fidelity eval.
Integrations — Claude Code, Codex, Gemini CLI, Cursor, Cline, Windsurf, DSH.
Model profiles — per-family budgets, injection formats and digest prompts.
Optimization findings — retrieval/perf experiments, fixed issues, backlog.
Profile validation — cold-start + long-horizon A/B for the dedicated model profiles.
Live endpoint A/B — harness, status and how to run it with provider keys.
v0.4.0 experiments — rename/defaults verification, recall re-check and improvement points.
Claude Code · Codex · Generic MCP clients
Install guide — profile registration, config rows, local fallback.

Highlights


L1 eviction	Uses the shipped `compaction/prune` + surface-replace seam — no custom session events, no replacement of the compaction backend.
Sticky context pools	Session-private L2 vs workspace-shared L3; capture deepest enabled level; capacities 10M/100M.
Context paging	Shared query-term extractor, FTS + cold abstracts + exact numeric/identifier anchors + lexical rerank; pages are injected only when budget allows. A private archive previously reported recall@5 93%, cold recall 94%, hit@1 65% (68% with `coldPartial: 0`), and p50 ~2.9 ms. In the v0.30.0 synthetic Codex Sol A/B, control answered 3/60, treatment with pre-retrieved evidence 60/60, and evicted probes 0/57 → 57/57; all six calls exited zero and all three actual tails were checked in order. Earlier DeepSeek, Terra and Claude results have a tail-order caveat; Claude Sonnet/Opus/Fable/Haiku each reported 60/60 with ~39.7–40.7k gateway-reported tokens per model. Small historical anchor-free runs reported 0/3 → 3/3 for hot, cold and cold partial/synonym cases; a real Claude tool loop answered 3/3 against the live MCP server.
Capacity compression	Cold tier (lossless, zero LLM) and bounded map-reduce digests (2-way concurrency, retry/fallback, `ids:` enforcement, quota GC) — after the model asks the user.
Surfaces	Seven model tools, `/memory` command set, nine `/api/memory.*` routes, Web panel + context ring, sensitive-content policy, backup/verify/restore.