Chapter 7. Memory and Context: Engineering for Long-Lived Swarms
The is a depletable "attention budget," not an infinite tape: a long-lived survives not on a bigger window but on the discipline of working with it. Three engineering decisions define the economics and reliability of such a swarm: a stable, append-only context prefix for the sake of the , the file system as external memory, and a strict protocol between and sessions. Everything else follows from these.
KV-cache: The Number-One Metric
Manus co-founder Yichao "Peak" Ji puts it bluntly: " hit rate is the single most important metric for a production " (Manus, 2025-07-18). The reason is arithmetic: Manus runs an input:output ratio of ~100:1, a typical task involves ~50 , and input on Claude Sonnet costs $0.30/MTok against $3.00/MTok uncached — a 10× difference. Almost the entire bill for an agent is input, and almost all of that input can be cached if you do not break the prefix.
Hence Manus's three rules:
- A byte-stable prefix. No timestamps at the start of the : "even a single-token difference invalidates the cache from that token onward."
- Append-only context. Past actions and observations are never edited; serialization is .
- Mask tools, do not remove them. Dynamically adding or removing tool definitions (which serialize at the start of the context) breaks the cache and confuses the model when past steps reference tools that have vanished. Manus constrains the action space through logit masking / constrained decoding, and later trimmed the set down to <20 atomic tools plus a CLI (Lance Martin, 2025-10-15).
This is not only vendor experience: an independent measurement by PwC US (not an academic group) across 500+ agent sessions on OpenAI, Anthropic, and Google showed an cost reduction of 41–80% and a time-to-first-token improvement of 13–31% (arXiv:2601.06007, 2026-01). With an important caveat: naively caching the entire context "can paradoxically increase " — the winning layout keeps dynamic content at the end of the system prompt and excludes volatile tool results from the cacheable zone.
The File System as Memory: The Convergence Point of 2026
The strongest signal on this topic is the independent convergence of three vendors on a single substrate. Manus calls the file system "the ultimate context: unlimited in size and by nature"; context compression must be _recoverable_ — drop the contents of a page but keep the URL; drop the contents of a file but keep the path. Anthropic shipped the memory tool — file CRUD under the /memories prefix, GA on the Messages since 2025 (no beta header, all Claude 4+ models, executed client-side). Letta, a pioneer of agent-managed memory (the MemGPT line), in early 2026 openly abandoned specialized server-side memory tools in favor of "generalized computer-use tools over git-backed file systems" — Context Repositories (Letta, 2026-03-16).
This is where the open part of the chapter ends. The rest opens with a free registration — no payment, just an email address.