I want to argue that agent memory should be a graph, so I am going to start with the strongest evidence that I am wrong.
Earlier this month, a team at Samsung's Warsaw research group published a systematic comparison of graph-based retrieval against ordinary vector retrieval. Across their datasets, GraphRAG came last. On COVID-QA, vector retrieval scored 0.3790 on factual correctness against GraphRAG's 0.3092; on eManual, 0.4793 against 0.3424. The cost difference is the part that should stop you: building the graph index consumed between 25.8 and 49.3 million tokens, against 1.5 to 3.7 million for their lightweight alternative. An order of magnitude, for worse results.1 Their title is not subtle — they argue vector retrieval is almost enough.
There is a second finding that cuts deeper for anyone building agents specifically. A benchmark published in April asked whether agentic search — letting the system retrieve repeatedly, refining as it goes, rather than in one shot — could compensate for not having explicit structure. Broadly, it can: the authors report that agentic search substantially improves dense retrieval and narrows the gap to GraphRAG. Their caveat is that graphs remain advantageous for complex multi-hop reasoning and produce more stable agent behaviour, once the offline construction cost is amortised.2 But the headline is that iteration substitutes for structure more than most graph advocates would like.
So if your question is "should I build a knowledge graph to answer questions over my document corpus," the honest 2026 answer is probably no. Start with vectors. Ship it.
That is the wrong question
Here is where I think the debate has gone sideways. Almost every one of those benchmarks measures document question answering: given a corpus of text, retrieve the passage that answers a query. And for that task, the sceptics are winning on the merits.
Agent memory is not that task. An agent operating over weeks does not primarily need to find the right passage. It needs to know what is currently true, how things relate to each other, and where a given belief came from. Those are not retrieval-quality problems that a better embedding model eventually solves. They are representation problems, and a vector index has no representation for any of them.
A vector store answers one question extremely well: what text is semantically similar to this text. That is genuinely useful and it is not nothing. But notice what it cannot express. It cannot express that this fact replaced that fact. It cannot express that these two differently-worded mentions are the same entity. It cannot express that a conclusion was derived from three specific sources rather than vibes. Similarity is a single scalar relationship between two blobs of text, and memory is not made of similarity.
Time is the argument
The clearest case is temporal, and it is the one in the diagram above because I think it is close to decisive.
Suppose a customer's billing address changed in March. Both the old and the new statement now sit in your index, each a perfectly good chunk of text, each highly similar to the query "what is the billing address." The retriever returns whichever scores higher, and the score is a function of phrasing, not of truth. Embeddings have no notion of validity intervals. There is no field in a vector where "this stopped being true on the fourteenth" could go.
The usual patch is metadata filtering — attach timestamps, filter by recency. It helps and it is not sufficient, because recency is not the same as validity. A document written yesterday can describe a state of affairs from two years ago. A fact from 2023 can still be current. And "most recent chunk mentioning X" breaks the moment two facts about the same entity have different lifespans, which in any real business is immediately.
The work that has most changed my thinking here is Zep, which stores agent memory as a temporally-aware knowledge graph — facts carry validity periods, and new information invalidates old edges rather than overwriting or duplicating them. On the Deep Memory Retrieval benchmark it edges out the prior state of the art, 94.8% against 93.4%. The more interesting result is on LongMemEval, designed for the messier temporal reasoning that enterprise use actually involves, where the reported improvement runs up to 18.5% in accuracy alongside a 90% reduction in response latency.3
The latency figure is the one I would point at. It suggests the win is not merely "the graph knew more" but "the graph did not have to stuff a pile of maybe-relevant history into the context window and hope." Which connects to something I have written about before: every unnecessary token you retrieve is not just cost, it is degraded reliability downstream. A structure that lets you fetch the current fact rather than every historical mention of it is doing harness work as much as memory work.
Provenance is the other argument
The second thing a graph gives you that a vector index cannot is an answer to "why does the system believe this."
In a top-k similarity search, the honest answer is: because these chunks scored highest against that query, under an embedding model whose geometry nobody can interrogate. That is a fine answer for a research assistant. It is not an answer for anything with a regulator attached, and it is not an answer you can put in front of an auditor who wants to know how a conclusion was reached.
A graph traversal is a path. It has named edges, dated assertions, identifiable sources. When something goes wrong you can walk backwards from the conclusion to the assertion that caused it, and when someone asks why, the answer is a route rather than a score. You cannot audit a cosine similarity.
This is why I think the calculus is genuinely different for regulated work than for general product work, in the same way it was different for the containment question. If your agent summarises articles, the cost of an unexplainable retrieval is a bad summary. If your agent touches filings, the cost is an obligation you cannot discharge.
The part nobody sells you
If graphs are so useful, why do the benchmarks look the way they do? Partly because the benchmarks measure document QA. But partly because the expensive, unglamorous middle of this work is genuinely hard, and it is not the part anyone demos.
The hard part is deciding that two things are the same thing. "Acme Corp," "ACME Inc.," "acme" in a subdomain, and a company number in a filing may all be one entity or four, and getting that wrong does not produce an error — it produces a graph that is confidently incoherent. Entity resolution is where these projects die, and it is why the token costs in that Samsung comparison are what they are: something has to read everything and decide what is what, and at present that something is a language model, at scale, at cost.
Then the design questions, which are the actual engineering and the reason I would call this a discipline rather than a configuration choice. What earns the status of a node, and what is merely an attribute? Which relationships are worth materialising as edges, given each one is a commitment you will maintain forever? What gets forgotten — because a memory that only accumulates is a memory that degrades, and deciding what to drop is a design decision nobody wants to own. Get these wrong and you have built an expensive, brittle representation of the wrong abstraction, which is strictly worse than a vector index that was at least cheap.
[ PLACEHOLDER: a real flat-retrieval failure you have hit — the moment the top-k came back plausible and wrong, or returned the superseded version of something, or could not answer a question that spanned two records. One concrete failure is the honest on-ramp into a forward-looking argument, and it is the only part of this essay that can come from experience rather than reading. ]
What I would actually do
Given all of the above, my position is less exciting than the title of this essay implies.
Start flat. The sceptics are right about the starting point. A vector index over your documents is cheap, fast to build, and will handle more than you expect — especially now that agentic, multi-round retrieval recovers much of what structure used to provide.
Instrument for the specific failures that indicate you have outgrown it. Not general dissatisfaction — these four signatures. Queries that need two hops and quietly return one. Answers that are correct about a state of the world that has since changed. Two records that are obviously the same entity to a human and separate to your system. And anyone asking "where did that come from" and getting no useful answer. Those are the failures structure fixes; nothing else is.
Introduce structure narrowly, over entities rather than documents. The mistake is deciding to "build a knowledge graph" as a project. The thing that pays is far smaller: identify the handful of entities your domain actually turns on — customer, matter, filing, order — and model those and their relationships over time, while leaving the prose in the vector store where it belongs. Hybrid is not a compromise here; it is the correct architecture, because the two systems are answering different questions.
Treat the graph as state, not as an index. This is the reframe I would most want to land. A retrieval index is a derived artefact you can rebuild from source at any time. Agent memory is not derived — it accumulates, it is authoritative about what the system currently believes, and it needs versioning, migration and correction paths like any other production database. Teams get into trouble by treating memory as search infrastructure when it is closer to a system of record.
The honest caveat, and it is the whole frame for this piece: I have not run a graph memory substrate in production. Everything above is reasoning from published work and from the failure modes of flat retrieval, which I have seen, rather than from a system I have operated and can vouch for. This is where I think this goes and why, offered as a working position I expect to revise — not a recommendation I have earned the right to make confidently. The strongest counter-argument is in the essay rather than omitted from it, and it is a real counter-argument: graph construction is expensive, entity resolution is unsolved, and iterative agentic retrieval keeps eating into the gap. If I build this and it disappoints, I will write that here too.