Raw-RAG tops Mem0-class agentic memory systems on six of seven long-conversation benchmarks

The memory stack got clever. The baseline that still indexes the transcript may have been under-specified.

On 22 October 2026, Zhengye Bao, Shengtao Zhang, Haokai Zhang, Shiqi Lou, Muning Wen, and Minnan Luo posted arXiv:2610.30755, "Is Complex Agentic Memory Always Needed? Naive RAG Is Stronger Than We Thought," with companion code at Litmeb/RAW-RAG. Their diagnostic system, Raw-RAG, chunks raw conversation history (timestamps and image captions included) into overlapping windows, embeds them in a flat index, retrieves the top dense candidates, reranks with a cross-encoder, and hands the top windows to a reader LLM. No summarisation, no graph, no memory mutation. The reader reconstructs what is current from the evidence.

Agentic memory is sold as necessary because raw-history RAG supposedly fails

The claim we are checking is the one that justifies the growing write-time stack: long-term conversation understanding requires explicit fact extraction, UPDATE/DELETE mutation, graph or hierarchical consolidation, and learned retrieval, because retrieving untouched history cannot keep up with superseded facts, deletions, and dependent state changes. Prior papers often report RAG trailing those agentic systems. If that claim holds, a carefully specified naive baseline should lose cleanly.

Bao et al. put the claim on LoCoMo, LongMemEval, MEME, and MemoryAgentBench

The counter-evidence is the paper itself and its public repository. Compared systems include Mem0, A-MEM, EverMemOS, Memora, MemMachine, MemOS, EMem, SimpleMem, and others, plus Full-context when the window fits. Settings cover LoCoMo and LongMemEval under aligned reader models, plus MEME and MemoryAgentBench under a shared evaluation procedure. Concurrent ReFind-style work that keeps raw logs with agent-controlled search is cited as related context; this piece stays on the fixed dense-retrieve-and-rerank baseline.

Six of seven wins, thin mutation rates, and coverage over utilisation

Raw-RAG ranks first among the compared memory systems in six of seven benchmark–model settings. Margins over the strongest agentic comparator run from 0.13 to 5.53 points depending on the setting. On LoCoMo with GPT-4o-mini, Raw-RAG scores 89.07 against MemMachine at 88.12. With GPT-4.1-mini it scores 93.70 against EverMemOS at 93.05. On LongMemEval with GPT-4.1-mini the lead shrinks to 87.53 against Memora at 87.40. On MEME with GPT-4o-mini Raw-RAG reaches 39.30 overall, ahead of MemMachine at 37.06. On MemoryAgentBench it leads Memora by 4.48 points with GPT-4o-mini (48.32 versus 43.84) and by 5.53 points with DeepSeek-V4-Flash (53.41 versus 47.88).

The one miss among memory systems is DeepSeek-V4-Flash on MEME, where MemMachine leads Raw-RAG by 1.52 points (44.71 versus 43.19). Full-context still beats Raw-RAG badly in that setting when the window fits, by about 20 points, which is a reminder that retrieval remains a ceiling when every phase-visible turn can be packed.

Two diagnostics attack the usual reasons for rejecting raw-history RAG. Mem0's UPDATE and DELETE together are 4.39% of 33,902 write decisions on LoCoMo and 2.46% of 428,858 on LongMemEval; effective text-changing mutations fall to 1.90% and 1.34%. Replacing UPDATE with append-only ADD and DELETE with NONE raises LoCoMo accuracy from 67.36% to 69.42% without reducing any category score. On MEME, a phase-visible oracle that feeds chronological raw sessions still lets strong readers resolve cascades and invalidations when the evidence is present.

Why the baseline wins is more interesting than the leaderboard. Systems differ far more in coverage of necessary evidence than in utilisation once that evidence is present. On MemoryAgentBench, Raw-RAG coverage is 45.29% against a utilisation band of roughly 80–90% across systems. On MEME, coverage spans about 41 points while utilisation spans about 21. Useful evidence includes answer values and the relations and dependency paths needed to derive them; Raw-RAG's conditional path coverage on MEME tracking and cascade after-state questions is 62.34%, ahead of MemMachine at 56.70%. A complete-store oracle over exported agentic memories still leaves an accuracy gap versus gold raw evidence on MEME, so ingestion-time compression can discard information later retrieval cannot recover. Ablations that add atomic facts, summaries, or graph expansion on top of Raw-RAG hurt overall MemoryAgentBench score by 5.78 to 19.62 points.

Ingestion cost is the other half of the story. On a 10-episode MEME slice, Raw-RAG's add-to-ready time is 50.32 seconds, against thousands for EverMemOS and Mem0. On a roughly 1M-token BEAM stream, cumulative add time stays near 0.33 hours while Memora runs to about 27 hours.

Production memory would need settings these offline benches do not settle

For the necessity claim to survive, agentic writes would need to beat this specified Raw-RAG pipeline on the buyer's interactive, multimodal, or privacy-constrained workloads, with weaker readers that cannot reconstruct state from chronological raw windows, or under deletion and retention rules that forbid keeping the transcript. Sub-point margins on LoCoMo and LongMemEval already show how thin the board can be. Storage, privacy, and concurrent agent-controlled search over raw logs remain open product questions the paper marks as limits rather than settled wins for either camp.

Overstated for these conversation benches; keep Raw-RAG as the bar

Verdict: overstated as a general claim that complex agentic memory is required for long-term conversation understanding on the evaluated offline public benches. Supported as a baseline-and-diagnosis result: vendors and papers that claim gains from mutation, graphs, or hierarchical consolidation should show them against a well-specified dense-retrieve-and-rerank raw-history pipeline, and say what the added write complexity contributes once coverage and path preservation are controlled.

Demand a named Raw-RAG comparator before pricing the write stack

We will say Bao et al. showed that Raw-RAG ranks first among compared memory systems in six of seven settings across LoCoMo, LongMemEval, MEME, and MemoryAgentBench, with margins of 0.13–5.53 points over the strongest agentic comparator, and that coverage of values plus dependency paths helps explain the result. We will treat production necessity, privacy-constrained deletion, and weaker-reader regimes as unsettled. When we cite Mem0-class products, we will ask whether their evals beat this Raw-RAG specification rather than a coarse, shallow, or rerank-free RAG straw man.