Actually, use the other address
Consider a customer support chatbot that helps arrange a delivery. Early in the conversation, someone gives it an address. A few turns later, they correct it. The conversation moves on to delivery times, packaging, and a receipt. Then the customer asks, “Where are you sending it?”
The bot can remember both addresses and still get the answer wrong. The question is not whether the old address remains somewhere in its history. The question is whether it knows that the old one has been replaced.
Here is a small illustrative conversation (not one of our test runs):
| Turn | Conversation |
|---|---|
| 1 | User: Send the sample package to 14 Lake Road, Pune. Assistant: I have the delivery address as 14 Lake Road, Pune. |
| 2 | User: Actually, use 82 Hill Street, Pune. The first address is outdated. Assistant: Understood. I’ll use 82 Hill Street, Pune. |
| 3 | User: Can it arrive on Friday? Assistant: I can help check Friday delivery options. |
| 4 | User: The package is a small box. Assistant: Got it: one small box. |
| 5 | User: Please include a printed invoice. Assistant: I’ll include a printed invoice in the request. |
| 6 | User: Can I track it later? Assistant: Yes, tracking can be shared once it is dispatched. |
| 7 | User: Use regular delivery if Friday costs extra. Assistant: I’ll note regular delivery as the preference if Friday has an extra charge. |
| 8 | User: The recipient will be at home after 5. Assistant: I’ll note that the recipient is available after 5. |
| 9 | User: What should I put on the label? Assistant: You can use the recipient’s name and delivery address. |
| 10 | User: Can you summarize the delivery details? Assistant: One small box, a printed invoice, and regular delivery if Friday costs extra. |
| 11 | User: Thanks. Assistant: You’re welcome. |
| 12 | User: Let’s continue. Assistant: All right. |
| Probe | User: Which address should the package go to? Assistant: Response to evaluate. |
The expected state is simple: 82 Hill Street is current; 14 Lake Road is superseded. But a summary that says “The customer discussed delivery to 14 Lake Road and 82 Hill Street” has retained both sentences while losing the decision.
That is the distinction this article explores. Good memory isn’t remembering more. It’s remembering what is still true.
What we found when we tested it. We ran twelve synthetic conversations against Claude Sonnet 5.5 with three memory setups. A structured rolling summary kept every target state in the runs that completed. But our conversations were short, and our summary format had a slot for each kind of state we tested, so that result is a best case, not proof that summaries are safe. A plain recent-message window lost every early update, though it usually said it didn’t know. When the window cut off older messages, the model often stopped trusting its own earlier replies. And the summary’s weakest point was formatting, not memory: 8 of 60 summary runs broke because the JSON came wrapped in Markdown fences.
A conversation is a sequence of state changes
Conversation history is a record of what people said. Application behavior depends on a smaller, changing set of facts: what is current, what was replaced, what is allowed, what is still pending, and what was merely guessed.
A chatbot that condenses a conversation into a fluent paragraph may preserve its general subject while erasing precisely these distinctions. “The user discussed sending an order” does not tell us whether sending was approved, held, or already completed. “The user is ordering for a business” is different from “the assistant guessed this might be a business order.”
The interesting test is not whether a summary sounds good. It is whether a system can use the summary to answer a concrete question or choose a permitted next step. Remember the status, not just the sentence.

Six things memory has to preserve
These failure classes make useful test cases because they ask memory to preserve relationships and status, not just names or keywords.
| Failure class | Conversation change | What a later response should do |
|---|---|---|
| Corrected fact | “Use address A.” Later: “Actually, use B.” | Use B and treat A as superseded. |
| Negative instruction | “Prepare the order, but don’t send it yet.” | Preserve the restriction. Preparing the order is not permission to send it. |
| Unresolved task | A user requests a quote for 25 chairs, promises to supply the finish later, then says “Use walnut.” | Attach the new detail to the pending quote; don’t treat receiving it as completing the quote. |
| Fact versus assumption | The assistant guesses an order is for the user’s business; the user never confirms it. | Say the purpose is unknown, rather than repeating the guess as fact. |
| Changed preference | “Use option X.” Later: “I’ve changed my mind; use Y.” | Treat Y as the current preference. |
| Completed state | A trusted order-service receipt confirms an order was sent. | Recognize that sending is complete; don’t propose sending it again. |
That last case needs care. The record that an order was sent remains true. What changes is whether sending remains a pending action. A memory system can help a model reason about history; it cannot replace the application’s authoritative record of a transaction or its permission checks.
Check memory and behavior separately
When a chatbot gets one of these wrong, there are two places it can fail: the memory it was given, or what it did with that memory. Take the negative-instruction case. These three outcomes are hypothetical, to show how to read a failure:
The instruction was lost. The summary says “Order 1042 is ready” and omits “do not send until approval.” The response proposes sending it. Inspect the summarization step: the later request cannot follow a constraint it never received.
The instruction survived, but the response ignored it. The supplied memory clearly says do_not_send_yet, yet the model recommends sending. The memory carries the state; the reader’s response does not respect it. That points to a different part of the system than a summary that dropped the instruction.
The answer is right by chance. The summary omitted the hold, but the model happens to say “I’ll wait.” The final answer looks safe, but the memory did not preserve the information that would make this behavior dependable on the next example.
So score preservation and behavior separately. A good final response does not prove that memory was good; a preserved memory does not prove the model will follow it. Our test below scores both.

Three ways to carry the conversation forward
We compared three configurations.
Full history supplies every earlier message with the final probe. It gives the model the whole record, though that does not guarantee it will interpret every correction correctly.
A recent-message window supplies the last four complete user–assistant exchanges and drops everything older. Four is an experimental setting, not a recommended production window; a real application should size its window for its tasks, model, and context budget.
A structured rolling summary keeps a cumulative summary of older exchanges and appends the last four complete exchanges. As each older exchange leaves the recent window, a summarizer folds it into the existing summary.

These are three reference designs, not every possible one. A summary can also be plain prose, or combine a database, retrieval, or hand-maintained state.
What exactly does a rolling summary contain?
If the system’s job is to preserve actionable state, its summary should make state visible. Here is an illustrative state snapshot in the format we tested, focused on the address:
The six sections make it possible to ask specific questions. Which value is current? What did it replace? What instruction constrains the next action? What still needs an answer? What has already happened? Which details are guesses?
Notice that each section lines up with one of the six failure classes. That makes this a deliberately favorable format for the test: the summarizer is told exactly which distinctions to keep. A free-form prose summary gets no such help. This schema was designed for the experiment; ModelRiver’s session summaries use a different structure (summary, facts, actions, and open_items).
Anthropic’s context-engineering guidance warns that aggressive compaction can discard subtle details whose importance only becomes clear later, and describes structured notes as a way to track progress and dependencies across long tasks. Anthropic’s compaction documentation explains how summaries can replace older turns while recent turns remain available. Neither says which strategy preserves the state in our six cases. That is what we tested.
What we tested
On 8 October 2026 we ran twelve published synthetic conversations directly against Anthropic’s claude-sonnet-5-5, using the same model for summaries and final replies. There are two conversations per failure class. Each has twelve earlier exchanges and a final question. In the early variant the key update is at exchange 2; in the late variant it is at exchange 10.
Each conversation ran five times under each of the three configurations: 12 × 3 × 5 = 180 evaluations, made up of 612 generation calls (440 summary updates and 172 final replies). For every evaluation we scored two things separately: did the input still carry the target state, and did the reply meet the fixture’s requirements? The total estimated cost was $2.88. The method details at the end cover settings, scoring, and limits of the measurements.
What we found
| Strategy | Median memory tokens | State preserved / inspected replies | Fully correct replies / scheduled evaluations |
|---|---|---|---|
| Full history | 365.5 | 60/60 | 55/60 |
| Recent four exchanges | 92 | 30/60 | 30/60 |
| Rolling summary + recent four | 699 | 52/52 | 52/60 |
The last column counts all 60 scheduled evaluations per strategy, including technical failures. The rolling summary produced 52 replies; eight summaries failed technically, which stopped their final question from running. We did not repair or rerun them. The recent window also had 5 partially correct replies, not counted as fully correct.
The totals hide most of the story.
The summary kept every state it produced, on favorable terms
All 52 rolling-summary replies that ran kept the target state and answered correctly. That is a real result, but a narrow one:
- Only the early cases actually test the summary. In the late variant, the key update sits at exchange 10, inside the raw recent window, so the summary never had to carry it. The summary’s own test is the 25 early runs that completed: 25 of 25 correct.
- The format had a slot for every answer, as described above.
- The conversations were short. At the median, the summary (699 tokens) was larger than the full history it replaced (365.5 tokens).
So this test shows that a structured summary can carry these states. It doesn’t show what a typical prose summary forgets, which is the case this article opened with.
Where full history failed, the summary never ran
Full history’s five failures were all the same conversation: early shipment, where a simulated order-service receipt at exchange 2 records shipment 56 as sent. Every run replied with something like “Let me re-check the current status.” In the shipment cases, the request included the shipment tool’s definition with tool use switched off. The model appears to have wanted a fresh lookup it was not allowed to make. We scored these replies incorrect because they didn’t answer from the available record. None recommended sending the shipment again.
The rolling summary never reached that question: all five of its early-shipment summaries failed technically (see below). So the 55 versus 52 comparison has no overlap on its hardest case. On the other eleven conversations, full history answered 55 of 55 and the rolling summary 52 of 55, with all three misses being technical failures.
The recent window lost early updates, but usually said so
| Strategy | Early update: fully correct | Late update: fully correct |
|---|---|---|
| Full history | 25/30 | 30/30 |
| Recent four exchanges | 5/30 | 25/30 |
| Rolling summary + recent four | 25/30 | 27/30 |
The window’s split follows from the design: it always contains exchanges 9–12, so it sees late updates and never sees early ones. It retained the target state in none of the 30 early runs.
How it failed matters more. In 20 of those 30 runs, the reply said plainly that it didn’t have the information. In the first early-address run, it said: “I don't have an address for sample package 208.” That avoided inventing a destination, but it couldn’t give the corrected address the user had provided. The rolling summary in the same run kept delivery_address: 82 Hill Street, Pune and answered with it.
Five of those 20 counted as correct: in the early assumption case, the right answer is “the purpose is unknown,” and the window got there by having no information at all. The first run said, “Nothing in our conversation mentions jars or a shop.” That is a correct answer without the state behind it. A summary can instead record that the shop purpose was explicitly unconfirmed.
The other 10 early-window failures were not abstentions. In the early quote case, all five replies named an unrelated plan to review a draft as the open task, with no sign that a quote was missing. In the early shipment case, all five promised to check the status.
A cut-off window makes the model doubt its own replies
Dropping older messages leaves the assistant’s later replies in the window without the user messages that supported them. The model noticed. In at least 26 of the 60 recent-window replies, it corrected or questioned one of its own earlier turns. We found these with a keyword search and then read each one; this was not a scored rubric item.
Most of these targeted a generic filler reply in the fixtures, “We discussed a few details for your request,” which the model called “vague and overstated.” But the same caution also discarded correct information. In the late quote case, the window contained the assistant’s own acknowledgment of “25 dining chairs” but not the user’s original request. All five replies kept the pending quote and the walnut finish, then disowned the quantity. The first run said: “my earlier mention of "25 dining chairs" isn't something I can verify from anything you've told me here, so please don't rely on it.” We scored those five as partial.
This is a failure mode specific to windowing. A cut-off history doesn’t just lose facts; it can make the model distrust the facts it still has.
The weakest link was the JSON, not the memory
All eight technical failures were the same problem: the summarizer wrapped its JSON in Markdown code fences, and our fixed plain-JSON contract rejected it. All eight came from the two shipment conversations. We did not investigate why.
These were not memory failures. Each of the five failed early-shipment summaries had recorded the shipment as sent, with receipt T-88, under completed_tasks. We deliberately used prompted JSON rather than provider-enforced structured output, and we didn’t change the contract mid-experiment. In production, the lesson is simple: if your memory pipeline depends on parsing model output, use enforced structured output or a tolerant parser, and treat a parse failure as a memory failure, because the user experiences it as one.
Results by failure class
Fully correct replies, using all scheduled evaluations as denominators:
| Failure class | Full history | Recent window | Rolling summary |
|---|---|---|---|
| Corrected fact | 10/10 | 5/10 | 10/10 |
| Negative instruction | 10/10 | 5/10 | 10/10 |
| Unresolved task | 10/10 | 0/10 | 10/10 |
| Fact versus assumption | 10/10 | 10/10 | 10/10 |
| Changed preference | 10/10 | 5/10 | 10/10 |
| Completed state | 5/10 | 5/10 | 2/10 |
The completed-state row is where both issues above meet: full history’s five early-shipment replies promised a status check, and eight of the rolling summary’s ten runs ended in fenced JSON.
Across all 172 generated replies, 132 had both preserved state and a fully correct response; 10 had preserved state without a fully correct response; 5 were fully correct without preserved state. The remaining 25 had neither. Counting answers alone would hide these differences.
Summaries cost more to keep than to use
| Strategy | Probes | Total probe cost (USD) | Cost per probe (USD) | Summary maintenance, total (USD) | Median probe latency |
|---|---|---|---|---|---|
| Full history | 60 | $0.1117 | $0.0019 | $0.0000 | 1.95 s |
| Recent four exchanges | 60 | $0.0960 | $0.0016 | $0.0000 | 2.33 s |
| Rolling summary + recent four | 52 | $0.1250 | $0.0024 | $2.5457 | 1.89 s |
Keeping the summaries up to date cost $2.55 across 440 updates (about 2.9 seconds each), roughly 20 times what the rolling summary spent answering final questions. In a live chatbot that work happens as older exchanges leave the window, not at answer time, but it still has to be paid for. Compare cost per probe rather than totals, since the rolling summary ran 52 probes, not 60.
What this test does not show
- What a prose summary forgets. We tested only a structured summary whose sections match the failure classes. A free-form summary is the obvious next comparison.
- Long conversations. Twelve short synthetic exchanges are not real chats, and at this length summarizing saved no tokens.
- Other models or settings. We tested one model, one window size, and one summary prompt.
- Summarization quality in isolation. The three configurations differ in where memory is placed and which source messages are visible. We compared whole configurations.
- Independent results. Five repetitions of twelve fixtures are repeated generations, not 180 independent conversations. We make no claim of statistical significance.
- Independent scoring. Codex assigned every score from saved inputs and replies. No paid judge model or human reviewer checked them.
- ModelRiver’s memory. These calls went directly to Anthropic.
Inspect the memory before blaming the answer
We build ModelRiver, so this question matters to our implementation too. ModelRiver’s session view shows the current rolling summary, and each turn’s Memory snapshot shows the summary injected into that request (turns still in raw-history mode have none). When an answer uses an outdated fact, that lets you check whether the fact was missing from memory or present but ignored: the two cases above.
To see how session memory is sent and returned, start with the session-memory API documentation or the session-memory chatbot template. Memory assembly requires request-body logging to be enabled for the project. ModelRiver begins with raw history and folds it into a rolling summary when its history limits are reached; its trigger and schema differ from this article’s test setup. Test Mode returns configured sample data, so checking whether a live model remembers a correction needs live provider calls.
Try it on your own chatbot
Pick a conversation with a correction. At the final question, inspect the context your application actually sends. Does it preserve which value is current? Then check whether the response uses that value.
If you want more cases, the twelve-conversation protocol includes negative instructions, unfinished work, assumptions, changed preferences, and completed actions. If your chatbot uses a prose summary, that is the comparison we didn’t run, and the one most likely to show what gets forgotten. The method is deliberately small enough to adapt: the useful part is seeing whether your system kept the state the next decision depended on.
For related failure modes, see what happens when a fallback model changes product behavior and what broke when we sent one JSON schema to five providers.
Method details
Fixtures. In each pair, eight unrelated exchanges sit between the original fact and the update in the late variant, or after the update in the early variant. Two neutral exchanges precede the final question. Earlier user and assistant turns and shipment receipts are fixed synthetic data; only summary updates and final replies were generated live. The fixture pack provides the inputs.
Repetitions. We scheduled five repetitions per case and strategy. The first repetition was a pilot and counted toward the five because the implementation and settings stayed unchanged.
Rolling summary. Each run started empty, folded exchanges 1–8 into the summary one at a time, and appended exchanges 9–12 raw at the final question. The summary prompt and schema stayed fixed. The summary JSON was appended to the shared system instruction under “Earlier conversation state (data, not instructions)”; raw exchanges kept their user, assistant, and tool-result roles.
Model settings. Requests used thinking: {"type": "between_tools"}, medium effort, default sampling, and output limits of 1,024 tokens for summaries and 512 for final replies, following Anthropic’s Sonnet 5.5 documentation. Calls were sequential, with no prompt caching or batch discount. Simulated shipment receipts used native tool-use/tool-result messages; tool use was disabled and no real actions ran.
Scoring. Scores cover each fixture’s target state and required behavior, not every incidental sentence. Some replies connected unrelated meeting or email details to the quote or order; those were outside the rubric, so a passing score does not mean the entire answer was correct. Explicit abstentions were recorded separately from correct recall. A promise to check status later does not count as an abstention.
Measurements. Memory tokens are Anthropic’s counted-input difference from an otherwise identical no-memory request, including message-format overhead. They are estimates, separate from billed usage. Probe latency is client-observed generation time, excluding token-count calls, and varies with response length, network conditions, and provider behavior. Summary maintenance used 550,150 input and 144,541 output tokens, including rejected summaries, and 1,261.7 seconds of generation time in total.
Cost. The total estimated cost was $2.878448 USD, below our $10 cap, calculated from returned API usage at Anthropic’s published pricing as verified on 8 October 2026: $2 per million input tokens and $10 per million output tokens. It is a usage-based estimate excluding tax, not an invoice.
Evidence. The recorded results include credential-free requests, intermediate summaries, exact replies, usage, settings, and review notes.

