Dr. Obieda Ananbeh

PhD in computer science

Sr.Software Engineer

Microsoft Certified

Java Developer

AI

Dr. Obieda Ananbeh

PhD in computer science

Sr.Software Engineer

Microsoft Certified

Java Developer

AI

Blog Post

AI agent memory needs evidence, not just a longer summary

November 14, 2026 Tech
AI agent memory needs evidence, not just a longer summary

A coding agent resumes a task and reads its saved note: “The retry logic is finished. Tests passed. Open the pull request.” Since that note was written, someone has changed the retry requirement. The agent remembers the earlier session accurately and still makes the wrong decision.

This is the part of agent memory I care about most: whether a saved claim still applies when the agent uses it. A longer summary cannot answer that on its own.

The issue is timely. In The components of agent memory, published November 6, Anthropic describes how agents can organize notes, check them against transcripts, and correct mistakes between sessions. At this week’s AI Engineer CODE summit, November 10–12, the discussion also turned to the infrastructure needed for longer coding sessions. The useful engineering question is what a returning agent should trust before it continues.

Remembering a claim is different from verifying it

Anthropic’s examples distinguish useful memory from accumulated history. In its database-task evaluation, some models saved untested explanations for their mistakes; others saved rules with evidence for why those rules were true. The article also describes a navigation error that persisted in memory until an offline pass checked the note against the original transcript and corrected it.

These are results from particular evaluations, not proof that every coding agent benefits from the same memory design. But the distinction is useful: a note can preserve an observation, an interpretation, or a guess. Flatten those into one confident summary and the next session inherits their uncertainty without seeing it.

Software adds another complication. An observation can have been correct and become stale. A passing test report belongs to the code, configuration, and requirements that produced it. It does not automatically describe the current checkout.

Save a checkpoint the next session can inspect

I would separate durable project knowledge from task progress. “This service must support the legacy client” may deserve a maintained project note. “The migration passed its tests” belongs in a task checkpoint with a reference to the actual run.

Aerospike’s November 3 guide to context engineering makes a related distinction between the context a model receives and the state an application preserves. It argues for keeping task checkpoints separate from facts that persist across sessions. You do not need a particular database to apply that separation.

For a coding task, a useful checkpoint can identify the requirement version, repository commit and uncommitted changes, relevant artifacts, exact test command, exit status, and saved output. It should also say what remains unresolved. “Not run” is more useful than an optimistic sentence that hides the gap.

Keep the large logs outside the prompt and retain references the next session can resolve. Preserve their access controls and retention policy too. A pointer to an expired CI artifact is not usable evidence.

A small stale-memory check you can run

This Python example demonstrates one narrow check: have the files referenced by a checkpoint changed or disappeared? It uses only the standard library and creates its demonstration files in a temporary directory. The test-result file is an explicitly labeled fixture, not the output of a real test suite.

from hashlib import sha256
from pathlib import Path
from tempfile import TemporaryDirectory


def digest(path):
    return sha256(path.read_bytes()).hexdigest()


def inspect_checkpoint(root, checkpoint):
    problems = []
    for name, expected in checkpoint["files"].items():
        path = root / name
        if not path.is_file():
            problems.append(f"missing: {name}")
        elif digest(path) != expected:
            problems.append(f"changed: {name}")
    return problems


with TemporaryDirectory() as directory:
    root = Path(directory)
    spec = root / "requirements.txt"
    report = root / "test-result.txt"
    spec.write_text("Retry failed requests at most twice.\n")
    report.write_text("Example fixture: retry-limit test passed.\n")
    checkpoint = {
        "files": {
            "requirements.txt": digest(spec),
            "test-result.txt": digest(report),
        }
    }
    print("Initial:", inspect_checkpoint(root, checkpoint))
    spec.write_text("Retry failed requests at most once.\n")
    print("After edit:", inspect_checkpoint(root, checkpoint))
    report.unlink()
    print("After deletion:", inspect_checkpoint(root, checkpoint))

Save it as example.py and run python3 example.py. Its output is:

Initial: []
After edit: ['changed: requirements.txt']
After deletion: ['changed: requirements.txt', 'missing: test-result.txt']

The first inspection finds matching files. Changing the requirement invalidates that match. Deleting the report leaves both a changed requirement and missing evidence. A returning agent now has a specific reason to reopen the requirement and rerun the relevant check instead of repeating “tests passed.”

The digest does not prove that the requirement is sensible, that the report is genuine, or that any test is adequate. An unchanged report can describe a bad test. This example also checks only the listed files; it does not capture dependencies, environment variables, external services, or the whole repository. Use application-controlled paths as shown, not arbitrary paths supplied by an untrusted note.

In a real workflow, bind the test evidence to the source revision and relevant environment, and capture the command and exit status from the runner. Store that evidence somewhere the agent cannot silently rewrite to turn a failed check into a passing one. File digests are a change detector, not an authenticity mechanism.

Give the returning agent a recovery task

This prompt is a starting point for an agent that can inspect the repository and its task artifacts. Replace the bracketed fields with your own paths and constraints.

Resume [task] using [checkpoint location]. Treat the checkpoint as a record of the previous session, not proof of current state. Before editing, compare its requirement version and repository state with the current files. Inspect the referenced artifacts and test evidence. Separate verified observations from untested explanations. If evidence is missing or the relevant inputs changed, identify the affected claims and rerun the necessary checks in [approved environment]. Do not rerun external writes merely because their outcome is unclear; first inspect the exact destination for the earlier result. Report what is verified now, what remains unknown, and the next justified action. Do not mark the task complete from the summary alone.

The prompt guides recovery; the surrounding software still needs to enforce any required checks. For a pull request, CI can require a result for the current revision. For a deployment, the release system can require evidence tied to the artifact being deployed.

Start with one interrupted task. Resume it after changing a relevant input and see whether the agent notices before acting. That gives you a concrete memory test: can the system use its saved work without confusing yesterday’s evidence with today’s state?

Taggs: