Persistence & Threading

Agentic AI · LangGraph

How a stateless agent — brilliant in a demo, broken in production — learns to remember: saving a snapshot after every step, keyed to a conversation, so it survives restarts, reboots, and days-long gaps.

Updated Jul 2026 · Read ~19 min · Session 8.1 (continuous CRM support-agent build)

Notebook Summary#

Copy this by hand. Everything here is expanded in the Detailed Notes below.

Detailed Study Notes#

A customer messages your support agent at 9:14 a.m.: their subscription was double-charged. The agent looks up the account, finds the duplicate transaction, and begins explaining the refund. At 9:16 the customer steps away. They come back at 2:30 p.m. and type, "what do I need to do next?" — and the agent has no idea who they are. Every tool call, every fetched record, every step of reasoning is gone. It starts from zero, and the customer has to repeat everything. A human clerk with a notepad would have done better.

That gap between a clever demo and a system that works at 2:30 p.m. is the whole subject of this lesson. Before we get there, a quick recap of where the agent stands.

Recap: the React loop and its three safety layers#

The agent reasons, then acts, in a loop — and three guards keep that loop from hurting itself. "React" here means reason + act: the model thinks about what it needs, calls a tool to get it, reads the result, and decides whether to think again or answer. The graph flows start → classify → agent ⇄ tool → respond → END, where the classifier labels the ticket (technical, billing, fraud, general) and routes it.

All three are real-but-local guards. They keep a single run healthy. What none of them address is what happens between runs — and that is where persistence begins.

Why stateless execution fails in production#

When a process ends, everything it held in memory is erased — completely and without discrimination. Every Python variable, every object, every LangGraph state dictionary lives in RAM, in a region the operating system leased to that process. When the process terminates — finished normally, crashed, killed with Ctrl-C, or knocked out by hardware failure — the OS reclaims every byte. It does not ask which parts mattered. There is no partial survival.

This is not a defect. It is the same isolation that lets a machine run thousands of processes without their memory bleeding into each other. It is also exactly why a naive agent forgets.

In production, process restarts are not a rare accident — they're routine: rolling Kubernetes deployments, autoscaling events, infrastructure failures. A non-persistent agent mid-workflow, three tool calls deep, loses all of it on any restart. For an in-flight conversation, every restart is total amnesia. So this isn't a theoretical risk you might hit; without persistence it's a guaranteed production failure.

Stateless vs stateful: the architectural mismatch#

The web is stateless by design; a conversation is not — and forcing one into the other breaks it. HTTP was built so each request carries everything needed to serve it, with the server remembering nothing about prior requests. That makes REST simple, scalable, and predictable: send a customer ID, get customer data back, done. It's the model behind essentially every large web service.

An agent conversation is the opposite. It's a sequence of exchanges where each one depends on the accumulated context of all the ones before. When a customer asks "what do I do next?", a correct answer depends on what they asked two minutes ago, which tool was called, and what it returned. You cannot make that stateless without destroying the intelligence of the interaction.

Primitive one: the thread#

A thread is a conversation's unique identity — a label that groups everything belonging to one conversation together. Despite the name, it is not an OS thread, a process, or a network connection. It is a logical identifier: the thread_id. It behaves like a foreign key — the address that tells each checkpoint which conversation it belongs to.

When you invoke the graph with a given thread_id, LangGraph loads the last checkpoint for that conversation and resumes. Invoke with a different id and you're in a different, isolated conversation. This is the thread-isolation guarantee: user 42's messages can never contaminate user 99's, because every write is tagged with its thread ID (often combined with a user ID into a composite key). It's also what lets you scale horizontally — ten instances of the same agent can serve ten conversations with no risk of cross-talk.

Primitive two: the checkpointer#

A checkpointer is an automatic state ledger: it snapshots the full state dict around every node and writes an immutable entry. It hooks into the execution engine and fires before a node runs (recording which node is about to run, the full state, the config) and again after it completes (recording the updated state). This is not configurable — you can't skip checkpointing for a node — and the ledger is append-only: prior snapshots are never overwritten or deleted. Every state the graph has ever been in is preserved, which is exactly what makes time travel possible.

A snapshot is written before and after each node; the ledger only grows. On restart with the same thread: load the latest → resume. No replay needed.

Where snapshots live: choosing a saver#

The checkpointer interface is fixed; only the backing store changes — and swapping it is one line of code. From the graph's perspective every saver is indistinguishable. That lets you develop against the fast in-memory saver, then switch to a durable store for production without touching graph logic.

Serialization and the schema contract#

To save a snapshot, the checkpointer serializes state into a storable form — which means your state must be serializable. With MemorySaver this happens in memory; with SqliteSaver and PostgresSaver the state is serialized to JSON and written to a database column. That creates a contract: every field in your TypedDict state must be JSON-serializable — strings, ints, floats, booleans, lists, dicts. No arbitrary Python objects.

from typing import TypedDict, Annotated
from langgraph.graph.message import add_messages

class SupportState(TypedDict):
    messages: Annotated[list, add_messages]  # JSON-safe list
    category: str
    iteration_count: int
    tool_results: dict
    schema_version: int                      # <-- guards against drift

Wiring it up: compile with a checkpointer, invoke with a thread#

Attaching a checkpointer changes execution in two ways, and the thread config tells it which conversation you mean. When LangGraph sees a checkpointer on a compiled graph, then: (1) before the first node, it checks the store for an existing checkpoint under this thread_id — loads it if present, uses the provided input as-is if not; and (2) after every node, it saves the updated state. Wiring this into an existing build is famously small: only three lines change — create the saver, pass it to compile(), and add a thread_id to each invoke. The graph, nodes, and tools are untouched, and the support.db file is created automatically on first run.

from langgraph.checkpoint.sqlite import SqliteSaver   # dev: MemorySaver — one-line swap

# support.db is created automatically on first run; survives every restart
checkpointer = SqliteSaver.from_conn_string("support.db")
graph = builder.compile(checkpointer=checkpointer)     # the only line that changes

# thread config MUST be nested under "configurable"
config = {"configurable": {"thread_id": "user-arjun-001"}}

# Script A — first turn: send the full initial state
graph.invoke(build_initial_state("My account C-1002 is showing past due"), config)

# ...server restarts (routine k8s deploy)... hours later, same thread_id.
# Script B — follow-up: send ONLY the new message. The checkpointer reloads
# messages, customer_data, and tool fingerprints transparently.
graph.invoke({"messages": [("user", "My email is arjun@company.com")]}, config)

Time travel: state history as a production capability#

Because every step is stored, you can walk backward through a conversation, inspect any past state, and resume from it. Imagine an agent that's been live for two weeks; a customer reports it gave them wrong information three days ago that caused financial harm. With a non-persistent agent, the state that existed then is simply gone — at best you have scattered logs. With checkpointing, you load the customer's thread, enumerate its checkpoints, find the one from three days ago, and inspect exactly what the model saw, what tools were called, and what they returned. That's a full audit trail.

Each entry from the history is a state snapshot with four parts. Keeping the first three straight is a common exam question — config and metadata look similar but do different jobs — and the fourth, next, is the quiet one that powers human-in-the-loop:

# current state
snapshot = graph.get_state(config)
print(snapshot.state["messages"])        # the data
print(snapshot.next)                      # () = done; ("agent_node",) = paused there

# walk the full history, newest first -> reverse for chronological
for snap in reversed(list(graph.get_state_history(config))):
    print(snap.metadata["step"], snap.metadata["source"])   # audit: which node, when
    print(snap.config["configurable"]["checkpoint_id"])     # resume handle

# rewind: re-invoke FROM a past checkpoint. Pass None as the input so the
# graph uses the checkpoint's saved state instead of overriding it.
old = {"configurable": {"thread_id": "user-arjun-001",
                        "checkpoint_id": "<id-from-history>"}}
graph.invoke(None, old)

Why this matters next: human-in-the-loop#

Persistence is the foundation that lets an agent pause mid-run, wait for a human, and resume — and without a checkpointer there is nowhere to pause. A non-persistent graph runs start to finish with no way to stop before a consequential action (say, writing to a repo) and no way to inject a human decision. Because the ledger records snapshot.next and preserves every state, execution can halt before a write, save that state, let a human review the draft, and then resume from the exact checkpoint. That's how a later session wires up interrupt_before and Command(resume=...) — but it only works because the checkpoint ledger exists first.

Interview & practice Q&A#

Key Takeaways#