Limits for long sessions

View as MarkdownView as Markdown

A session can stay open for months. These server-zd limits decide how far it can grow.

Key	Default	What it bounds
limits.maxFlowStateBytes	64 MiB	The active state of one session: the Attribute and Channel values it holds now. History, and the superseded values that history keeps for time travel, do not count.
values.maxValueBytes	16 MiB	The largest single value, such as one message, one tool call record, or one framework run state.
history.windowEvents	About 50,000 events, or 5,000 with dev and local	The Flow history kept for each run. Older events are pruned.
retention.closedFlowDuration	7 days	How long a stopped session stays readable.

Each run records these values when it starts. A new value applies only to sessions that start after the server restarts with it.

How a session stores its state

• AgentMessages keeps the newest messages, one AttributeMap instance per message. Storing a message writes only that message. When it holds more than 64 messages, the oldest move into AgentMessageChunks instances, which are never rewritten. A chunk holds at most 32 messages and 1 MiB, so it stays well under values.maxValueBytes.
• A message larger than the chunk byte limit skips AgentMessages and goes straight into a chunk of its own when it is stored. So a message too large for values.maxValueBytes fails its turn right away.
• AgentMessageIndex records where the messages are. It changes only when chunks are written or deleted.
• Each message is written twice, once in AgentMessages and once in its chunk. Once the AgentMessages copy is removed, it no longer counts toward the run budget.
• With compaction, RunModel loads only the messages after the summary. Without compaction, every model call loads and sends the whole history, and nothing is deleted.
• FrameworkRunState holds the framework's state while a turn waits for tool results. RunModel rewrites it after every model call that asks for tools. It holds the turn's work so far, tool outputs included. The OpenAI Agents SDK state also holds the turn's input. The Vercel AI SDK state holds each tool output twice. Only the current value counts toward the run budget. History keeps each earlier value for time travel, but the run budget does not count it.
• AgentToolCalls records are deleted when the turn ends. They then no longer count toward the run budget.

What happens at the limit

The server checks the run budget for Step heartbeats, Step completions, and RPCs that change Flow state, such as send_user_message. It checks only writes that add active state. Once a session is over the budget, the server refuses such a write. What happens next depends on what was writing.

• A model call or tool call. The session ends, because the next turn would hit the same limit. Its status becomes failed. last_turn_error holds the reason, such as "active run state of 67492 bytes exceeds the maximum of 65536 bytes", and the Flow closes failed with the same text. The Step's result is not stored. A model answer you paid for is lost. A tool that ran keeps its effect, with no stored result. No further model call is made.
• Any other Step. A Step without a failure route, such as AwaitUserMessage, closes the Flow failed with that reason. The same happens when recording the failure above does not fit, which is common when turns are small. The stored AgentState then keeps its last value. AgentSessionClient.get_agent_snapshot in Python and readAgentSnapshot in TypeScript also read how the Flow closed. So they report failed and the reason for any session that stored a state. Use them, not the get_agent_snapshot RPC alone. A session that fails before it stores a state has a snapshot with no state. Read the reason from the Flow's result.
• A new message to a failed session. The closed Flow refuses it. AgentSessionClient.send_user_message in Python and sendAgentUserMessage in TypeScript raise AgentSessionFailedError with the reason. A direct RPC call gets FlowNotActiveOrNotFoundError.
• Queued messages of a failed session. They never run. They stay readable in the snapshot's queued_messages until retention deletes the session, so you can send them to a new session.
• A message that does not fit. send_user_message fails with RESOURCE_EXHAUSTED and the same text, and the message is not queued. No Step ran, so the session does not fail. It keeps waiting, and a shorter message may still fit. Only the caller sees this error.

A failed session cannot be recovered. Raising the limit applies only to new sessions. Plan sessions to stay well below 64 MiB.

History and retention

The Flow history keeps about the newest 50,000 events of a session, or 5,000 with dev and local. Read the conversation with get_agent_snapshot or list_agent_messages, not from Flow history. Time travel can target only a point inside the history window, or the beginning of the run. Operators can tune history.windowEvents and the per-run history window.

A session never completes by itself. Stop it with StopFlow when it is done. A stopped session, and its get_agent_snapshot RPC, stays available for the retention period. Then Dex purges it. To keep a transcript after that, copy it with list_agent_messages first.

Compaction

Compaction keeps the model input under the context window. To turn it on, set compaction in AgentSessionOptions, or in the agent definition of an agent on the built-in agent loop, with the model's context window in tokens. The runner must support compaction. The built-in agent loop does, and so do the adapters for the OpenAI Agents SDK, Pydantic AI, LangChain and LangGraph, Google ADK, the Vercel AI SDK, and Mastra.

Before a model call whose input would reach 85 percent of the window, the session runs a CompactContext Step instead. It makes one model call that folds the earlier messages into a summary. The next model call receives the summary and the newest messages. When the earlier messages do not fit in the window, the runner leaves the oldest of them out of the summary call and tells the model so. With the built-in agent loop, a long turn with many tool calls compacts in the middle of the turn.

A framework adapter compacts differently, because the framework builds the model input:

• Only between turns. The adapter compacts before the first model call of a turn, and never summarizes the running turn. A model call that continues a turn after its tool calls does not compact, so one turn's growth must fit in the rest of the window. Lower trigger_ratio (triggerRatio) when a turn can add more than about 15 percent of the window.
• An estimate that errs high. The adapter counts three bytes of each stored message as one token, which is more than a provider counts for English text. It adds the instructions and tool definitions it can see. Set reserved_input_tokens (reservedInputTokens) in the policy for what it cannot see, such as the system prompt of a compiled LangGraph graph, dynamic instructions, or MCP tools.
• A summary call without tools. The summary call receives the earlier turns as plain text. It uses the agent's model, unless you pass a summary model. LangGraph needs one, because a compiled graph does not expose its model.
• The record stays in Dex. A framework's own compaction, such as summarization middleware or provider-side compaction, changes only that framework's model input. Dex stores every message either way, and only a Dex summary lets retention delete messages. Use one of the two. With Google ADK, the summary also keeps the session state and the agent that replied last.
• Not the Claude Agent SDK. Claude Code compacts its own transcript inside the CLI. The adapter stores every transcript entry, including Claude Code's compaction entries, and takes no compaction policy. Its stored messages have no Dex summary, so retention never deletes them.

Turn on compaction only after every Worker of the Flow type runs a release that has the CompactContext Step. Upgrade the clients that read the session's snapshots first too. A client from an earlier release cannot read the compacting_context status that a snapshot shows during compaction.

Message retention

Compaction also bounds the session's storage. Retention deletes the oldest chunk once every message in it is summarized and at least retained_message_limit messages, 1,000 by default, are stored after it. The newest 1,000 messages and every message after the summary always stay, so the stored messages stop growing however long the session runs.

Set the limit, the AgentMessages size, and the chunk limits in AgentSessionOptions as retained_message_limit, message_window_size, message_chunk_size, and message_chunk_max_bytes. A limit of None (null in TypeScript) keeps every message. If you lower values.maxValueBytes below about 2 MiB, lower message_chunk_max_bytes with it.

A deleted message is gone from list_agent_messages and get_agent_snapshot. Paging stops at the oldest stored message. Each page's deleted_before_sequence says that retention deleted the messages before it. If the application needs the whole transcript, copy new messages out with list_agent_messages before retained_message_limit more messages are stored after them. A simple rule is to copy after each turn_completed event, starting after the last sequence you copied. When a turn can store more messages than the limit, raise the limit or set it to None.

Estimate the run budget

Estimate the run budget from the retained messages. With the default limit of 1,000, plan for about the limit times the size of a message, plus the messages after the summary, plus the largest framework run state and tool call records that one turn holds. Deleted chunks and earlier values of rewritten Attributes do not count.

For example, in a measured session each turn stored a 100-character question and a 600-character answer, about 2.9 KB. The session compacted at an 8,000-token window, with a limit of 100. At turns 200, 400, 600, and 800, the stored messages stayed between 162 KB and 185 KB, and the whole live state between 167 KB and 191 KB. Over its life the session stored 2.3 MB of messages.

To keep long sessions within the budget:

• Start a new session for new, unrelated work instead of growing one session forever.
• Keep tool outputs small. Return what the model needs, such as IDs, a summary, or the top results, instead of whole documents. Each output is stored in its tool call record, in the run state while the turn runs, and in the messages.
• Move large artifacts, such as files, images, and long documents, to your own blob storage. Have the tool return a reference to them.