Deploy agents
View as MarkdownThis page covers running agent framework sessions in production. One Flow execution is one agent session. Every model call is a RunModel Step, and every tool call is its own Step. Server limits on this page are server-zd defaults. See Server reference and Application for the rest of the setup.
Install the packages
Install an adapter with its framework. Python adapters are on PyPI. Each one brings in the dex-agents core and the Dex SDK it supports. They need Python 3.11 or later.
pip install dex-agents-pydantic-ai "pydantic-ai-slim[openai]"
pip install dex-agents-langgraph
pip install dex-agents-openai-agents
pip install dex-agents-google-adk
pip install dex-agents-claude-agent-sdk
TypeScript adapters are on npm. The core and the Dex SDK are peer packages, so the application owns exactly one copy of each. They need Node 22 or 24.
npm i @superdurable/dex-agents-vercel-ai @superdurable/dex-agents @superdurable/dex ai zod
npm i @superdurable/dex-agents-mastra @superdurable/dex-agents @superdurable/dex @mastra/core zod
npm i @superdurable/dex-agents-openai-agents @superdurable/dex-agents @superdurable/dex @openai/agents zod
npm i @superdurable/dex-agents-claude-agent-sdk @superdurable/dex-agents @superdurable/dex zod
To write a runner of your own, install only the core:
pip install dex-agents
npm i @superdurable/dex-agents @superdurable/dex@1.30.0
All agents packages share one version line. A Python adapter requires the core of the same version. An npm adapter peer-depends on it. Each release pins the Dex SDK it supports and the framework range it was tested with; see the compatibility table. The packages are before 1.0, so a minor release may change their API. Pin exact versions in your lockfile and read the changelog before an upgrade.
Build the same session Flow in every process that starts sessions or calls their RPCs.
Stream backend
Live model text, reasoning summaries, tool progress, and approval events go to the AgentEvents Stream. streamStore.backend defaults to memory for both dexcli dev and a production server, so live events need no extra server configuration. Do not set the backend to disabled for agents. With disabled, a Step write is dropped without failing the Step, and Client writes and Stream reads fail with FAILED_PRECONDITION. The default is equivalent to:
streamStore:
backend: memory
The memory backend keeps Streams in the memory of the active server-zd process. A server-zd cluster has one active process, so a single node needs nothing more. A standby holds no Streams. A process starts with empty Streams whenever it becomes active: after a restart, a standby takeover, or a step down. A standby answers Stream calls with UNAVAILABLE, and clients reconnect to the active process.
Losing Stream data never loses a session. The conversation, tool calls, and pending approvals live in Attributes. A client that misses events reloads the session with the get_agent_snapshot RPC and keeps reading the Stream from there. After turn_completed or turn_failed, a client replaces the turn's live text with the stored messages. Never treat the Stream as the record of what happened.
Size AgentEvents
AgentEvents has one capacity for each Flow type and Stream name. Every session of the agent shares it; it is not a per-session budget. The default is 10 MiB. Set it with event_stream_capacity_bytes in the Python AgentSessionOptions or eventStreamCapacityBytes in the TypeScript createAgentSessionFlow options.
The server charges each message its serialized value, Flow ID, and source, plus 512 bytes. When usage reaches 90 percent of the capacity, a background trimmer removes the oldest messages of all sessions until usage is at 80 percent. If trimming falls behind and usage reaches the capacity, writes are rejected. A Step drops a rejected write without failing. Size the capacity so it holds the newest turn of every session that a client watches:
bytes per turn = messages per turn × (average message bytes + 512)
capacity ≥ concurrent sessions × bytes per turn ÷ 0.8
Say a turn writes 200 messages of about 300 bytes each, counting the Flow ID and source. That is about 162 KB per turn. With 500 sessions streaming at once, live events take about 81 MB. Dividing by 0.8 gives about 102 MB, so a capacity of 128 MiB keeps the newest turn of every session. The default 10 MiB keeps about 50 such turns.
The fixed 512 bytes dominate small text deltas. The Python core writes one Stream message for each text delta that a runner writes. The TypeScript core merges consecutive deltas for up to 100 ms or 1,024 characters. Expect more messages per turn from Python adapters.
One Stream message is at most 100 KiB, set by streamStore.maxMessageBytes. A larger write is dropped. The memory backend keeps every Stream in server memory, so plan server memory for the sum of the capacities of every agent Flow type and other Stream.
Size Workers
Each running model call, tool call, and RPC handler holds one Worker task slot. A session that waits for a user message, an approval, or tool results holds none; the wait lives in Dex Server. Size each Worker pool for the turns that call the model at the same time, plus the tool calls that run at the same time, plus the RPC reserve.
The tool calls of one model response run as parallel Steps. Give a Worker at least enough slots for the largest batch of tool calls that a model makes. The default task limit is the CPU count. Model calls mostly wait on the network, so agent Workers usually need a higher maxConcurrentTasks. See capacity and shutdown for the option names in each SDK.
Keep the RPC reserve. A UI calls get_agent_snapshot, send_user_message, and resolve_interrupt as RPCs. Without a reserve, they wait behind model calls that fill the Worker. When you do not set it, the reserve is one slot if the task limit is at least two, otherwise zero. Steps never use reserved slots. A growing Step task queue means Steps need more capacity. See Step task queue for its metrics.
server-zd allows 256 streams for each app Worker ID by default (matching.maxConcurrentTasksCap). A Worker opens one stream per task slot, so its task limit must fit within it. To grow past it, run more Worker processes.
Keep headroom. When no Worker has a free slot, a Step waits in the queue. The wait uses no attempt and counts toward neither the method timeout nor the heartbeat timeout. A turn or tool call that waits for a slot takes longer. See Tool call limits.
Long model calls and timeouts
The packages set these Step options. AgentSessionOptions does not change them.
| Step | Method timeout | Heartbeat timeout | Attempts | Failure route |
|---|---|---|---|---|
| RunModel | 15 minutes | 2 minutes | 3 within 30 minutes | RecordTurnFailure |
| CompactContext | 15 minutes | 2 minutes | 3 within 30 minutes | RecordTurnFailure |
| CallTool | 2 hours, the Dex default | 1 minute, the Dex default | 3 | RecordToolFailure |
| CallSingleAttemptTool | 2 hours, the Dex default | 1 minute, the Dex default | 1 | RecordToolOutcomeUnknown |
RunModel, CompactContext, and both tool Steps send a heartbeat every 20 seconds while the runner works. A model or tool call that is slow but silent therefore stays within its heartbeat timeout. The method timeout still ends a hung call. In Python, keep runner and tool code non-blocking: a blocking call on the event loop also stops the background heartbeat.
One model call must finish within 15 minutes. Set the provider or framework request timeout below that, so a slow call fails with the provider's error and later attempts still fit in the 30-minute retry window. A Claude Agent SDK session runs a whole turn segment in one RunModel, so that segment must finish within 15 minutes. See Heartbeat for how Dex detects a lost attempt.
Separate Worker pools
Dex routes work by Flow type. Every Worker that registers an agent's Flow type can receive its model calls, tool calls, and RPCs. The packages register every session Step together, so one agent's model calls and tool calls share a Worker pool. Split along these lines when one kind of work crowds out another:
- Give each agent its own Flow type and its own Worker deployment, so a busy agent cannot take the slots of another.
- Run heavy or special tool work, such as a browser, a sandbox, or a GPU job, in a service with its own hosts and scaling. The tool calls that service with the call's idempotency key. The session Worker then only waits on the network, and the service scales on its own.
Limits for long sessions
A session can stay open for months. These server-zd limits decide how far it grows.
| Key | Default | What it bounds |
|---|---|---|
| limits.maxFlowStateBytes | 64 MiB | All storage of one session: current state, Attribute and Channel values, and the superseded values kept for time travel |
| values.maxValueBytes | 16 MiB | The largest single value, such as one message, one tool call record, or one framework run state |
| history.windowSemanticEvents | About 10,000 events | The Flow history kept for each run. Older events are pruned. |
| retention.closedFlowDuration | 7 days | How long a stopped session stays readable |
Each run records these values when it starts. A new value applies only to sessions that start after the server restarts with it.
The packages store state this way:
- AgentMessages keeps the newest messages, one AttributeMap instance per message. Storing a message writes only that message. When more than 64 messages are in it, the oldest move into AgentMessageChunks instances that are never rewritten. A chunk holds at most 32 messages and 1 MiB, so it stays well under values.maxValueBytes. A larger message skips the window and goes straight into a chunk of its own when it is stored, so a message too large for values.maxValueBytes fails its turn right away, as it did before. AgentMessageIndex records where the messages are, and changes only when chunks are written or deleted. Each message is written twice, once in the window and once in its chunk, and the window copy counts as a superseded value while history keeps it. With compaction, RunModel loads only the messages after the summary. Without compaction, every model call loads and sends the whole history, and nothing is deleted.
- FrameworkRunState holds the framework's state while a turn waits for tool results. RunModel rewrites it after every model call that asks for tools. It holds the turn's work so far, tool outputs included. The OpenAI Agents SDK state also holds the turn's input, and the Vercel AI SDK state holds each tool output twice. Each earlier value stays as a superseded value and counts toward the run budget while history keeps it.
- AgentToolCalls records are deleted when the turn ends. A deleted value also becomes a superseded value, so it still counts toward the run budget until the history window prunes it.
The server checks the run budget for Step heartbeats and Step completions, among other writes, and only when the write adds storage. Once a session is over the budget, such a Step fails without retry. A model call or tool call then fails its turn. A Step without a failure route fails the whole session instead, when its own write crosses the budget. AwaitUserMessage, which stores the next user message, is such a Step. Transactional RPC writes, such as send_user_message, are not checked. Plan sessions to stay well below 64 MiB.
The Flow history keeps about the newest 10,000 events of a session. Read the conversation with get_agent_snapshot or list_agent_messages, not from Flow history. Time travel can target only a point inside the history window. Time travel to the beginning is always available.
A session never completes by itself. Stop it with StopFlow when it is done. A stopped session, and its get_agent_snapshot RPC, stays available for the retention period. Then Dex purges it. Copy a transcript that must outlive the session with list_agent_messages before then.
Compaction and the storage bound
Compaction, from the agents 0.2.0 packages, keeps the model input under the context window. Set compaction in AgentSessionOptions, or in the agent definition of a Stepwise agent, with the model's context window in tokens. The runner must support compaction; Stepwise does, and the framework adapters do not yet. Before a model call whose input would reach 85 percent of the window, the session runs a CompactContext Step instead. It makes one model call that folds the earlier messages into a summary, and the next model call receives the summary and the newest messages. When the earlier messages do not fit in the window, Stepwise leaves the oldest of them out of the summary call and tells the model so. A long turn with many tool calls compacts in the middle of the turn.
Enable compaction only after every Worker of the Flow type runs a release that has the CompactContext Step. The clients that read the session's snapshots need agents 0.2.0 too: a 0.1.0 client cannot read the compacting_context status that a snapshot shows during compaction.
Compaction also bounds the session's storage. Retention deletes the oldest chunk once every message in it is summarized and at least retained_message_limit messages, 1,000 by default, are stored after it. The newest 1,000 messages and every message after the summary always stay, so the stored messages stop growing however long the session runs. Set the limit, the window size, and the chunk limits in AgentSessionOptions as retained_message_limit, message_window_size, message_chunk_size, and message_chunk_max_bytes. A limit of None (null in TypeScript) keeps every message. If you lower values.maxValueBytes below about 2 MiB, lower message_chunk_max_bytes with it.
A deleted message is gone from list_agent_messages and get_agent_snapshot. Paging stops at the oldest stored message, and each page's deleted_before_sequence says that retention deleted the messages before it. If the application needs the whole transcript, copy new messages out with list_agent_messages before retained_message_limit more messages are stored after them. A simple rule is to copy after each turn_completed event, starting after the last sequence you copied. When a turn can store more messages than the limit, raise the limit or set it to None.
Estimate the run budget from the retained messages. In a measured session, each turn stored a 100-character question and a 600-character answer, about 2.9 KB, and the session compacted at an 8,000-token window. With a limit of 100, the stored messages stayed between 162 KB and 185 KB, and the whole live state of the session between 167 KB and 191 KB, at turns 200, 400, 600, and 800, while the session stored 2.3 MB of messages over its life. With the default limit of 1,000, plan for about the limit times the size of a message, plus the messages after the summary, plus the superseded values that the history window keeps. A deleted chunk counts as a superseded value until the history window prunes it.
To keep long sessions within the budget:
- Start a new session for new, unrelated work instead of growing one session forever.
- Keep tool outputs small. Return what the model needs, such as IDs, a summary, or the top results, instead of whole documents. Each output is stored in its tool call record, in the run state while the turn runs, and in the messages.
- Move large artifacts, such as files, images, and long documents, to your own blob storage. Have the tool return a reference to them.
Approvals and deadlines
A tool call that needs approval waits in its own AwaitToolApproval Step. The wait holds no Worker slot. It survives Worker restarts and deploys.
Without a deadline, an approval waits until someone answers, and the turn waits with it. New user messages queue until the turn ends. Set interrupt_deadline in the Python AgentSessionOptions or interruptDeadlineMs in TypeScript. A call still pending at the deadline is rejected with the message "approval deadline passed", and the model sees the rejection.
The session computes the deadline when the model asks for the call, from the Worker clock. Keep Worker clocks synchronized. A changed deadline option applies only to approvals raised after the change. Dex Timers take whole seconds, so the deadline Timer is rounded up and can fire up to a second late. resolve_interrupt returns false from the deadline on, so a late answer never wins.
resolve_interrupt can fail with RpcLockConflictError while a Step holds the same lock. A retry is safe. The Python AgentSessionClient retries for up to 5 seconds.
Worker kills and redeploys
A crash repeats at most the one call that was running.
| Running when the Worker dies | What happens |
|---|---|
| RunModel | That model call runs again, up to 3 attempts within 30 minutes. Then the turn fails. If tools already ran, a note in the history tells the next turn's model not to repeat them. |
| CompactContext | Only the summary call runs again, up to 3 attempts within 30 minutes. The session stores one summary. If every attempt fails, the turn fails, and the next user message compacts again. |
| CallTool, for a read or an idempotent write | The call runs again with the same idempotency key. After the last attempt, a read records failed and a write records outcome_unknown. |
| CallSingleAttemptTool | The call records outcome_unknown and does not run again. The model sees that the outcome is unknown. |
| A wait for a user message, an approval, or tool results | Nothing runs again. The wait lives in Dex Server. |
| A tool the framework cannot defer | The tool runs inside RunModel and repeats when RunModel runs again. It is at-least-once. |
| A Claude Agent SDK turn segment | Deferred custom tools run in their own CallTool or CallSingleAttemptTool Steps, as above. Model calls, inline read tools, and CLI tools such as Bash, Write, Edit, and Task run inside RunModel and repeat when it runs again. They are at-least-once. |
A single-attempt tool can still run twice in narrow cases, such as an ASYNC rerun after a node crash. See Tool call limits.
A graceful Worker stop waits for active handlers for the drain timeout, 30 seconds by default. Then it cancels them. A cancelled model call runs again, and a cancelled CallTool retries. A single-attempt tool that stops on cancellation records outcome_unknown. Set the drain timeout above your usual model and tool call times, and set the container's termination grace period above the drain timeout. Wait for the new Workers to register before you stop the old ones.
Tool call limits
A single-attempt tool runs in a CallSingleAttemptTool Step. That Step serves non_idempotent_write and unclassified tools. It should run the tool at most once, and record outcome_unknown when the tool may have run. When the tool certainly did not run, the call records failed instead, so the model can call the tool again. Two cases break the at-most-once promise. Plan for both. They follow from how Dex hands out any Step attempt. See Failure and redelivery.
A single-attempt tool can run twice
Dex starts an attempt when it hands the attempt to a Worker. It never hands the same attempt number to another Worker. If the Worker dies or loses its connection to Dex while the tool runs, the attempt fails. A single-attempt tool has no retry left, so the call records outcome_unknown. The earlier run may have finished its effect, so the session does not run the tool again.
Only two narrow cases can still run a single-attempt tool twice:
- The Step uses ASYNC durability, and a Dex Server node crashes before Dex wrote the attempt start to object storage. Dex rolls the start back and runs the same attempt number again, possibly on a different Worker. The two runs can overlap.
- Cancellation is cooperative. When Dex cancels an attempt, an interrupted run may keep running while the failure route starts.
To make a second run harmless, pass the call's idempotency key to the service that the tool writes to. Every tool call receives one: the session's Flow ID, the turn ID, and the tool call ID, joined by slashes, such as session-42/turn-3/call-7. The key is the same for every attempt of the call. When the service deduplicates on it, classify the tool as idempotent_write, so it also retries after a failure.
A call that never ran can record outcome_unknown
When a tool Step fails, its failure route reads attempt_progress from the recovery error. See Failure and redelivery. CallTool and CallSingleAttemptTool use it the same way:
| attempt_progress | What the call records |
|---|---|
| NOT_STARTED | No attempt reached a Worker, so the tool did not run. The call records failed. Its error starts with "the tool did not run, so it is safe to call again". The model sees a failed tool call and can call the tool again. A call whose tool already asked the user a question ran before, so it follows the next row. |
| NOT_COMPLETED, or UNSPECIFIED when the server reported no progress | The tool may have run. A read records failed. Any other call records outcome_unknown, and the session does not run its tool again. |
To recognize such a call in get_agent_snapshot or your own code, compare the start of its error with TOOL_CALL_NOT_RUN_ERROR_PREFIX in Python, toolCallNotRunErrorPrefix in TypeScript, or ToolCallNotRunErrorPrefix in Go.
NOT_COMPLETED is conservative, so a call that never ran can still record outcome_unknown. For example, the server crashed after it recorded the attempt start but before the task reached the Worker. Treat that outcome as unknown.
When no Worker accepts a tool Step, the call waits in the queue. This happens when every Worker of the Flow type is at its task limit, or when none is running. The call does not fail and does not use an attempt. It runs when a Worker takes it. If the Flow times out or stops first, the call never runs.
Keep tool Steps dispatchable, so calls do not wait long:
- Size each Worker pool as in Size Workers, with headroom for the largest batch of tool calls.
- Keep at least one Worker of every agent Flow type running. When a pool scales to zero, a tool call waits in the queue until a Worker starts.
- During a deploy, start the new Workers and wait for them to register before you stop the old ones.
Reconcile outcome_unknown
outcome_unknown means the effect may or may not have happened. The session never retries such a call, and the model sees that the outcome is unknown. Reconcile it outside the session:
- Find the call. While the turn runs, get_agent_snapshot lists it under tool_calls, with the failure in error. The AgentEvents Stream has a tool_call_finished event with the status outcome_unknown. After the turn, its tool result in the message history says that the outcome is unknown.
- Check the target system for the effect, by the call's idempotency key or by your own business key.
- A call that Dex knows never ran records failed, not outcome_unknown. If the Flow's history still shows that the tool Step's attempt never reached a Worker, such as for a call that started before a server upgrade, the call did not run.
- Tell the session the real outcome with send_user_message, so the next turn's model works from it.
Upgrade agent code while sessions wait
A waiting session runs the code of the Worker that takes its next Step. Nothing replays the agent loop, so a deploy takes effect at the next model call or tool call, even in the middle of a turn. During a rolling deploy, two Steps of one session can run on different versions.
- Keep each agent's Flow type stable. Running sessions resume by it.
- Keep a tool available while a running turn or a pending approval still refers to it.
- A recorded tool call keeps the effect and approval rule that applied when the model asked for it. Changed tool policies apply to later calls.
- A changed prompt or model takes effect at the next model call. A changed tool takes effect at its next call.
- Between the Steps of a turn, FrameworkRunState holds state that the framework serialized. The framework version on the new Worker must restore it. Stay within the framework range that the adapter pins, and upgrade the adapter and the framework together.
- Upgrading from agents 0.1.0: stop every 0.1.0 Worker of the Flow type before any 0.2.0 Worker starts. Do not run them side by side. A 0.1.0 session keeps every message in AgentMessages. Once a 0.2.0 Worker stores a message in it, it moves the older messages into AgentMessageChunks, a few chunks per Step. If a 0.1.0 Worker then runs a Step of that session, the turn fails, because it cannot read the moved messages.
See Versioning Flow code for the general rules.
Session settings during a rollout
Start sessions with session settings, and call update_session_settings, only after every Worker of the Flow type runs agents 0.2.0. A 0.1.0 Worker drops the settings of a session that it starts, and it neither writes nor reads the copy of the settings that a turn keeps. A turn that a 0.1.0 Worker starts runs without settings, so it can use the Worker's default model for some of its model calls.
Credentials
Keep model API keys and tool credentials in the Worker's environment or in a secret store that the Worker reads when a call runs. Each framework reads its model key from the environment, such as OPENAI_API_KEY, GOOGLE_API_KEY, or ANTHROPIC_API_KEY. The adapters add no credential handling.
Never pass a credential in the session's start input, a user message, a tool argument, a tool output, or an Attribute. Session settings are part of the start input and of the session's state, and get_agent_snapshot and the browser snapshot JSON return them, so they hold only references, such as a model or a credential name, that the Worker resolves. Dex stores them in Flow history and state. Dex Web shows them, history keeps superseded values for time travel, and a stopped session stays readable for the retention period. Any authorized Client can read AgentEvents.
For a credential that belongs to one user, let the tool look it up at call time from a secret store, with a key that is not the secret itself. When a Flow of your own must hold a short-lived secret, store it in a hard-delete Attribute and delete it when the Flow no longer needs it.
Workers and Clients authenticate to Dex Server with bearer credentials. See authentication.