---
title: "Run Agents in Production"
description: "The checklist before you run agent sessions in production: the server you run, authentication, retention, session and spend limits, upgrades, tracing, approval deadlines, and Worker sizing."
canonical_url: "https://docs.superdurable.io/agent-frameworks/production/"
source: "docs/content/agent-frameworks/production.mdx"
locale: "en"
language: "python"
---
> Index of every page for AI agents: https://docs.superdurable.io/llms.txt
# Run Agents in Production
Code on this page is in Python. Other languages: [TypeScript](https://docs.superdurable.io/agent-frameworks/production.typescript.md)
Agent sessions run on the same Dex Server as every other Flow. Check each item before you put sessions in front of users. Each line links to the page with the details.
| Item | What to know | Details |
| --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Dex Server | Dex Server runs in Dex Cloud or on your own infrastructure with object storage. One active node serves a cluster. | [Choose How to Run Dex](https://docs.superdurable.io/components/choose-how-to-run.md), [Run Dex in Production](https://docs.superdurable.io/components/production.md) |
| Authentication | Public API authentication is off by default. Turn it on before the server is reachable from other machines. | [Authentication](https://docs.superdurable.io/reference/server/authentication.md) |
| Retention | A closed session stays readable for 7 days by default, then the server may delete it. Keep transcripts you need longer in your own store. | [Retention](https://docs.superdurable.io/reference/server/configuration.md#retention) |
| Session size | A session ends when its active state reaches the state limit, 64 MiB by default. Message retention, and compaction where the harness supports it, keep it below that. | [Session limits](https://docs.superdurable.io/reference/agents/session-limits.md) |
| Model spend | Check spend per account or API key around each model call; Dex leaves spend limits to your code. | [Spend limits](https://docs.superdurable.io/reference/agents/spend-limits.md) |
| Credentials | Keep model API keys and tool credentials in the Worker's environment or a secret store, never in session input or messages. | [Credentials](https://docs.superdurable.io/reference/agents/credentials.md) |
| Approval deadlines | Set a deadline when a call must not wait forever. An approval with no deadline waits until someone answers, and the turn waits with it. | [Approval deadlines](https://docs.superdurable.io/reference/agents/approvals.md#approval-deadlines) |
| Upgrades | A deploy takes effect at the next model call or tool call. Some releases cannot run next to the one before: stop every older Worker before a newer one starts. Such a stop turns a running single-attempt tool into `outcome_unknown` and makes RPCs fail with **Unavailable** until Workers return; stop Workers gracefully first. | [Upgrades](https://docs.superdurable.io/reference/agents/upgrades.md#in-flight-calls) |
| Workers | Each running model call, tool call, and RPC holds one Worker task slot. Size Workers for parallel tool calls. | [Workers and timeouts](https://docs.superdurable.io/reference/agents/workers.md) |
| Streaming | The AgentEvents Stream carries live text to your UI. Size it for your traffic. | [AgentEvents Stream](https://docs.superdurable.io/reference/agents/event-stream.md) |
| Tracing | Sessions can emit OpenTelemetry spans for each model call and tool call. Content capture is off by default. | [Tracing](https://docs.superdurable.io/reference/agents/tracing.md) |
| Deployment shape | Web replicas hold a Client. Worker processes run the session Flow. | [Deployment shape](https://docs.superdurable.io/agent-frameworks/production.md#deployment-shape) |
| Calls during a deploy | `send_user_message` and `resolve_interrupt` are RPCs. They need a running Worker, and a retried `send_user_message` can queue a message twice. | [RPCs during a deploy](https://docs.superdurable.io/agent-frameworks/production.md#rpcs-during-a-deploy) |
| Unknown outcomes | A write that a crash interrupted records `outcome_unknown` and does not run again. Decide who reconciles it. | [Reconcile outcome_unknown](https://docs.superdurable.io/agent-frameworks/tool-calls.md#reconcile-outcome-unknown) |
## Deployment shape
A production setup has three kinds of process:
> **Diagram**: Users reach web app replicas, which hold a Client and send RPCs and reads to Dex Server. Worker processes pull tasks from Dex Server over outbound connections and run the session Flow: model calls, tool calls, and approvals. Dex Server keeps session state in object storage.
- **Web app replicas** hold a Client. They start sessions, send messages, and
stream replies. They need the session Flow's definition to encode calls, but
never run a Step.
- **Worker processes** run the session Flow: every model call, tool call, and
approval wait. For a small app, the web process can host the Worker too, as
the Agent Quick Start program does.
- **Dex Server** keeps every session. Workers connect out to it, so they need no
inbound port.
During a rolling deploy, each Step runs the code of the Worker that takes it.
A turn in progress finishes its next model call or tool call on the new code.
A Worker that lacks a Step type declines it, and another Worker takes it. A
few releases cannot run next to the one before, and need every older Worker
stopped first; the [Upgrades release notes](https://docs.superdurable.io/reference/agents/upgrades.md#release-notes)
name them.
## RPCs during a deploy
Your application talks to a session through RPCs, such as `send_user_message`
and `resolve_interrupt`. An RPC runs on a Worker. It never waits in a queue:
if no Worker takes it within 10 seconds, the call fails with **Unavailable**.
During a deploy that stops every Worker, these calls fail until a new Worker
connects. The session itself loses nothing.
- **Retry on Unavailable.** Retry with backoff until a Worker is back. A failed
call with **Unavailable** changed nothing.
- **resolve_interrupt is safe to retry.** An approval that already went through
returns **False** the second time, and nothing runs again.
- **send_user_message can send twice.** Dex runs a repeated RPC again. If the
response is lost, a retry can queue the same message twice. Before you resend,
read the session with `get_agent_snapshot` and look for the message in
**messages** and `queued_messages`. If it is queued twice, remove the copy
with `delete_queued_message`.
This helper does that. Each tab shows the whole module. It retries a send that may not have arrived, checks the
session before each retry, and deletes an extra queued copy. It tells messages
apart by their text, so call it once for each message a user sends.
**Python**
```python
"""Sends a user message to a session once, even when a send must be retried."""
from __future__ import annotations
import asyncio
from typing import Protocol
import grpc # type: ignore[import-untyped]
from dex import DexServiceError
from dex_agents import AgentSnapshot, MessageRole
# A lost response leaves a send's outcome unknown; these codes mean "check first".
UNCERTAIN_CODES = (grpc.StatusCode.UNAVAILABLE, grpc.StatusCode.DEADLINE_EXCEEDED)
class MessageSessions(Protocol):
async def get_agent_snapshot(self, session_id: str) -> AgentSnapshot: ...
async def send_user_message(self, session_id: str, text: str) -> bool: ...
async def delete_queued_message(self, session_id: str, message_id: str) -> bool: ...
async def send_user_message_once(
sessions: MessageSessions,
session_id: str,
text: str,
*,
attempts: int = 5,
retry_delay_seconds: float = 1.0,
) -> None:
"""Queues one user message, retrying a failed send without queuing it twice.
Before each retry it reads the session, and stops when a new copy of the
text is there. At the end it deletes any extra queued copy. It tells
messages apart by their text, so call it once for each message a user sends.
Raises:
DexServiceError: When no attempt reached a Worker, or on any other error.
"""
copies_before = _copies(await sessions.get_agent_snapshot(session_id), text)
for attempt in range(1, attempts + 1):
try:
await sessions.send_user_message(session_id, text)
break
except DexServiceError as error:
if error.code not in UNCERTAIN_CODES or attempt == attempts:
raise
await asyncio.sleep(retry_delay_seconds)
if _copies(await sessions.get_agent_snapshot(session_id), text) > copies_before:
break
await _delete_extra_copies(sessions, session_id, text, copies_before + 1)
async def _delete_extra_copies(
sessions: MessageSessions, session_id: str, text: str, wanted: int
) -> None:
snapshot = await sessions.get_agent_snapshot(session_id)
extra = _copies(snapshot, text) - wanted
queued = [item for item in snapshot.queued_messages if item.message.text == text]
if extra <= 0:
return
for item in reversed(queued[max(0, len(queued) - extra) :]):
await sessions.delete_queued_message(session_id, item.message_id)
def _copies(snapshot: AgentSnapshot, text: str) -> int:
delivered = sum(
1 for message in snapshot.messages if message.role == MessageRole.USER and message.text == text
)
queued = sum(1 for item in snapshot.queued_messages if item.message.text == text)
return delivered + queued
```
Example: [examples/python/agent-harnesses/agent_harness_examples/quick_start/safe_send.py](https://github.com/superdurable/dex/tree/main/examples/python/agent-harnesses/agent_harness_examples/quick_start/safe_send.py)
[Limits](https://docs.superdurable.io/intro/limits.md#requests-that-can-repeat)
lists the other requests that can repeat.
## Next
- [Limits](https://docs.superdurable.io/intro/limits.md): the defaults and known gaps of Dex as a whole.
- [Sessions](https://docs.superdurable.io/agent-frameworks/sessions.md#latency-and-cost): what each Step adds to a turn.