Flow

View as MarkdownView as Markdown

A Flow is the top-level unit of work in Dex. A Flow type describes one kind of business process, such as an order, a subscription, or an approval. A Flow run is one instance of it, such as order 42.

Dex owns the Flow's lifecycle and state. Your code does not need a database or a job queue to keep the process going across crashes and restarts.

When to use a Flow

Use one Flow for each business identity whose process Dex should own. Put work that shares that identity and lifecycle into Steps of the same Flow. Steps can run in parallel inside one Flow.

Start a separate Flow when the work has its own owner and lifecycle. Two Flows coordinate by calling each other's RPCs. Use a SubFlow when a parent Step waits for a child Flow, for example to split complex logic into smaller Flows or to fan out work.

Guarantees

• A Flow ID has at most one active run at a time.
• What one Step method writes and publishes takes effect all at once, or not at all. See Step guarantees.
• With the default SYNC durability, a Step's result is stored durably before the next Step starts. It survives Worker crashes and Dex Server restarts.
• When no Worker is free, a Step waits in a durable queue. It runs when a Worker becomes available.

What a Flow contains

You define a Flow by implementing the Flow interface in your language. It declares three things:

• Steps: the start Step and every other Step the Flow can reach. Steps run the background work.
• Persistence schema: the Attributes and Channels that hold the Flow's durable state. Dex rejects reads and writes to names that are not declared.
• RPC handlers: methods that a client calls on a running Flow. See RPC.

The start Step's input type is also the Flow's start input type. This Flow has two Steps, one Attribute, one Channel, and one RPC.

example_flow.pyPython

123456789101112131415161718

status = Attribute("status", str)
notify = Channel("notify", None)


class ExampleFlow(Flow[int]):
    def __init__(self) -> None:
        self.finish = Finish()
        self.example = Example(self.finish)

    def get_steps(self) -> StepList[int]:
        return StepList.start_step(self.example).other_steps(self.finish)

    def get_persistence_schema(self) -> PersistenceSchema:
        return PersistenceSchema.of(status, notify)

    @rpc
    def describe(self, context: Context) -> RPCResult[str]:
        return RPCResult.of(status.get(context))

Example: examples/python/dex_examples/primitives/flow/example_flow.py

Identifiers

The word Flow can mean a Flow definition or a Flow run. These identifiers keep them apart.

Identifier	Who sets it	Meaning
FlowType	Application (via Flow implementation)	Stable type name Dex uses to route Worker calls and search results.
Flow ID	Application client on start	Business identity for one Flow run. The ID reuse policy controls whether another run may start with the same Flow ID.
Run ID	Dex Server	Identity of one run of the Flow ID. Time travel keeps it and creates a new run attempt. Address a Flow by its Flow ID; the Run ID is diagnostic.
StepType	Application (via Step implementation)	Stable name for a Step definition inside the Flow.
StepExecutionID	Dex Server	One execution of a Step type, formatted as StepType-Number.

Most SDKs use the language's type name as the FlowType unless you override it. Do not change these names while runs are open.

Lifecycle

A Client starts a Flow with a Flow ID and the start input. Dex then runs the start Step on a Worker. Each Step returns a decision that moves to the next Steps or closes the Flow. Several Steps can be active at the same time.

A Flow closes in one of these ways:

• a Step completes or fails it;
• its timeout fails or cancels it;
• a client stops it;
• a new start with the same Flow ID terminates it. This happens only when the new start uses an ID reuse policy that terminates the running Flow. A SubFlow started with ALWAYS_RESTART terminates an earlier child the same way.

Instead of failing or canceling the Flow, a timeout can run a handler that decides the outcome. While the Flow runs, clients read and change it through its RPCs.

Start options and configuration

Two sets of options shape a Flow run:

• StartFlowOptions apply once, at start. They set the Flow timeout, the ID reuse policy, the start request ID, and initial Attribute values.
• FlowConfig holds runtime settings, such as the default Step durability. You can update it while the Flow runs.

Start a Flow directly with a stable business Flow ID. Do not read or search first to check whether it already exists. To retry one start safely, do three things. Use an ID reuse policy that rejects a second run. Send the same request ID on every retry. Set the start option that returns the existing run instead of an already-started error. Then a retry of a start that already succeeded returns that run. With a policy that allows another run, a retry can start a second run instead.

Reference

The SDK reference covers:

• the Flow definition;
• StartFlowOptions and the Flow timeout and timeout handler;
• Flow ID, RequestID, and already-started handling;
• StartFlow best practices;
• FlowConfig and how to update it.