Skip to main content
Super Durable
English
English简体中文
GitHubLog inSign up Cloud
Menu

Step

View as Markdown

A Step is one unit of background work in a Flow. It can wait durably for something to happen, then do its work, then choose what runs next. Dex runs Steps on your Workers and records each result.

A native struct or class implements the Step. Its type is the StepType. One Step type can run many times in a Flow, even in parallel. Each run is a StepExecution, identified by a StepExecutionID such as Charge-2.

Step input​

Each Step execution has its own typed input. Dex stores it with the execution. The input tells executions of the same Step type apart. The start Step's input is the Flow's start input.

WaitFor and Execute​

A Step has one or two methods.

  • WaitFor is optional. It returns a durable Wait. Dex stores the wait, checks its conditions for you, and calls Execute once they are met.
  • Execute is required. It does the work and returns a StepDecision.

If a Step has no WaitFor, Execute runs immediately.

This Step waits for an approval message, then moves to a second Step that completes the Flow.

step_flow.pyPython
approval = Channel("Approval", str)


class StepSecond(Step[int]):
    def execute(self, context: Context, input: int) -> StepDecision:
        return graceful_complete(input + 1)


class Example(Step[int]):
    def __init__(self, second: StepSecond) -> None:
        self.second = second

    def wait_for(self, context: Context, input: int) -> Wait:
        return Wait.until(approval.for_one())

    def execute(self, context: Context, input: int) -> StepDecision:
        return go_to(StepSecond, input + 1)

Example: examples/python/dex_examples/primitives/step/step_flow.py

Waiting​

A wait does not hold a Worker or a thread. Dex keeps it in the Flow's state and wakes the Step when a condition is met. A Step can wait on three kinds of condition:

Combine conditions with AnyOf or AllOf. AnyCombinationOf accepts any one of several groups of conditions. After the wait, Execute reads what satisfied it, such as the Channel messages it consumed. See Waits.

To wait for parallel branches, use a Join. It is not another kind of condition: it waits on a ChannelMap that the branches report to. See Await Parallel Steps.

StepDecision​

Execute returns a StepDecision. It can go to one next Step or to several Steps at once. This is how you build branching, loops, and parallel work.

A decision can also close the Flow or end a branch:

  • GracefulComplete asks Dex to complete the Flow once no other Step is active.
  • ForceComplete completes the Flow immediately.
  • ForceFail fails the Flow immediately.
  • DeadEnd ends this branch without closing the Flow.

Completing decisions can return the Flow output, which a client reads as the Flow result. When several completing decisions return an output, the result keeps each one. Only a Flow that completes keeps these outputs. If the Flow does not complete because it fails, is canceled, is terminated, or times out, its result has no outputs.

A decision can also cancel other active Steps. See the Parallel Steps and Parallel SubFlows patterns.

StepOptions​

A Step returns StepOptions to control how its methods run. WaitFor and Execute each have their own settings.

  • Timeout limits one attempt of a method.
  • Retry policy runs the same method again after a failed attempt.
  • Failure policy decides what happens when retries run out. By default the Flow fails. Execute can go to a recovery Step instead. WaitFor can proceed to Execute. See WaitFor failure policy, Execute failure policy, and Last failure info.
  • Durability chooses when Dex stores the result. SYNC, the default, stores it before the next work starts. ASYNC is faster, but a server failure can make the method run again.
  • Heartbeat lets a long method report progress and save a checkpoint.
  • State loading and Attribute locks choose which state the method loads and which Attributes it locks.

A StepDecision can override the options for one Step execution.

Guarantees​

Dex saves each WaitFor or Execute call on its own. It saves the call's Attribute writes and Channel publications together with its result, and only when the method succeeds. Calls to external systems are not part of this save.

A Step method's changes take effect all at once. They are not protected from other Steps and RPCs, though. A method reads the Flow state as it was when Dex handed the method to a Worker. Other Steps and RPCs can change that state before the method's changes take effect. Dex does not check that the values the method read are still current. So two Steps or RPCs that read and then write the same Attribute can overwrite each other's update. To prevent this, have both declare a lock on that Attribute. Dex then runs them one at a time. See Attribute locks.

A Step method can run more than once. A retry runs it again after a failure. Dex cannot always tell whether a failed attempt ran the work. Make each external effect idempotent. Key it by the Flow ID and the StepExecutionID, or by your own business key. See Failure and redelivery for the cases where Dex cannot tell.

When retries run out, the recovery Step learns whether the work certainly did not run, or may have run.

Canceling a Step discards everything its current method has not saved: its Attribute writes, its Channel publications, and its decision. What its WaitFor already saved stays. A decision or an RPC never cancels the Steps that it starts itself. Channel guarantees say what happens to the messages a canceled Step consumed.

Dex does not stop a canceled method's code. The method should check for cancellation and return. Until it does, its code keeps running. Its calls to external systems still take effect, even after the Flow has closed.

GracefulComplete ends its own branch and asks Dex to complete the Flow. Dex begins to complete the Flow as soon as no Step is active. Until then, other Steps and RPCs can still start new Steps, and those Steps delay completion too. A later branch that ends with DeadEnd does not withdraw the request. After the last Step ends, the Flow can stay RUNNING briefly while Dex finishes pending Attribute Sync writes and transactional RPCs that already started.

Without a pending GracefulComplete, a Flow with no active Step keeps running. An RPC can still start new Steps in it. End the last branch with a completing decision. If a Flow must not wait forever, set a Flow timeout.

Dex starts ready Steps roughly in the order they became ready, but not strictly. Do not rely on the order in which parallel Steps start or finish. Use a Channel or an Attribute lock when the order matters.

Reference​

The SDK reference covers: