SubFlow

View as MarkdownView as Markdown

A SubFlow is a Flow run that a parent Flow starts from one of its Steps. The child is a normal Flow with its own Steps, Attributes, and Channels. The parent waits for it like any other condition in WaitFor.

When to use a SubFlow

SubFlow has two main uses:

1. Move complicated logic into a smaller Flow. This makes business logic easier to maintain.
2. Fan out work. One Flow run cannot do too many operations at once, but it can hand work to many SubFlows. The SubFlows run at the same time, so the parent reaches much higher concurrency.

Guarantees

Dex Server computes the SubFlow ID from the parent Flow ID, the Step execution, and the condition index. So a retried or repeated start finds the same child instead of starting a second one. By default, the parent keeps waiting for a running child, uses the result of a completed child, and restarts a child that ended abnormally.

The SubFlow ID joins its parts with a dash. So a Step whose Step type contains a dash cannot start SubFlows. Otherwise two parents could compute the same SubFlow ID and share one child. Dex fails such a WaitFor attempt without retry. Give every Step that starts SubFlows a Step type without a dash.

When you start the parent Flow ID again, the new run usually computes the same SubFlow IDs as the earlier run. Dex then applies the reuse policy. With the default policy, the new run gets the earlier run's child, running or completed, until Dex deletes that child. To start a fresh child in every run, use ALWAYS_RESTART. It terminates the earlier child if that child is still running. When both children must keep running, use a different parent Flow ID.

A child keeps running when its parent Step is canceled or its parent Flow closes. Stop it yourself, or set a Flow timeout on the child so it ends on its own.

Start and wait for a SubFlow

The target Flow must be registered on the Worker. Its start Step must accept the input passed to SubFlow. WaitFor starts the child. Execute reads its result after the child completes.

parent_flow.pyPython

123456789101112131415

class SubFlowParent(Step[int]):
    def __init__(self, target: Flow[int]) -> None:
        self.target = target
        self.options = SubFlowOptions(
            timeout=timedelta(hours=1),
            timeout_policy=FlowTimeoutPolicy.CANCEL,
        )

    def wait_for(self, context: Context, input: int) -> Wait:
        return Wait.until(SubFlow.run(self.target, input, self.options))

    def execute(self, context: Context, input: int) -> StepDecision:
        result = SubFlow.get_condition_results(context)
        output = result.single_output(int)
        return graceful_complete(f"{SubFlow.get_flow_id(context)}|{output}")

Example: examples/python/dex_examples/primitives/subflow/parent_flow.py

Wait for several SubFlows

Use allOf to make the parent wait for all SubFlows to complete.

Use anyOf when the parent does not need to wait for all of them. A SubFlow keeps running after WaitFor stops waiting. Execute can still get the SubFlow ID to stop it, or pass the ID to another Step to decide how to handle it. See SubFlow results for combining SubFlows with other conditions.

Reference

The SubFlow page of each SDK reference lists the API names in that language.

SubFlowOptions accept the same options as starting a Flow from a Client, plus a reuse policy. See SubFlowOptions and SubFlowReusePolicy for the options and the reuse policy for each state of an existing child.