跳到主要内容

Drain External Channels

处理外部发布到 Channel 的消息;队列为空时完成这条短生命周期的 Flow。

问题与做法

当一条 Flow 要处理 Worker 外部发来的消息,并且消息可能突发到来时,使用这个模式。Client 调用 RPC,由 RPC 把每条消息追加到 Channel。关键要求是在 Channel 为空时完成 Flow,从而尽可能让 Flow 保持短生命周期。有新消息到来时,Flow 会重新启动。

这个 decision 会在记录下一步动作时原子地检查 Channel。Channel 为空时完成 Flow;Channel 中还有排队消息,或有消息已从该 Channel 消费并提交给尚未完成 Execute 处理的 Step 时,Dex 会走 fallback movement 回到 ProcessMessage。前一次 execution 完成后,下一个请求会使用相同的 Flow ID 启动新的 Flow execution,并将消息作为启动输入。

ForceCompleteIfChannelsEmpty 检查的 Channel 只能由一个 Step 消费。

Step Graph

Definition graph

DrainingExternalChannelFlow

Valid

python · examples/python/dex_examples/patterns/drain-channels/external_publishing/draining_channel_flow.py

核心实现

class ProcessMessage(Step[str]):
def __init__(self, queue_channel: Channel[str]) -> None:
self.queue_channel = queue_channel

def wait_for(self, context: Context, input: str) -> Wait:
if input is None:
return Wait.until(self.queue_channel.for_one())
return Wait.skip_immediately()

async def execute( # type: ignore[override]
self, context: AsyncContext, input: str
) -> StepDecision:
if input is not None:
print(f"DrainingExternalChannelFlow process message: {input}")
else:
values = self.queue_channel.results(context)
if not values:
raise RuntimeError("No channel message found")
value = values[0]
if value is None:
raise RuntimeError("No channel message value found")
print(f"DrainingExternalChannelFlow process message: {value}")

await asyncio.sleep(DRAIN_WINDOW_SECONDS)

return force_complete_if_channels_empty(
None,
StepMovement.of(self, None),
self.queue_channel,
)


class DrainingExternalChannelFlow(Flow[str]):
QUEUE_CHANNEL = "queueChannel"

queue_channel = Channel(QUEUE_CHANNEL, str)

def __init__(self) -> None:
self.process_message = ProcessMessage(self.queue_channel)

def get_steps(self) -> StepList[str]:
return StepList.start_step(self.process_message)

def get_persistence_schema(self) -> PersistenceSchema:
return PersistenceSchema.of(self.queue_channel)

@rpc
def publish_external_channel_message(
self, context: Context, input: str
) -> RPCResult[str]:
self.queue_channel.publish(context, input)
return RPCResult(input)

例子: examples/python/dex_examples/patterns/drain-channels/external_publishing/draining_channel_flow.py

Controller

Controller 会先调用活跃 Flow 的发布 RPC。调用表明 Flow 不活跃时,Controller 会使用该消息作为输入启动新的 Flow;其他错误会返回给调用方。

@blueprint.get("/start-or-publish")
async def start_or_publish_draining_channel() -> str:
flow_id = required_query("workflowId")
try:
await app_state.client.invoke_rpc(
app_state.drain_external.publish_external_channel_message,
flow_id,
"message from start-or-publish endpoint",
)
except FlowNotActiveError:
run_id = await app_state.client.start_flow(
app_state.drain_external,
flow_id,
"first message from start-or-publish",
start_options(),
)
return f"Started the workflow with runId {run_id}"
return "Published to the Flow"

例子: examples/python/dex_examples/patterns/drain-channels/external_publishing/controller.py

相关内容