feat(workflow): workflow runner and public API (Part 5/8) - #592
feat(workflow): workflow runner and public API (Part 5/8)#592kalenkevich wants to merge 1 commit into
Conversation
Part 5/9 of the feature/workflows split. Brings the engine together into a runnable workflow and exposes the public surface. - workflow.ts: the Workflow orchestrator — triggers, routing/fan-out, dynamic entry, and resume/fast-forward. - workflow_agent.ts: BaseAgent adapter so a Workflow runs under the ADK Runner (streams events via the shared AsyncQueue). - dynamic_node_scheduler.ts + utils/rehydration_utils.ts: dynamic scheduling and event-driven state reconstruction for resume. - node.ts: the node() user API. - workflow/index.ts + core/src/index.ts: public barrel exports; typedoc.json marks the internal AsyncQueue/ScheduleDynamicNode/NodeContextOptions as intentionally-not-exported. - register_builtin_nodes.ts: side-effect module imported by node()/workflow so the built-in Function/Tool/Parallel builders are registered even when those entry points are imported directly (not via the barrel). Adapted to Part 1's review APIs: workflow_agent uses AsyncQueue, workflow uses the commonPrefixOf util function. Tests (53): workflow, workflow_advanced, routing, parallel, dynamic_workflow, dynamic_resume, resume, runner_integration, auth_gate, hitl. Full core suite green (2444), docs:check clean, tsc clean. The LLM-agent-as-node tests (node_api, multi_agent, llm_agent) land in Part 7 with the agent builder.
AmaadMartin
left a comment
There was a problem hiding this comment.
Reviewed the public-API surface, the runner loop and the resume path against the files at head. Good things first: it reuses the shared AsyncQueue rather than adding a second queue, there is no any/@ts-expect-error anywhere in the diff, and I checked the new barrel names against common.ts/index.ts for collisions (none today). Three things I would not ship as-is: intentionallyNotExported hides three types that are genuinely on NodeContext's public surface, WorkflowConfig marks a strictly-required choice optional, and the fast-forward path hands user code an object cast to NodeContext that is missing every method. The rest are nits.
| "intentionallyNotExported": [ | ||
| "AsyncQueue", | ||
| "ScheduleDynamicNode", | ||
| "NodeContextOptions" | ||
| ] |
There was a problem hiding this comment.
Not a nit. These three names are silenced in the docs build rather than exported, but all three are reachable from the public surface of NodeContext, which this PR exports from @google/adk.
"intentionallyNotExported": [
"AsyncQueue",
"ScheduleDynamicNode",
"NodeContextOptions"
]// core/src/workflow/node_context.ts
readonly channel: AsyncQueue<Event>; // :45
scheduler?: ScheduleDynamicNode; // :77
constructor(opts: NodeContextOptions) // :85AsyncQueue is not exported from core/src/index.ts or core/src/common.ts today, so someone writing a custom node cannot name the type of ctx.channel, and cannot construct a NodeContext in their own test without re-declaring the options shape by hand. intentionallyNotExported makes docs:check green without fixing what it was reporting.
Either export them (AsyncQueue via common.ts, NodeContextOptions/ScheduleDynamicNode via the workflow barrel) and drop this block, or, if scheduler really is internal wiring, make it non-public instead of hiding it from the docs. This is the one thing in the PR that is genuinely hard to change after release.
| export * from './tools/mcp/mcp_session_manager.js'; | ||
| export * from './tools/mcp/mcp_tool.js'; | ||
| export * from './tools/mcp/mcp_toolset.js'; | ||
| export * from './workflow/index.js'; |
There was a problem hiding this comment.
Nit. Star re-export of a ~45-symbol barrel, where the rest of this file is deliberate named exports.
export * from './workflow/index.js';Only ./common.js and the three tools/mcp/* lines use export *; everything else is export {X} / export type {Y}. With a star, the top-level @google/adk surface changes silently whenever workflow/index.ts changes, and if a name ever collides with an existing export ESM drops it from both sides with no error. I checked the current names (Node, Graph, Edge, START, RetryConfig, ErrorClass, RequestInput, NodeState, ...) against common.ts and index.ts — no collision today, so this is only about keeping the surface intentional.
Separately: the workflow barrel is added here but not to core/src/common.ts, which is what core/src/index_web.ts re-exports — so the workflow API is absent from the web entry point. Intentional? Nothing in workflow/ looks node-only.
| // --- Nodes --- | ||
| export {BaseNode, START} from './base_node.js'; | ||
| export type {BaseNodeConfig} from './base_node.js'; | ||
| export {Node, node} from './node.js'; |
There was a problem hiding this comment.
Not a nit. Public naming, and this is the PR that fixes it in place.
export {Node, node} from './node.js';Node and node both land in the flat @google/adk namespace differing only in case, and Node shadows the DOM / @types/node global — import {Node} from '@google/adk' in a file that also touches DOM types is a quiet footgun, and Node/node next to each other in an import list is easy to mistype. The same, more mildly, for Edge, Graph, START and DEFAULT_ROUTE (lines 21-22, 40): generic words in a root barrel.
Python gets namespacing for free (workflow.Node via the module); a flat root export does not, so parity does not force the name here. WorkflowNode for the subclassing base (keeping node() as the factory) would read consistently with WorkflowAgent/WorkflowConfig in this same file.
| export interface WorkflowConfig extends BaseNodeConfig { | ||
| /** Edge definitions used to build the workflow graph. */ | ||
| edges?: EdgeItem[]; | ||
| /** | ||
| * An imperative entry function driving execution via `ctx.runNode(...)`. | ||
| * Mutually exclusive with {@link edges}. | ||
| */ | ||
| dynamicEntry?: DynamicEntry; | ||
| /** | ||
| * Maximum number of graph-scheduled nodes running in parallel. `undefined` | ||
| * means unlimited. Does not throttle dynamic (`ctx.runNode`) children. | ||
| */ | ||
| maxConcurrency?: number; | ||
| } |
There was a problem hiding this comment.
Not a nit. Exactly one of edges / dynamicEntry is required, but the type marks both optional.
export interface WorkflowConfig extends BaseNodeConfig {
edges?: EdgeItem[];
dynamicEntry?: DynamicEntry;
maxConcurrency?: number;
}So new Workflow({name: 'x'}) typechecks and then throws at :101-105, and passing both typechecks and throws at :96-100. The repo guideline is to reflect strictly-required options in the type; a union does it at compile time:
export type WorkflowConfig = BaseNodeConfig & {maxConcurrency?: number} & (
| {edges: EdgeItem[]; dynamicEntry?: never}
| {dynamicEntry: DynamicEntry; edges?: never}
);That also retires the config.edges! non-null assertion at :109. Keep the runtime throws for JS callers. Caveat I could not fully check: this turns an interface into a type alias, so if Parts 6-8 do interface X extends WorkflowConfig they would need type X = WorkflowConfig & {...} — I only looked at this part's diff.
| export function makeFastForwardContext( | ||
| parent: NodeContext, | ||
| prior: RehydratedNode, | ||
| ): NodeContext { | ||
| return { | ||
| output: prior.output, | ||
| route: prior.route, | ||
| branch: prior.branch ?? parent.branch, | ||
| interruptIds: [], | ||
| } as unknown as NodeContext; |
There was a problem hiding this comment.
Not a nit. A four-field object is cast to a full NodeContext and then handed to user code on the resume path.
return {
output: prior.output,
route: prior.route,
branch: prior.branch ?? parent.branch,
interruptIds: [],
} as unknown as NodeContext;Inside the workflow loop this is safe — handleCompletion only reads those four fields. But DynamicNodeScheduler.schedule returns it directly (dynamic_node_scheduler.ts:73), and NodeContext.runNode returns the scheduler's result verbatim (node_context.ts:140-148). So in a dynamicEntry workflow that is resumed, this is what the author's own code receives:
const r = await ctx.runNode(child, input);
r.emit(evt); // TypeError: r.emit is not a function
r.state; // undefinedThe double cast is exactly what stops the compiler from reporting it. Same cast at workflow.ts:337 for the static-graph resume path. Suggest a real shared result type that both paths return honestly:
export interface NodeResult {
output: unknown;
route?: RouteValue | RouteValue[];
branch?: string;
interruptIds: string[];
}with runNode() typed Promise<NodeContext | NodeResult> — or build a genuine child NodeContext for the fast-forward case so the public contract still holds. Two sites either way.
| for await (const event of channel) { | ||
| yield event; | ||
| } | ||
| await settle; |
There was a problem hiding this comment.
Not a nit, optional. Nothing cleans up if the consumer stops reading.
for await (const event of channel) {
yield event;
}
await settle;If the caller breaks out of its for await (or the Runner cancels the invocation), this generator's return() runs, the loop exits and await settle never executes — the workflow keeps going, nodes keep making model/tool calls, and they keep pushing into an AsyncQueue nobody drains. A try { ... } finally { channel.close(); await settle; } gives it one exit path.
Related, and the reason I'd fix them together: workflow.ts:534-539 awaits in-flight siblings with Promise.allSettled after a node fails but never cancels them, so a workflow that is already throwing still pays for every sibling to run to completion. node_runner.ts:185-193 already builds each child's abortSignal from an AbortController chained to invocationContext.abortSignal, so a workflow-level controller aborted in cleanupPending (and in the finally above) would be picked up by cooperative nodes for free.
| private atConcurrencyLimit(loop: LoopState): boolean { | ||
| return !!this.maxConcurrency && loop.pending.size >= this.maxConcurrency; |
There was a problem hiding this comment.
Nit. maxConcurrency: 0 silently means unlimited.
private atConcurrencyLimit(loop: LoopState): boolean {
return !!this.maxConcurrency && loop.pending.size >= this.maxConcurrency;
}0 is falsy, so the one value a caller might reach for to mean "stop scheduling" turns the cap off entirely — the opposite of the intent. Also -1 / 1.5 are accepted.
return this.maxConcurrency !== undefined && loop.pending.size >= this.maxConcurrency;plus a constructor check rejecting anything < 1. The doc comment at :50-52 is honest that dynamic ctx.runNode() children are not throttled — but that does mean a dynamicEntry workflow has no cap at all on in-flight children. Is bounding those planned for a later part?
| * Replay/checkpointing, dynamic scheduling, and task/chat isolation scopes are | ||
| * added in later phases; hook points are marked with TODO(phase-N). | ||
| */ | ||
| export class Workflow extends BaseNode { |
There was a problem hiding this comment.
Not a nit. This PR turns the whole workflow subsystem into public @google/adk API, and none of the new entry points are marked experimental.
export class Workflow extends BaseNode {The repo has the decorator for exactly this case (core/src/utils/experimental.ts), used at core/src/tools/openapi_tool/openapi_toolset.ts:17 and core/src/skills/gcp_skill_registry.ts:22:
@experimental
export class Workflow extends BaseNode {Same for WorkflowAgent (workflow_agent.ts:32) and Node (node.ts:44). Parts 6-8 still change this surface (the barrel itself says LLM nodes land in Part 7), so marking it now is what buys the freedom to change it without a semver break — and it is much easier to add before the first release than to explain afterwards.
| } | ||
| } | ||
|
|
||
| // eslint-disable-next-line require-yield |
There was a problem hiding this comment.
Nit. Bare suppression, no reason on the directive line.
// eslint-disable-next-line require-yieldBoth uses here are legitimate — the AsyncGenerator signature is mandated by the base class and there is genuinely nothing to yield — but the repo standard is to name the cause inline so the next reader does not have to work it out:
// eslint-disable-next-line require-yield -- child events stream via ctx.channel; this generator yields nothingAlso at workflow_agent.ts:92 (runLiveImpl throws).
| } as unknown as Session; | ||
| return new InvocationContext({ | ||
| invocationId: 'inv-1', | ||
| session, | ||
| agent: { | ||
| name: 'wf', | ||
| runAsync: async function* () {}, | ||
| } as unknown as BaseAgent, |
There was a problem hiding this comment.
Nit. The same fixture, with the same two double-casts, is copy-pasted across five test files.
} as unknown as Session;
...
} as unknown as BaseAgent,createIc() + driveWorkflow() appear verbatim in workflow_test.ts:22-34, parallel_test.ts:28-40, hitl_test.ts:26-38, dynamic_workflow_test.ts:21-33 and auth_gate_test.ts:147-159. Two things:
createSession()(core/src/sessions/session.ts:63) returns a realSession, so the first cast can just go — that is the factory the rest ofcore/testuses.- For the agent, a three-line local class extending
BaseAgentremoves the second cast and typechecks for real.
Then lift the pair into core/test/workflow/testing_utils.ts and import it. That is ~60 duplicated lines gone, and Parts 6-8 edit one fixture instead of six. Nothing else in the diff reaches for any or @ts-expect-error, which is why these stood out.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
Problem: With the engine core and node types in place, the workflow needs a runnable orchestrator and a public API.
Solution — Part 5 of 8 (dynamic scheduling + rehydration were merged into the runner PR because they are runner-coupled and share its tests). Stacked on Part 4.
Included:
workflow.ts— theWorkfloworchestrator: triggers, routing/fan-out, dynamic entry, resume/fast-forward.workflow_agent.ts—BaseAgentadapter so aWorkflowruns under the ADKRunner(streams via the sharedAsyncQueue).dynamic_node_scheduler.ts+utils/rehydration_utils.ts— dynamic scheduling and event-driven state reconstruction for resume.node.ts— thenode()user API.workflow/index.ts+core/src/index.ts— public barrel;typedoc.jsonmarks internalAsyncQueue/ScheduleDynamicNode/NodeContextOptionsas intentionally-not-exported.register_builtin_nodes.ts— side-effect module imported bynode()/workflowso the built-in Function/Tool/Parallel builders register even when those entry points are imported directly (not via the barrel).Adapted to Part 1's review APIs (
AsyncQueue,commonPrefixOf).Testing Plan
Tests (53):
workflow,workflow_advanced,routing,parallel,dynamic_workflow,dynamic_resume,resume,runner_integration,auth_gate,hitl. Full core suite 2444 green; docs:check clean; tsc clean.Manual E2E: N/A (covered by the runner integration tests; LLM-agent-as-node tests land in Part 6).
Checklist
Additional context
Stacked split — merge in order (…Part 4 → Part 5 → Part 6 → …).