feat(workflow-executor): evaluate deterministic condition steps without AI - #1837
feat(workflow-executor): evaluate deterministic condition steps without AI#1837Scra3 wants to merge 2 commits into
Conversation
…ut AI Decision steps in the new Automatic mode carry their branching logic as build-time preRecordedArgs (optionConditions + fallbackOption, wire-final operator names from the PRD-472 contract). The executor now resolves each condition's value from the run's Get Data outputs and evaluates top-to-bottom, first-match-wins — never calling the AI and never awaiting input, because the builder chose this mode precisely to remove AI judgement from the branch. A null/missing/unresolvable value is "not met" (met: null), never an error, and no match selects the fallback, so the step can never end undefined. The evaluation trace is persisted in executionParams for the run view; unknown operators are rejected at the schema boundary so a run never reaches evaluation with a comparison it cannot honor. Part of PRD-472 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
5 new issues
|
![macroscopeapp[bot]](https://avatars.githubusercontent.com/in/900172?s=60&v=4){kind=small}
| function toTimestamp(value: unknown): number | null { | ||
| if (typeof value !== 'string' || !ISO_DATE_PREFIX.test(value)) return null; | ||
|
|
||
| const parsed = Date.parse(value); | ||
|
|
||
| return Number.isNaN(parsed) ? null : parsed; |
There was a problem hiding this comment.
🟠 High executors/deterministic-condition-evaluator.ts:7
toTimestamp accepts invalid calendar dates such as 2026-02-30 and converts them to normalized timestamps, so malformed values compare equal to 2026-03-02 and can satisfy equality, membership, or ordering conditions. Validate that the parsed date round-trips to the original calendar date before returning its timestamp.
-function toTimestamp(value: unknown): number | null {
- if (typeof value !== 'string' || !ISO_DATE_PREFIX.test(value)) return null;
-
- const parsed = Date.parse(value);
-
- return Number.isNaN(parsed) ? null : parsed;
-}
+function toTimestamp(value: unknown): number | null {
+ if (typeof value !== 'string' || !ISO_DATE_PREFIX.test(value)) return null;
+
+ const datePart = value.slice(0, 10);
+ const normalized = new Date(`${datePart}T00:00:00.000Z`);
+ if (Number.isNaN(normalized.getTime()) || normalized.toISOString().slice(0, 10) !== datePart) {
+ return null;
+ }
+
+ const parsed = Date.parse(value);
+
+ return Number.isNaN(parsed) ? null : parsed;
+}🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @packages/workflow-executor/src/executors/deterministic-condition-evaluator.ts around lines 7-12:
`toTimestamp` accepts invalid calendar dates such as `2026-02-30` and converts them to normalized timestamps, so malformed values compare equal to `2026-03-02` and can satisfy equality, membership, or ordering conditions. Validate that the parsed date round-trips to the original calendar date before returning its timestamp.
| operator: z.enum(CONDITION_OPERATORS), | ||
| /** Absent for `present`/`blank`. */ | ||
| value: z.unknown().optional(), | ||
| }); |
There was a problem hiding this comment.
🟠 High validated/step-definition.ts:69
DeterministicConditionSchema accepts { operator: 'equal' }, so the evaluator compares against undefined and silently falls through to the fallback option instead of rejecting the invalid workflow configuration. Add a refinement requiring value for every operator except present and blank.
| }); | |
| }).superRefine((condition, ctx) => { | |
| if (!['present', 'blank'].includes(condition.operator) && condition.value === undefined) { | |
| ctx.addIssue({ | |
| code: 'custom', | |
| path: ['value'], | |
| message: 'value is required for this operator', | |
| }); | |
| } | |
| }); |
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @packages/workflow-executor/src/types/validated/step-definition.ts around line 69:
`DeterministicConditionSchema` accepts `{ operator: 'equal' }`, so the evaluator compares against `undefined` and silently falls through to the fallback option instead of rejecting the invalid workflow configuration. Add a refinement requiring `value` for every operator except `present` and `blank`.
| .optional(), | ||
| }) | ||
| // No silent fallback to manual/AI: a deterministic step without its conditions must fail loud. | ||
| .superRefine((step, ctx) => { |
There was a problem hiding this comment.
🟠 High validated/step-definition.ts:97
A deterministic condition definition with a stale fallbackOption or optionConditions[].option passes this schema and later returns status: 'success' with a selectedOption absent from step.options, so the gateway cannot select an outgoing branch and the workflow cannot advance. The superRefine check only requires preRecordedArgs for deterministic steps; it must also reject every configured option that is not present in step.options, matching the manual path's membership validation.
Also found in 1 other location(s)
packages/workflow-executor/src/executors/condition-step-executor.ts:137
evaluateDeterministicallyacceptsmatchedOption/fallbackOptionwithout checking that the selected string exists instep.options, unlikereadUserChoice. IfpreRecordedArgsis stale or malformed (for example, its fallback names a removed outgoing transition), the executor persists and returnssuccesswith aselectedOptionthat cannot identify any outgoing branch, so the workflow cannot advance correctly. Validate every configured option againststep.optionswhile mapping, or validateselectedOptionbefore returning success.
🚀 Reply "fix it for me" or copy this AI Prompt for your agent:
In file @packages/workflow-executor/src/types/validated/step-definition.ts around line 97:
A deterministic condition definition with a stale `fallbackOption` or `optionConditions[].option` passes this schema and later returns `status: 'success'` with a `selectedOption` absent from `step.options`, so the gateway cannot select an outgoing branch and the workflow cannot advance. The `superRefine` check only requires `preRecordedArgs` for deterministic steps; it must also reject every configured option that is not present in `step.options`, matching the manual path's membership validation.
Also found in 1 other location(s):
- packages/workflow-executor/src/executors/condition-step-executor.ts:137 -- `evaluateDeterministically` accepts `matchedOption`/`fallbackOption` without checking that the selected string exists in `step.options`, unlike `readUserChoice`. If `preRecordedArgs` is stale or malformed (for example, its fallback names a removed outgoing transition), the executor persists and returns `success` with a `selectedOption` that cannot identify any outgoing branch, so the workflow cannot advance correctly. Validate every configured option against `step.options` while mapping, or validate `selectedOption` before returning success.
|
Coverage Impact This PR will not change total coverage. Modified Files with Diff Coverage (4)
🤖 Increase coverage with AI coding...🚦 See full report on Qlty Cloud » 🛟 Help
|
…safe and routable
The deterministic evaluator could turn a data or config mismatch into a silent
misroute — a `status: 'success'` step carrying the wrong branch.
- Numeric strings: Sequelize returns Postgres/MySQL `numeric`/`decimal`/`bigint`
columns as strings while datasource-sequelize maps those types to the `Number`
primitive, so the builder's `value: 100` met `"150.00"` at runtime and every
comparison bailed → no option matched → silent fallback. A strictly numeric
string is now coerced against a real number (both sides stay uncoerced when
neither is a number, since ordering operators are Number/Date-only per the
contract); `'abc'` vs `100` is still not evaluable.
- Selected option: the deterministic path emitted `matchedOption ?? fallbackOption`
without checking `step.options`, unlike the manual path. `optionConditions` and
`options` are two different server-side derivations, so drift produced a success
outcome the orchestrator cannot route and the run died far from the cause. It now
throws `InvalidStepDefinitionError` before persisting anything.
- `not_equal` contradicted the file's own documented policy ("a type mismatch can
never satisfy a negated operator") by returning true on mismatch; equality is now
tri-state, so a mismatch satisfies neither `equal` nor `not_equal`.
- Timezone: an offset-less ISO datetime was parsed host-local, so "deterministic"
evaluation varied per machine. Offset-less datetimes are read as UTC, pinned by a
test that runs under a non-UTC TZ.
- `contains`/`not_contains` were extended to array membership beyond the contract
(§1: String only). Restricted back to strings — dead flexibility the builder never
emits, and the "mismatch is never satisfied" invariant keeps it from misrouting.
- Removed the unreachable `default` branch by replacing the operator switch with an
exhaustive lookup keyed by `ConditionOperator`, so a new operator fails to compile
instead of silently returning null (lint's `default-case` forbids a bare switch).
Also asserts three spec behaviors that were unasserted: deterministic mode ignores
`incomingPendingData`, `or` matches on a mix of not-evaluable and true, and an
unresolvable reference satisfies neither `blank` nor `present`.
Part of PRD-472
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Why
PRD-472: Decision steps can now be configured with explicit conditions instead of a natural-language question. The runtime must evaluate them itself — no LLM call, no human wait.
Stacked on #1836 (the version gating).
What
StepExecutionMode.Deterministic;ConditionStepDefinitionSchemaaccepts it withpreRecordedArgs { optionConditions, fallbackOption }per the cross-repo contract.superRefinemakespreRecordedArgsmandatory in that mode; operators and aggregators are enum-validated, so an unknown operator fails loud instead of evaluating to something arbitrary.doExecute: resolves each condition's value from the run history (sourceStepId+fieldNameagainst the Get Data step's persisted fields, most-recent occurrence for loops), evaluates top-to-bottom with first-match-wins and and/or aggregation.deterministic-condition-evaluator.ts): null/unresolvable value →met: null(not evaluable, never an error), type mismatch → false including for negated operators, ISO dates compared as timestamps.executionParamspersisted as{ evaluations[{ option, outcome, conditions[{ index, met }] }], selectedOption, usedFallback }— the data the run view needs to show why an option won.Tests
Full package suite: 1575 passed / 0 failed. 24 evaluator unit tests (happy / null / type-mismatch per operator) + 10 deterministic-branch executor tests.
Part of PRD-472
🤖 Generated with Claude Code
Note
Add deterministic evaluation mode for condition steps without AI
Deterministicexecution type for condition steps in the workflow executor that evaluates conditions against prior Get Data step outputs, bypassing AI and user input entirely.deterministic-condition-evaluator.tsimplementing operator semantics (equal, numeric/date ordering, in/not_in, contains, present/blank) with coercion for numeric strings and ISO dates.ConditionStepExecutorevaluates orderedoptionConditionswithand/oraggregation, selects the first matching option or usesfallbackOption, and persists a detailed evaluation trace torunStore.ConditionStepDefinitionSchemanow requirespreRecordedArgswhenexecutionTypeisDeterministicand rejects unknown operators or aggregators at parse time.step.options, the executor throwsInvalidStepDefinitionErrorrather than silently failing.Macroscope summarized 8d4ec64.