Turns `packages/core/e2e/e2e.test.ts` into a cross-language conformance
suite and adds `workbench/python` as its first non-JavaScript subject.
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* Add DurableAgent compat tests, e2e agent tests, and migrate to AI SDK v6
- Port ToolLoopAgent test suite as DurableAgent compatibility spec (34 tests,
all expected to fail — each maps to a feature gap to implement)
- Add e2e workflow definitions using mock LLM providers (no API keys needed)
- Add e2e test file for DurableAgent workflows
- Migrate all AI SDK types from V2 to V3 (LanguageModelV2 → V3, etc.)
- Drop AI SDK v5 support: ai peer dep ^5||^6 → ^6, @ai-sdk/provider ^2||^3 → ^3
- Update ai catalog version from 5.0.104 to 6.0.116
- Simplify CompatibleLanguageModel to just LanguageModelV3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Address PR review feedback
- Remove providerExecuted guard on tool-result stream parts (V3: all
tool-results are provider-executed by definition)
- Remove providerExecuted spread from tool-output-available UI chunks
- Replace inline MockLanguageModelV3 with import from ai/test (works
without msw in AI SDK v6)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove streamTextIterator mock from compat tests, use it.fails for gaps
Tests now exercise the real DurableAgent code path instead of mocking
the core iterator. 5 tests pass (features DurableAgent already has),
29 are marked it.fails() for known API gaps that will alert when fixed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Implement Tier 1+2 gaps, add @workflow/ai/test mock provider, wire e2e in CI
DurableAgent API additions:
- Add `instructions` (string | SystemModelMessage | SystemModelMessage[])
as alias for deprecated `system` on constructor
- Add `onStepFinish` and `onFinish` on constructor, merged with stream
options (constructor first, then stream — matching ToolLoopAgent)
- Add `timeout` on stream options (converted to AbortSignal)
- Add `text`, `finishReason`, `totalUsage` to onFinish event
Test infrastructure:
- Add @workflow/ai/test export with `mockModel()` wrapper that wraps
MockLanguageModelV3 from ai/test as an async step function
- E2e workflows now use mockModel() + convertArrayToReadableStream
from @workflow/ai/test instead of inline V2 mock models
- Add e2e-agent.test.ts to test:e2e script so it runs in CI
- Flip 6 compat tests from it.fails → it (now passing)
Score: 11 passing / 23 it.fails (was 5/29)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove e2e agent tests — mock models can't serialize across step boundary
The workflow runtime serializes step arguments, and function closures
(like mock model doStream callbacks) aren't serializable. Mock models
only work in unit tests where 'use step' is a no-op. Real e2e agent
tests would need either a mock HTTP server or real provider credentials.
Also removes 'use step' from mockModel wrapper (closures aren't
serializable) and reverts test:e2e script and example workbench dep.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add working e2e agent tests with mock model step factories
Mock model factories use the same 'use step' pattern as real providers
(anthropic, openai). Closure variables are bound to locals at the step
body level so the SWC plugin detects them via __private_getClosureVars.
All 4 e2e tests pass against local dev server:
- agentBasicE2e: text response (11s)
- agentToolCallE2e: single tool call + text (11s)
- agentMultiStepE2e: 3 sequential tool calls (12s)
- agentErrorToolE2e: FatalError recovery (11s)
Also adds e2e-agent.test.ts to test:e2e script for CI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Use @workflow/ai/test package imports for e2e mock models
Split mock provider into two files to work around SWC constructor
closure bug: mock-create.ts has the model creation logic,
mock.ts has the 'use step' wrappers that capture only serializable
args (strings, plain object arrays).
Exports mockTextModel(text) and mockSequenceModel(responses) —
same 'use step' pattern as real providers (anthropic, openai, etc.).
E2e workflows now import directly from @workflow/ai/test.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Simplify mock provider, add comprehensive e2e tests for all features + gaps
Mock provider:
- Replace mock-create.ts with mock-function-wrapper.ts that simply wraps
MockLanguageModelV3 constructor in a function (SWC class closure bug)
- mockTextModel/mockSequenceModel use mockProvider() from wrapper file
- Bind closure vars at step body level (_text = text) for SWC detection
- Fix AbortController not available in workflow VM sandbox
E2e tests (13 total, all passing):
- Core: basic text, tool call, multi-step, error recovery (4)
- Callbacks: onStepFinish constructor+stream, onFinish constructor+stream (2)
- Features: instructions, timeout (2)
- GAPs documented: onStart, onStepStart, onToolCallStart,
onToolCallFinish, prepareCall (5 — complete but callbacks not called)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add tool approval (needsApproval) gap tests, fix SWC closure var binding
Unit tests: 2 new it.fails() tests for tool approval
- needsApproval: true should pause agent (pending tool call, no result)
- needsApproval as function should receive tool input
E2e tests: 1 new test for tool approval gap
- Documents that needsApproval is currently ignored (tool executes anyway)
Also fixes:
- Bind closure vars at step body level in mock provider (_text = text,
_responses = responses) so SWC plugin detects them
- Guard AbortController usage in workflow VM (not available in sandbox)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add default args for agent e2e workflows in UI definitions
The nextjs-turbopack UI calls workflows with hardcoded default args.
Without these entries, agent workflows were called with no args,
causing prompt=undefined → ModelMessage validation failure.
Also removes debug logging.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add DurableAgent chat UI with tools, update docs for AI SDK v6
Chat UI:
- Tab-based layout with Workflows (existing) and DurableAgent Chat tabs
- Chat powered by DurableAgent + WorkflowChatTransport + ai-elements
- Tools: getWeather (fake data), calculate (math expressions)
- Uses createUIMessageStreamResponse for proper stream serialization
- Reconnect route at /api/chat/[runId]/stream
- ai-elements components: conversation, message, prompt-input, tool
- onStepFinish + onFinish callbacks with console logging
Docs (AI SDK v6 migration):
- system → instructions in DurableAgent constructor examples (10 places)
- LanguageModelV2Prompt → LanguageModelV3Prompt in type references (3 places)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix tool rendering, add reasoning support, model picker, observability links
- Fix tool part rendering: use `input`/`output` props (not `args`/`result`)
and `tool-{name}` part type (AI SDK v6 format)
- Add reasoning support for Opus 4.5 via providerOptions
- Model picker: Haiku 4.5, Sonnet 4, Opus 4.5 (reasoning), GPT-5.2, GPT-5.3
- Fix observability links: localhost:3456 for local, Vercel dashboard for prod
- Add suggestions above prompt input
- Add MessageParts component handling text, tool, reasoning, step-start
- Add loading spinner for submitted state
- Update docs: system → instructions, LanguageModelV2Prompt → V3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix tool output rendering: use input/output props on ToolInput/ToolOutput
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix: Documentation for `PrepareStepInfo` and `PrepareStepResult` interfaces references the obsolete `LanguageModelV2` type while the codebase has fully migrated to `LanguageModelV3`.
This commit fixes the issue reported at docs/content/docs/ai/message-queueing.mdx:36
**Bug explanation:**
The codebase migrated from AI SDK V2 to V3. In `packages/ai/src/agent/types.ts`, `CompatibleLanguageModel` is defined as `LanguageModelV3` (from `@ai-sdk/provider`). The actual TypeScript interfaces in `packages/ai/src/agent/durable-agent.ts` use `string | (() => Promise<CompatibleLanguageModel>)` which resolves to `LanguageModelV3`.
However, the documentation in `docs/content/docs/ai/message-queueing.mdx` at lines 36 and 43 still referenced `LanguageModelV2` for the `model` field in both `PrepareStepInfo` and `PrepareStepResult`. This is inconsistent because:
1. The `messages` fields in the same interfaces were correctly updated to `LanguageModelV3Prompt`
2. The actual source code uses `LanguageModelV3` via `CompatibleLanguageModel`
3. There is no `LanguageModelV2` type anywhere in the codebase
This would mislead developers reading the documentation into using the wrong type.
**Fix explanation:**
Changed both `LanguageModelV2` references to `LanguageModelV3` on lines 36 and 43 of the documentation file, matching the actual codebase types. Verified no other stale `LanguageModelV2` references remain in the docs directory.
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Co-authored-by: pranaygp <pranay.gp@gmail.com>
* Fix instructions tests: flip from it.fails to it, update snapshots
The 3 instructions tests (string, SystemModelMessage, array) now pass.
The snapshots include the assistant reply message from the agent loop,
which is a behavioral difference from ToolLoopAgent (DurableAgent
captures the prompt after the full loop iteration).
Score: 14 passing / 22 it.fails (was 11/25)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix getReadable call: pass startIndex as options object
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix docs type errors and turbopack build
Docs:
- Add await to convertToModelMessages() calls (now async in AI SDK v6)
- Change LanguageModelV3Prompt → ModelMessage[] in type references
- Change LanguageModelV3 → LanguageModel in PrepareStepInfo
- Update docs-globals.d.ts convertToModelMessages return type
- Add LanguageModel to import inference map
DurableAgent:
- Update OutputSpecification to match AI SDK v6 Output interface
(type→name, parsePartial→parsePartialOutput, parseOutput→parseCompleteOutput,
responseFormat now PromiseLike)
Turbopack build:
- Remove streamdown plugins from MessageResponse (plugins prop API
changed in streamdown 2.4.0, causing type mismatch in CI)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix pnpm-workspace.yaml: use double quotes for catalog entries
The stage-workbench-with-tarballs.mjs script only strips double quotes
when parsing catalog keys. Single-quoted @-scoped entries (e.g.,
'@types/node') weren't matched, causing "unresolved catalog dependencies"
errors in CI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix CI build failures, add changeset
- Remove streamdown plugins from reasoning.tsx (same CI type mismatch)
- Cast ToolHeader type prop and WorkflowChatTransport to fix type errors
- Fix pnpm-workspace.yaml single→double quotes for staging script
- Add minor changeset for @workflow/ai (breaking: AI SDK v6 migration)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Sync webpack workbench with turbopack: add chat UI deps and symlinks
- Symlink app-shell.tsx, chat-client.tsx, agent_chat workflow,
chat API routes into nextjs-webpack
- Add matching deps: streamdown, @streamdown/*, shiki, cmdk, nanoid,
motion, @radix-ui/react-use-controllable-state
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix circular symlink: restore chat-client.tsx as real file in turbopack
The previous commit accidentally converted turbopack's chat-client.tsx
into a circular symlink pointing to itself. Webpack's symlink to it
then couldn't resolve, breaking both builds on Vercel.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add missing deps to webpack: use-stick-to-bottom, radix-ui, @vercel/blob
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix model picker: merge body params via prepareSendMessagesRequest
WorkflowChatTransport sends { messages } by default, ignoring the
body option from ChatRequestOptions. Use prepareSendMessagesRequest
to merge { messages, ...body } so the model selection reaches the API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix WorkflowChatTransport: forward body/headers from ChatRequestOptions
The transport hardcoded body: undefined when calling
prepareSendMessagesRequest, so extra body params (like model selection)
from sendMessage({ body: { model } }) were silently dropped.
Now forwards options.body and options.headers to both
prepareSendMessagesRequest and the default request body.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix model IDs: use real AI Gateway model names
gpt-5.2 and gpt-5.3 don't exist in the AI Gateway.
Replace with gpt-4o and gpt-4o-mini.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Use correct AI Gateway model IDs: Opus 4.5, GPT-5.2, GPT-5.3
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Enable reasoning for all model providers
- Anthropic: thinking.type='enabled' with 10k token budget
- OpenAI: reasoningEffort='high'
- Instructions kept for all models (no longer conditionally removed)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix OpenAI reasoning: use 'medium' effort (GPT-5.3 doesn't support 'high')
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Address all PR review comments
- Change changeset from minor to patch (repo convention)
- Use ?? instead of || for system/instructions fallback
- Clean up timeout: store ID, clearTimeout in finally, { once: true } listeners
- Update class docstring example to use instructions
- Map unrecognized finish reasons to 'other' with validation
- Fix duplicate test, align assertion for unrecognized type
- Support ^ exponentiation in calculate tool
- Remove debug console.log from chat client
- Fix ReactNode/ComponentProps type imports in UI components
- Remove unused MockLanguageModelV3 re-exports from mock.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Remove accidentally created empty mock2.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Change changeset back to minor for breaking AI SDK v6 migration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix agent e2e tests: add Vercel world setup for CI
The agent e2e tests only configured the local filesystem world but not
the Vercel world backend. On CI (Vercel prod tests), this caused
VercelOidcTokenError because the world wasn't initialized.
Now matches the setup pattern from e2e.test.ts: configures Vercel world
with OIDC token and project config from CI environment variables.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Deduplicate e2e test utilities: extract shared code to utils.ts
Extract manifest fetching, workflow lookup, world setup, and types
into shared utils.ts. Both e2e.test.ts and e2e-agent.test.ts now
import from the same source, eliminating ~200 lines of duplication.
Shared utilities:
- WorkflowManifest interface
- fetchManifest() with caching
- getWorkflowMetadata() with retry and fallback
- setupWorld() handling local/Vercel/Postgres backends
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix missing deploymentUrl args in e2e.test.ts after utils refactor
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Add @workflow/ai dep to all workbenches for agent e2e tests
All workbenches now have @workflow/ai as a dependency and the
100_durable_agent_e2e.ts symlink, so agent e2e tests run everywhere.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Fix missing imports in e2e.test.ts: add fetchManifest and sleep
The utils refactor removed these imports but they're still used:
- fetchManifest: used in stepFunctionAsStartArgWorkflow test
- sleep (setTimeout): used in webhookWorkflow test
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
Pranay Prakash