mirror of
https://github.com/shadcn-ui/ui.git
synced 2026-09-14 20:00:29 +08:00
feat(helpers): human-in-the-loop mocking for the AI SDK (#11484)
* feat(helpers): add human-in-the-loop mocking * feat(helpers): adopt useChat message generics and finalize human-in-the-loop * chore(helpers): tighten changesets * refactor(registry): use named message type aliases with createChat * docs(changelog): add helpers human-in-the-loop entry * docs: add changelog
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@shadcn/helpers": minor
|
||||
---
|
||||
|
||||
Add human-in-the-loop mocking for the AI SDK.
|
||||
@@ -0,0 +1,5 @@
|
||||
---
|
||||
"@shadcn/helpers": minor
|
||||
---
|
||||
|
||||
Make `createChat` generic over the UI message type, like `useChat`.
|
||||
@@ -0,0 +1,62 @@
|
||||
---
|
||||
title: August 2026 - Human in the Loop
|
||||
description: Mock paused tool calls, approvals, and continuations for the AI SDK with @shadcn/helpers.
|
||||
date: 2026-08-12
|
||||
---
|
||||
|
||||
**@shadcn/helpers** can now mock human-in-the-loop flows for the AI SDK. A
|
||||
scripted conversation can pause for real user input, wait for an approval, and
|
||||
continue with whatever the user decided.
|
||||
|
||||
Everything streams through the real
|
||||
`useChat` lifecycle, so your tool cards, approval prompts, and question flows
|
||||
behave exactly as they would in production.
|
||||
|
||||
```ts showLineNumbers
|
||||
import { createChat } from "@shadcn/helpers/ai-sdk"
|
||||
|
||||
const chat = createChat<ChatMessage>()
|
||||
.user("Help me plan the next prototype.")
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("A couple of questions before I start.")
|
||||
writer.tool("askQuestions", { input: { questions } })
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.name === "askQuestions" && toolCall.output
|
||||
? `Starting with ${toolCall.output.answers.direction}.`
|
||||
: "Starting now."
|
||||
)
|
||||
})
|
||||
```
|
||||
|
||||
Pass `needsApproval` to pause behind the user's decision. The scripted output
|
||||
streams after approval, and denial streams automatically.
|
||||
|
||||
```ts showLineNumbers
|
||||
chat
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("That will archive 3 drafts. I need your approval.")
|
||||
writer.tool("archiveDrafts", {
|
||||
input: { count: 3 },
|
||||
needsApproval: true,
|
||||
output: { archived: 3 },
|
||||
})
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.approved ? "Archived 3 drafts." : "Okay, leaving them in place."
|
||||
)
|
||||
})
|
||||
```
|
||||
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button asChild size="sm">
|
||||
<Link
|
||||
href="/docs/helpers/ai-sdk#human-in-the-loop"
|
||||
className="mt-6 no-underline!"
|
||||
>
|
||||
Read the Docs
|
||||
</Link>
|
||||
</Button>
|
||||
</div>
|
||||
@@ -10,7 +10,8 @@ Because the conversation streams through the real `useChat` lifecycle, your
|
||||
components behave exactly as they would in production.
|
||||
|
||||
It supports every part type the AI SDK does: reasoning, tools, data, files,
|
||||
sources, and custom parts.
|
||||
sources, and custom parts. Tool calls can also pause for real user input,
|
||||
including approvals. See [Human in the Loop](#human-in-the-loop).
|
||||
|
||||
```ts showLineNumbers
|
||||
import { createChat } from "@shadcn/helpers/ai-sdk"
|
||||
@@ -323,10 +324,12 @@ writer.tool("getWeather", {
|
||||
})
|
||||
```
|
||||
|
||||
Type the tool name, input, and output by passing your tool definitions to
|
||||
`createChat`.
|
||||
Type the tool name, input, and output by passing your message type to
|
||||
`createChat`. It takes the same type parameter as `useChat`.
|
||||
|
||||
```ts
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
type Tools = {
|
||||
getWeather: {
|
||||
input: { city: string }
|
||||
@@ -334,11 +337,148 @@ type Tools = {
|
||||
}
|
||||
}
|
||||
|
||||
type DataParts = Record<string, never>
|
||||
type ChatMessage = UIMessage<unknown, Record<string, never>, Tools>
|
||||
|
||||
const chat = createChat<unknown, DataParts, Tools>()
|
||||
const chat = createChat<ChatMessage>()
|
||||
```
|
||||
|
||||
Define the message type once and use it for both `createChat` and `useChat`.
|
||||
|
||||
```ts
|
||||
const { messages } = useChat<ChatMessage>({
|
||||
messages: chat.get(0),
|
||||
transport: chat.transport(),
|
||||
})
|
||||
```
|
||||
|
||||
A tool call can also wait for the user instead of finishing in the script.
|
||||
See [Human in the Loop](#human-in-the-loop).
|
||||
|
||||
---
|
||||
|
||||
## Human in the Loop
|
||||
|
||||
A tool call can pause and hand control to the real user. The helper supports
|
||||
both AI SDK human-in-the-loop flows: client-executed tools, where the user
|
||||
supplies the output, and approval-gated tools, where the user approves or
|
||||
denies before a scripted output streams.
|
||||
|
||||
### Pause a Tool Call
|
||||
|
||||
Leave a tool call unresolved to pause the turn. The input streams, the turn
|
||||
finishes, and the part stays in the `input-available` state until the client
|
||||
supplies its output with `addToolOutput`.
|
||||
|
||||
```ts
|
||||
chat.assistant(({ writer }) => {
|
||||
writer.text("Answer these and I'll tailor the prototype.")
|
||||
writer.tool("askQuestions", { input: { questions } })
|
||||
})
|
||||
```
|
||||
|
||||
### Require Approval
|
||||
|
||||
Pass `needsApproval: true` to pause behind the user's decision. `output` then
|
||||
means "stream this after approval" instead of "resolve immediately". Denial
|
||||
streams `tool-output-denied` automatically, and `errorText` with
|
||||
`needsApproval` scripts a tool that fails after approval.
|
||||
|
||||
```ts
|
||||
chat.assistant(({ writer }) => {
|
||||
writer.text("That will archive 3 drafts. I need your approval.")
|
||||
writer.tool("archiveDrafts", {
|
||||
input: { count: 3 },
|
||||
needsApproval: true,
|
||||
output: { archived: 3 },
|
||||
})
|
||||
})
|
||||
```
|
||||
|
||||
Calling `output()`, `error()`, or `denied()` on the handle of a
|
||||
`needsApproval` call throws. The user's decision resolves it.
|
||||
|
||||
### Continuations
|
||||
|
||||
A callback turn scripted immediately after a paused turn becomes a
|
||||
continuation. It does not materialize when the script is built. It runs when
|
||||
the follow-up request arrives, and its context includes `toolCall`: the
|
||||
paused call joined with what the user did.
|
||||
|
||||
```ts
|
||||
chat
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("askQuestions", { input: { questions } })
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.name === "askQuestions" && toolCall.output
|
||||
? `Starting with ${toolCall.output.answers.direction}.`
|
||||
: "Starting now."
|
||||
)
|
||||
})
|
||||
```
|
||||
|
||||
For a client-executed tool, `toolCall.output` is the output the user
|
||||
submitted. For an approval, `toolCall.approved` and `toolCall.denied` carry
|
||||
the decision, and the scripted output streams before the callback's content.
|
||||
|
||||
| Property | Description |
|
||||
| ------------ | -------------------------------------------------------------- |
|
||||
| `name` | The tool name. Comparing it narrows `input` and `output`. |
|
||||
| `toolCallId` | The paused call's ID. |
|
||||
| `input` | The tool input. |
|
||||
| `output` | The user-submitted output, or the gated output after approval. |
|
||||
| `approved` | `true` when the user approved. Approval calls only. |
|
||||
| `denied` | `true` when the user denied. Approval calls only. |
|
||||
|
||||
Continuation context also includes `messages`, the live transcript, and
|
||||
`toolCalls`, every paused call when a turn has more than one.
|
||||
|
||||
Continuations must stay pure. Regenerating re-resolves them against the
|
||||
current transcript, so a changed decision produces the other branch.
|
||||
|
||||
### Client Wiring
|
||||
|
||||
The client side uses the AI SDK as-is. Submit a client-executed tool's output
|
||||
with `addToolOutput`, answer an approval with `addToolApprovalResponse`, and
|
||||
let `sendAutomaticallyWhen` send the follow-up request.
|
||||
|
||||
```tsx
|
||||
import {
|
||||
lastAssistantMessageIsCompleteWithApprovalResponses,
|
||||
lastAssistantMessageIsCompleteWithToolCalls,
|
||||
} from "ai"
|
||||
|
||||
const { messages, addToolOutput, addToolApprovalResponse } = useChat({
|
||||
messages: chat.get(0),
|
||||
transport: chat.transport(),
|
||||
sendAutomaticallyWhen: (options) =>
|
||||
lastAssistantMessageIsCompleteWithToolCalls(options) ||
|
||||
lastAssistantMessageIsCompleteWithApprovalResponses(options),
|
||||
})
|
||||
```
|
||||
|
||||
```tsx
|
||||
// In a pending tool part renderer.
|
||||
addToolOutput({
|
||||
tool: "askQuestions",
|
||||
toolCallId: part.toolCallId,
|
||||
output: { answers },
|
||||
})
|
||||
|
||||
// In an approval-requested part renderer.
|
||||
addToolApprovalResponse({ id: part.approval.id, approved: true })
|
||||
```
|
||||
|
||||
The continuation streams as a new step of the paused assistant message. The
|
||||
client merges its parts into that message instead of adding a new one, and
|
||||
the helper emits the step boundary and omits the continuation's message id so
|
||||
the merge stays in place and the automatic send does not re-trigger.
|
||||
|
||||
In development, the helper warns when a `needsApproval` call has no
|
||||
continuation turn after it, and when a continuation resolves without a
|
||||
pending tool call.
|
||||
|
||||
---
|
||||
|
||||
## Data
|
||||
@@ -355,13 +495,15 @@ type DataParts = {
|
||||
}
|
||||
}
|
||||
|
||||
const chat = createChat<unknown, DataParts>().assistant(({ writer }) => {
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-sf",
|
||||
data: { city: "San Francisco", status: "loading" },
|
||||
})
|
||||
})
|
||||
const chat = createChat<UIMessage<unknown, DataParts>>().assistant(
|
||||
({ writer }) => {
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-sf",
|
||||
data: { city: "San Francisco", status: "loading" },
|
||||
})
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
Send the same `type` and `id` again to update the part in place. This is useful
|
||||
@@ -499,7 +641,9 @@ chat.get(2) // The first two messages.
|
||||
chat.get(0) // An empty initial conversation.
|
||||
```
|
||||
|
||||
`get()` returns cloned messages and does not change the chat.
|
||||
`get()` returns cloned messages and does not change the chat. It stops before
|
||||
the first [continuation turn](#continuations), since a continuation has no
|
||||
message without a live transcript, and throws when `count` reaches past one.
|
||||
|
||||
Use `next()` to find the next predefined user message after the messages already
|
||||
shown.
|
||||
@@ -550,7 +694,9 @@ const { messages, sendMessage } = useChat({
|
||||
When `sendMessage()` runs, the transport finds the assistant message that
|
||||
follows the current transcript and streams it through the normal AI SDK chat
|
||||
lifecycle. It uses message IDs first and falls back to matching the role and
|
||||
text of the latest message.
|
||||
text of the latest message. Automatic sends after tool results and approval
|
||||
responses resolve the next [continuation](#continuations); only a
|
||||
regeneration replays a turn by its message ID.
|
||||
|
||||
### Options
|
||||
|
||||
@@ -665,7 +811,7 @@ type Metadata = {
|
||||
model: string
|
||||
}
|
||||
|
||||
const chat = createChat<Metadata>()
|
||||
const chat = createChat<UIMessage<Metadata>>()
|
||||
.user("Hello", { metadata: { model: "demo" } })
|
||||
.assistant("Hi.", {
|
||||
id: "assistant-welcome",
|
||||
@@ -696,32 +842,33 @@ export and option.
|
||||
Creates a typed conversation and returns the fluent chat interface.
|
||||
|
||||
```ts
|
||||
function createChat<
|
||||
METADATA = unknown,
|
||||
DATA_PARTS extends UIDataTypes = UIDataTypes,
|
||||
TOOLS extends UITools = UITools,
|
||||
>(
|
||||
options?: CreateChatOptions<METADATA, DATA_PARTS, TOOLS>
|
||||
): AiSdkChat<METADATA, DATA_PARTS, TOOLS>
|
||||
function createChat<UI_MESSAGE extends UIMessage = UIMessage>(
|
||||
options?: CreateChatOptions<UI_MESSAGE>
|
||||
): AiSdkChat<UI_MESSAGE>
|
||||
```
|
||||
|
||||
#### Type Parameters
|
||||
|
||||
| Parameter | Description |
|
||||
| ------------ | --------------------------------------------------------- |
|
||||
| `METADATA` | The metadata shape stored on each `UIMessage`. |
|
||||
| `DATA_PARTS` | A map of names to payloads for typed `data-*` parts. |
|
||||
| `TOOLS` | A map of tool names to their `input` and `output` shapes. |
|
||||
| Parameter | Description |
|
||||
| ------------ | ------------------------------------------------------------------------------------------------------------------ |
|
||||
| `UI_MESSAGE` | The `UIMessage` type for the conversation, the same type parameter `useChat` accepts. Define it once and share it. |
|
||||
|
||||
The message type carries the metadata shape, the typed `data-*` parts, and the
|
||||
tool definitions.
|
||||
|
||||
```ts
|
||||
type ChatMessage = UIMessage<Metadata, DataParts, Tools>
|
||||
```
|
||||
|
||||
#### Options
|
||||
|
||||
| Option | Type | Default | Description |
|
||||
| ------------------ | ------------------------------------------ | ---------------------------- | ------------------------------------------------------- |
|
||||
| `messages` | `UIMessage<METADATA, DATA_PARTS, TOOLS>[]` | `[]` | Start from an existing transcript. Messages are cloned. |
|
||||
| `messageIdPrefix` | `string` | `"msg"` | Prefix for generated message IDs. |
|
||||
| `toolCallIdPrefix` | `string` | `"call"` | Prefix for generated tool call IDs. |
|
||||
| `sourceIdPrefix` | `string` | `"source"` | Prefix for generated source IDs. |
|
||||
| `now` | `Date \| string` | `"2026-01-01T00:00:00.000Z"` | Fixed time used for generated `createdAt` metadata. |
|
||||
| Option | Type | Default | Description |
|
||||
| ------------------ | ---------------- | ---------------------------- | ------------------------------------------------------- |
|
||||
| `messages` | `UIMessage[]` | `[]` | Start from an existing transcript. Messages are cloned. |
|
||||
| `messageIdPrefix` | `string` | `"msg"` | Prefix for generated message IDs. |
|
||||
| `toolCallIdPrefix` | `string` | `"call"` | Prefix for generated tool call IDs. |
|
||||
| `sourceIdPrefix` | `string` | `"source"` | Prefix for generated source IDs. |
|
||||
| `now` | `Date \| string` | `"2026-01-01T00:00:00.000Z"` | Fixed time used for generated `createdAt` metadata. |
|
||||
|
||||
IDs found in `messages` are reserved, so newly generated IDs continue after
|
||||
the existing transcript.
|
||||
@@ -767,11 +914,11 @@ type FilePayload = {
|
||||
|
||||
#### `assistant()` Input
|
||||
|
||||
| Input | Result |
|
||||
| ---------------------- | ---------------------------------------------------------------------- |
|
||||
| `string` | One text part that streams word by word. |
|
||||
| `UIMessagePart[]` | Static AI SDK parts that stream in their existing order. |
|
||||
| `({ writer }) => void` | A synchronous callback for scripting parts, tools, errors, and timing. |
|
||||
| Input | Result |
|
||||
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `string` | One text part that streams word by word. |
|
||||
| `UIMessagePart[]` | Static AI SDK parts that stream in their existing order. |
|
||||
| `({ writer }) => void` | A synchronous callback for scripting parts, tools, errors, and timing. After a paused turn it becomes a [continuation](#continuations) and receives `toolCall`, `toolCalls`, and `messages`. |
|
||||
|
||||
#### `assistant()` Options
|
||||
|
||||
@@ -825,21 +972,24 @@ Calling `reasoning()` without content uses
|
||||
|
||||
#### Tool Options
|
||||
|
||||
| Option | Type | Description |
|
||||
| ------------------ | ------------------------- | ---------------------------------------------------- |
|
||||
| `toolCallId` | `string` | Use a specific tool call ID. |
|
||||
| `title` | `string` | Add a display title to the tool part. |
|
||||
| `toolMetadata` | `Record<string, unknown>` | Add provider or application metadata. |
|
||||
| `providerExecuted` | `boolean` | Mark the call as executed by the provider. |
|
||||
| `input` | `TOOLS[NAME]["input"]` | Set the typed tool input. |
|
||||
| `output` | `TOOLS[NAME]["output"]` | Immediately finish with a typed output. |
|
||||
| `errorText` | `string` | Immediately finish with an error. |
|
||||
| `dynamic` | `boolean` | Emit a `dynamic-tool` part instead of `tool-<name>`. |
|
||||
| Option | Type | Description |
|
||||
| ------------------ | ------------------------- | ----------------------------------------------------------------------------------------------- |
|
||||
| `toolCallId` | `string` | Use a specific tool call ID. |
|
||||
| `title` | `string` | Add a display title to the tool part. |
|
||||
| `toolMetadata` | `Record<string, unknown>` | Add provider or application metadata. |
|
||||
| `providerExecuted` | `boolean` | Mark the call as executed by the provider. |
|
||||
| `input` | `TOOLS[NAME]["input"]` | Set the typed tool input. |
|
||||
| `output` | `TOOLS[NAME]["output"]` | Immediately finish with a typed output. With `needsApproval`, stream it after approval instead. |
|
||||
| `errorText` | `string` | Immediately finish with an error. With `needsApproval`, stream it after approval instead. |
|
||||
| `dynamic` | `boolean` | Emit a `dynamic-tool` part instead of `tool-<name>`. |
|
||||
| `needsApproval` | `boolean` | Pause the turn behind the user's decision. See [Human in the Loop](#human-in-the-loop). |
|
||||
| `approvalId` | `string` | Use a specific approval ID. |
|
||||
|
||||
`tool()` returns a handle with `sleep(delayMs)`, `output(value)`,
|
||||
`error(errorText?)`, and `denied()`. Use the handle when the tool lifecycle
|
||||
needs events between its input and result. Calling `tool.error()` without a
|
||||
message uses `"Tool call failed."`.
|
||||
message uses `"Tool call failed."`. The resolution methods throw on a
|
||||
`needsApproval` call.
|
||||
|
||||
#### Data Input
|
||||
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
import { createChat } from "@/lib/ai"
|
||||
import { type MessagePartRenderProps } from "@/components/message-parts"
|
||||
import {
|
||||
@@ -65,11 +67,13 @@ type MessagePartsTools = {
|
||||
}
|
||||
}
|
||||
|
||||
const messagePartsChat = createChat<
|
||||
type MessagePartsMessage = UIMessage<
|
||||
unknown,
|
||||
MessagePartsData,
|
||||
MessagePartsTools
|
||||
>()
|
||||
>
|
||||
|
||||
const messagePartsChat = createChat<MessagePartsMessage>()
|
||||
.user("Can you review this screenshot and check the deployment?", {
|
||||
files: [
|
||||
{
|
||||
@@ -126,7 +130,6 @@ const messagePartsChat = createChat<
|
||||
|
||||
const messagePartsMessages = messagePartsChat.get()
|
||||
|
||||
type MessagePartsMessage = (typeof messagePartsMessages)[number]
|
||||
type MessagePartsPart = MessagePartsMessage["parts"][number]
|
||||
|
||||
export default function MessageExample() {
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
import { createChat } from "@/lib/ai"
|
||||
import { type MessagePartRenderProps } from "@/components/message-parts"
|
||||
import {
|
||||
@@ -65,11 +67,13 @@ type MessagePartsTools = {
|
||||
}
|
||||
}
|
||||
|
||||
const messagePartsChat = createChat<
|
||||
type MessagePartsMessage = UIMessage<
|
||||
unknown,
|
||||
MessagePartsData,
|
||||
MessagePartsTools
|
||||
>()
|
||||
>
|
||||
|
||||
const messagePartsChat = createChat<MessagePartsMessage>()
|
||||
.user("Can you review this screenshot and check the deployment?", {
|
||||
files: [
|
||||
{
|
||||
@@ -126,7 +130,6 @@ const messagePartsChat = createChat<
|
||||
|
||||
const messagePartsMessages = messagePartsChat.get()
|
||||
|
||||
type MessagePartsMessage = (typeof messagePartsMessages)[number]
|
||||
type MessagePartsPart = MessagePartsMessage["parts"][number]
|
||||
|
||||
export default function MessageExample() {
|
||||
|
||||
@@ -77,7 +77,7 @@ const scriptedEvents = [
|
||||
{ type: "user-left", name: "Aleena", order: 5.5, delayMs: 2000 },
|
||||
] satisfies ScriptedChatEvent[]
|
||||
|
||||
const chat = createChat<Metadata>()
|
||||
const chat = createChat<ChatMessage>()
|
||||
.user(
|
||||
"Hey everyone — dinner this Saturday? I was thinking something casual at my place."
|
||||
)
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import * as React from "react"
|
||||
import { useChat } from "@ai-sdk/react"
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
import { createChat, getMessageText } from "@/lib/ai"
|
||||
import { PartReasoning } from "@/registry/bases/radix/blocks/preview-03/components/part-reasoning"
|
||||
@@ -51,7 +52,9 @@ type Tools = {
|
||||
}
|
||||
}
|
||||
|
||||
const chat = createChat<unknown, Record<string, never>, Tools>()
|
||||
type ChatMessage = UIMessage<unknown, Record<string, never>, Tools>
|
||||
|
||||
const chat = createChat<ChatMessage>()
|
||||
.user("Check the deployment health for the checkout flow.")
|
||||
.sleep(1000)
|
||||
.assistant(({ writer }) => {
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
import { createChat } from "@/lib/ai"
|
||||
import { type MessagePartRenderProps } from "@/components/message-parts"
|
||||
import {
|
||||
@@ -65,11 +67,13 @@ type MessagePartsTools = {
|
||||
}
|
||||
}
|
||||
|
||||
const messagePartsChat = createChat<
|
||||
type MessagePartsMessage = UIMessage<
|
||||
unknown,
|
||||
MessagePartsData,
|
||||
MessagePartsTools
|
||||
>()
|
||||
>
|
||||
|
||||
const messagePartsChat = createChat<MessagePartsMessage>()
|
||||
.user("Can you review this screenshot and check the deployment?", {
|
||||
files: [
|
||||
{
|
||||
@@ -126,7 +130,6 @@ const messagePartsChat = createChat<
|
||||
|
||||
const messagePartsMessages = messagePartsChat.get()
|
||||
|
||||
type MessagePartsMessage = (typeof messagePartsMessages)[number]
|
||||
type MessagePartsPart = MessagePartsMessage["parts"][number]
|
||||
|
||||
export default function MessageExample() {
|
||||
|
||||
@@ -20,6 +20,61 @@ const chat = createChat()
|
||||
})
|
||||
```
|
||||
|
||||
## Human in the loop
|
||||
|
||||
A tool call left unresolved pauses the turn: the input streams, the turn
|
||||
finishes, and the client decides what happens next. A callback turn scripted
|
||||
after a paused turn becomes a continuation. It materializes when the follow-up
|
||||
request arrives and receives the resolved `toolCall` in its context.
|
||||
|
||||
For client-executed tools, the user supplies the output through
|
||||
`addToolOutput` and the continuation reads it:
|
||||
|
||||
```ts
|
||||
const chat = createChat()
|
||||
.user("Help me plan the release.")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("askQuestions", { dynamic: true, input: { questions } })
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(`Got it. Starting with ${toolCall?.output?.answers.direction}.`)
|
||||
})
|
||||
```
|
||||
|
||||
For approval-gated tools, `needsApproval` pauses behind the user's decision.
|
||||
`output` (or `errorText`) then means "stream this if approved"; denial streams
|
||||
`tool-output-denied` automatically:
|
||||
|
||||
```ts
|
||||
const chat = createChat()
|
||||
.user("Clean up old deployments.")
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("This will delete 3 deployments. Approve?")
|
||||
writer.tool("deleteDeployments", {
|
||||
input: { count: 3 },
|
||||
needsApproval: true,
|
||||
output: { deleted: 3 },
|
||||
})
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.approved
|
||||
? "Deleted 3 deployments."
|
||||
: "Okay, leaving them in place."
|
||||
)
|
||||
})
|
||||
```
|
||||
|
||||
Wire the client with the AI SDK's own triggers:
|
||||
`sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls` for
|
||||
client-executed tools, or `lastAssistantMessageIsCompleteWithApprovalResponses`
|
||||
plus `addToolApprovalResponse({ id: part.approval.id, approved })` for
|
||||
approvals.
|
||||
|
||||
Continuation callbacks must be pure; regenerating re-resolves them against the
|
||||
current transcript. `get()` stops before the first continuation turn, since a
|
||||
continuation has no message without a live transcript.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
|
||||
@@ -71,7 +71,8 @@
|
||||
"devDependencies": {
|
||||
"@tanstack/ai": "0.40.0",
|
||||
"@tanstack/ai-client": "0.20.0",
|
||||
"ai": "7.0.22",
|
||||
"ai": "7.0.0-canary.159",
|
||||
"zod": "3.25.76",
|
||||
"rimraf": "^6.0.1",
|
||||
"tsup": "^8.5.0",
|
||||
"typescript": "^5.9.2",
|
||||
|
||||
@@ -10,17 +10,35 @@ import {
|
||||
weatherOutput,
|
||||
weatherSuccess,
|
||||
} from "./test-utils"
|
||||
import type { DataParts, Tools } from "./test-utils"
|
||||
import type { DataParts, TestMessage, Tools } from "./test-utils"
|
||||
|
||||
testChatContract("AI SDK", () =>
|
||||
createChat().user("One").assistant("A").user("Two").assistant("B")
|
||||
)
|
||||
|
||||
describe("AI SDK chat", () => {
|
||||
it("narrows the continuation toolCall by name", () => {
|
||||
createChat<TestMessage>()
|
||||
.user("Weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", { input: { city: "San Francisco" } })
|
||||
})
|
||||
.assistant(({ toolCall }) => {
|
||||
if (toolCall?.name === "getWeather") {
|
||||
expectTypeOf(toolCall.input).toEqualTypeOf<
|
||||
Tools["getWeather"]["input"]
|
||||
>()
|
||||
expectTypeOf(toolCall.output).toEqualTypeOf<
|
||||
Tools["getWeather"]["output"] | undefined
|
||||
>()
|
||||
}
|
||||
})
|
||||
})
|
||||
|
||||
it("preserves the native message type through get and next", () => {
|
||||
type Metadata = { model: string }
|
||||
|
||||
const chat = createChat<Metadata, DataParts, Tools>()
|
||||
const chat = createChat<UIMessage<Metadata, DataParts, Tools>>()
|
||||
.user("Weather?", { metadata: { model: "test" } })
|
||||
.assistant("Sunny.", { metadata: { model: "test" } })
|
||||
const messages = chat.get()
|
||||
@@ -97,7 +115,7 @@ describe("AI SDK chat", () => {
|
||||
})
|
||||
|
||||
it("supports assistant ids and metadata in chat fixtures", () => {
|
||||
const chat = createChat<{ model: string }>()
|
||||
const chat = createChat<UIMessage<{ model: string }>>()
|
||||
.user("Hello")
|
||||
.assistant("Hey.", {
|
||||
id: "assistant-1",
|
||||
@@ -243,34 +261,32 @@ describe("AI SDK chat", () => {
|
||||
})
|
||||
|
||||
it("materializes assistant writer parts into the final chat transcript", () => {
|
||||
const chat = createChat<unknown, DataParts, Tools>().assistant(
|
||||
({ writer }) => {
|
||||
writer.reasoning("I need to call the weather tool.")
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-1",
|
||||
data: weatherLoading,
|
||||
})
|
||||
const chat = createChat<TestMessage>().assistant(({ writer }) => {
|
||||
writer.reasoning("I need to call the weather tool.")
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-1",
|
||||
data: weatherLoading,
|
||||
})
|
||||
|
||||
const weather = writer.tool("getWeather", {
|
||||
title: "Checking weather",
|
||||
input: {
|
||||
city: "San Francisco",
|
||||
},
|
||||
})
|
||||
const weather = writer.tool("getWeather", {
|
||||
title: "Checking weather",
|
||||
input: {
|
||||
city: "San Francisco",
|
||||
},
|
||||
})
|
||||
|
||||
writer.sleep(100)
|
||||
writer.sleep(100)
|
||||
|
||||
weather.output(weatherOutput)
|
||||
weather.output(weatherOutput)
|
||||
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-1",
|
||||
data: weatherSuccess,
|
||||
})
|
||||
writer.text("It's sunny and 72 degrees in San Francisco.")
|
||||
}
|
||||
)
|
||||
writer.data({
|
||||
type: "data-weather",
|
||||
id: "weather-1",
|
||||
data: weatherSuccess,
|
||||
})
|
||||
writer.text("It's sunny and 72 degrees in San Francisco.")
|
||||
})
|
||||
|
||||
const [message] = chat.get()
|
||||
|
||||
@@ -298,7 +314,7 @@ describe("AI SDK chat", () => {
|
||||
|
||||
it("produces identical transcripts for identical scripts", () => {
|
||||
function script() {
|
||||
return createChat<unknown, DataParts, Tools>()
|
||||
return createChat<TestMessage>()
|
||||
.user("Hello")
|
||||
.assistant(({ writer }) => {
|
||||
writer.reasoning("Checking the forecast.")
|
||||
@@ -336,7 +352,7 @@ describe("AI SDK chat", () => {
|
||||
})
|
||||
|
||||
it("hydrates a chat from an existing transcript", () => {
|
||||
const source = createChat<unknown, DataParts, Tools>()
|
||||
const source = createChat<TestMessage>()
|
||||
.user("What's the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer
|
||||
|
||||
@@ -10,14 +10,32 @@ import { createChatRuntime, wait } from "../core"
|
||||
import type { Chat, ChatOptions } from "../core"
|
||||
import { createAiSdkFormat } from "./format"
|
||||
|
||||
// Local equivalents of the AI SDK's InferUIMessage* utilities; not all of
|
||||
// them are exported from the pinned ai version.
|
||||
type MessageMetadata<UI_MESSAGE extends UIMessage> =
|
||||
UI_MESSAGE extends UIMessage<infer METADATA> ? METADATA : unknown
|
||||
|
||||
type MessageData<UI_MESSAGE extends UIMessage> =
|
||||
UI_MESSAGE extends UIMessage<unknown, infer DATA_PARTS>
|
||||
? DATA_PARTS
|
||||
: UIDataTypes
|
||||
|
||||
type MessageTools<UI_MESSAGE extends UIMessage> =
|
||||
UI_MESSAGE extends UIMessage<unknown, UIDataTypes, infer TOOLS>
|
||||
? TOOLS
|
||||
: UITools
|
||||
|
||||
/** Options for creating an AI SDK chat, optionally hydrated from existing messages. */
|
||||
export type CreateChatOptions<
|
||||
METADATA = unknown,
|
||||
DATA_PARTS extends UIDataTypes = UIDataTypes,
|
||||
TOOLS extends UITools = UITools,
|
||||
> = ChatOptions & {
|
||||
messages?: Array<UIMessage<METADATA, DATA_PARTS, TOOLS>>
|
||||
}
|
||||
export type CreateChatOptions<UI_MESSAGE extends UIMessage = UIMessage> =
|
||||
ChatOptions & {
|
||||
messages?: Array<
|
||||
UIMessage<
|
||||
MessageMetadata<UI_MESSAGE>,
|
||||
MessageData<UI_MESSAGE>,
|
||||
MessageTools<UI_MESSAGE>
|
||||
>
|
||||
>
|
||||
}
|
||||
|
||||
export type YieldMessagePartsOptions = {
|
||||
/** Also yield an initial snapshot with zero parts. */
|
||||
@@ -30,17 +48,23 @@ export type StreamMessagePartsOptions = YieldMessagePartsOptions & {
|
||||
}
|
||||
|
||||
/** The chat type returned by the AI SDK adapter's `createChat`. */
|
||||
export type AiSdkChat<
|
||||
METADATA = unknown,
|
||||
DATA_PARTS extends UIDataTypes = UIDataTypes,
|
||||
TOOLS extends UITools = UITools,
|
||||
> = Chat<
|
||||
UIMessage<METADATA, DATA_PARTS, TOOLS>,
|
||||
UIMessagePart<DATA_PARTS, TOOLS>,
|
||||
ChatTransport<UIMessage<METADATA, DATA_PARTS, TOOLS>>,
|
||||
METADATA,
|
||||
DATA_PARTS,
|
||||
TOOLS
|
||||
export type AiSdkChat<UI_MESSAGE extends UIMessage = UIMessage> = Chat<
|
||||
UIMessage<
|
||||
MessageMetadata<UI_MESSAGE>,
|
||||
MessageData<UI_MESSAGE>,
|
||||
MessageTools<UI_MESSAGE>
|
||||
>,
|
||||
UIMessagePart<MessageData<UI_MESSAGE>, MessageTools<UI_MESSAGE>>,
|
||||
ChatTransport<
|
||||
UIMessage<
|
||||
MessageMetadata<UI_MESSAGE>,
|
||||
MessageData<UI_MESSAGE>,
|
||||
MessageTools<UI_MESSAGE>
|
||||
>
|
||||
>,
|
||||
MessageMetadata<UI_MESSAGE>,
|
||||
MessageData<UI_MESSAGE>,
|
||||
MessageTools<UI_MESSAGE>
|
||||
>
|
||||
|
||||
/**
|
||||
@@ -96,16 +120,24 @@ export async function* streamMessageParts<
|
||||
}
|
||||
}
|
||||
|
||||
/** Creates an AI SDK chat, optionally hydrated from existing messages. */
|
||||
export function createChat<
|
||||
METADATA = unknown,
|
||||
DATA_PARTS extends UIDataTypes = UIDataTypes,
|
||||
TOOLS extends UITools = UITools,
|
||||
>(options: CreateChatOptions<METADATA, DATA_PARTS, TOOLS> = {}) {
|
||||
/**
|
||||
* Creates an AI SDK chat, optionally hydrated from existing messages. Generic
|
||||
* over the UI message type, mirroring `useChat`.
|
||||
*/
|
||||
export function createChat<UI_MESSAGE extends UIMessage = UIMessage>(
|
||||
options: CreateChatOptions<UI_MESSAGE> = {}
|
||||
) {
|
||||
const { messages, ...chatOptions } = options
|
||||
|
||||
return createChatRuntime(createAiSdkFormat<METADATA, DATA_PARTS, TOOLS>(), {
|
||||
...chatOptions,
|
||||
messages,
|
||||
})
|
||||
return createChatRuntime(
|
||||
createAiSdkFormat<
|
||||
MessageMetadata<UI_MESSAGE>,
|
||||
MessageData<UI_MESSAGE>,
|
||||
MessageTools<UI_MESSAGE>
|
||||
>(),
|
||||
{
|
||||
...chatOptions,
|
||||
messages,
|
||||
}
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,14 +1,18 @@
|
||||
import { readUIMessageStream } from "ai"
|
||||
import {
|
||||
lastAssistantMessageIsCompleteWithApprovalResponses,
|
||||
lastAssistantMessageIsCompleteWithToolCalls,
|
||||
readUIMessageStream,
|
||||
} from "ai"
|
||||
import type { ChatTransport, UIMessage } from "ai"
|
||||
import { describe, expect, it } from "vitest"
|
||||
|
||||
import { createChat } from "./index"
|
||||
import { readStream, weatherLoading, weatherOutput } from "./test-utils"
|
||||
import type { DataParts, Tools } from "./test-utils"
|
||||
import type { DataParts, TestMessage, Tools } from "./test-utils"
|
||||
|
||||
describe("AI SDK transport", () => {
|
||||
it("chains tool output after an in-stream sleep", async () => {
|
||||
const chat = createChat<unknown, DataParts, Tools>()
|
||||
const chat = createChat<TestMessage>()
|
||||
.user("Weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer
|
||||
@@ -42,7 +46,7 @@ describe("AI SDK transport", () => {
|
||||
})
|
||||
|
||||
it("streams denied static tool parts", async () => {
|
||||
const chat = createChat<unknown, DataParts, Tools>()
|
||||
const chat = createChat<TestMessage>()
|
||||
.user("Use the available tools.")
|
||||
.assistant([
|
||||
{
|
||||
@@ -88,7 +92,7 @@ describe("AI SDK transport", () => {
|
||||
})
|
||||
|
||||
it("streams the next scripted assistant response through the transport", async () => {
|
||||
const chat = createChat<unknown, DataParts, Tools>()
|
||||
const chat = createChat<TestMessage>()
|
||||
.user("Hello")
|
||||
.assistant("Hey, how's it going?")
|
||||
.user("What's the weather?")
|
||||
@@ -423,7 +427,7 @@ describe("AI SDK transport", () => {
|
||||
).toBeGreaterThan(1)
|
||||
})
|
||||
it("streams hydrated assistant turns through the transport", async () => {
|
||||
const source = createChat<unknown, DataParts, Tools>()
|
||||
const source = createChat<TestMessage>()
|
||||
.user("What's the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer
|
||||
@@ -434,7 +438,7 @@ describe("AI SDK transport", () => {
|
||||
writer.text("It's sunny.", { mode: "instant" })
|
||||
})
|
||||
|
||||
const replay = createChat<unknown, DataParts, Tools>({
|
||||
const replay = createChat<TestMessage>({
|
||||
messages: source.get(),
|
||||
})
|
||||
const [userMessage] = replay.get(1)
|
||||
@@ -682,3 +686,246 @@ describe("AI SDK live client integration", () => {
|
||||
expect(getText(regenerated)).toBe("Still nothing scripted.")
|
||||
})
|
||||
})
|
||||
|
||||
describe("AI SDK human-in-the-loop", () => {
|
||||
function createApprovalChat() {
|
||||
return createChat<TestMessage>()
|
||||
.user("Fetch the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("I need your approval first.", { mode: "instant" })
|
||||
writer.tool("getWeather", {
|
||||
input: { city: "San Francisco" },
|
||||
needsApproval: true,
|
||||
output: weatherOutput,
|
||||
})
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(toolCall?.approved ? "Fetched it." : "Okay, skipping.", {
|
||||
mode: "instant",
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
async function readTurnChunks(
|
||||
transport: ChatTransport<UIMessage<unknown, DataParts, Tools>>,
|
||||
messages: Array<UIMessage<unknown, DataParts, Tools>>
|
||||
) {
|
||||
// Mirrors useChat's automatic sends, which pass the last message's id
|
||||
// with the submit-message trigger. The transport must not treat that as
|
||||
// a regeneration of the identified turn.
|
||||
const stream = await transport.sendMessages({
|
||||
trigger: "submit-message",
|
||||
chatId: "chat-1",
|
||||
messageId: messages[messages.length - 1]?.id,
|
||||
messages,
|
||||
abortSignal: undefined,
|
||||
})
|
||||
|
||||
return readStream(stream)
|
||||
}
|
||||
|
||||
function respondToApproval(
|
||||
message: UIMessage<unknown, DataParts, Tools>,
|
||||
approved: boolean
|
||||
) {
|
||||
return {
|
||||
...message,
|
||||
parts: message.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? ({
|
||||
...part,
|
||||
state: "approval-responded",
|
||||
approval: { id: "approval-1", approved },
|
||||
} as (typeof message.parts)[number])
|
||||
: part
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
it("streams the approval request and the client parks the call", async () => {
|
||||
const chat = createApprovalChat()
|
||||
const chunks = await readTurnChunks(
|
||||
chat.transport({ delayMs: undefined }),
|
||||
chat.get(1)
|
||||
)
|
||||
|
||||
expect(chunks.map((chunk) => chunk.type)).toEqual([
|
||||
"start",
|
||||
"text-start",
|
||||
"text-delta",
|
||||
"text-end",
|
||||
"tool-input-available",
|
||||
"tool-approval-request",
|
||||
"finish",
|
||||
])
|
||||
expect(chunks[5]).toMatchObject({
|
||||
approvalId: "approval-1",
|
||||
toolCallId: "call-1",
|
||||
})
|
||||
|
||||
const stream = await chat.transport({ delayMs: undefined }).sendMessages({
|
||||
trigger: "submit-message",
|
||||
chatId: "chat-1",
|
||||
messageId: undefined,
|
||||
messages: chat.get(1),
|
||||
abortSignal: undefined,
|
||||
})
|
||||
let clientMessage: UIMessage<unknown, DataParts, Tools> | undefined
|
||||
|
||||
for await (const message of readUIMessageStream({ stream })) {
|
||||
clientMessage = message as UIMessage<unknown, DataParts, Tools>
|
||||
}
|
||||
|
||||
expect(clientMessage?.parts).toMatchObject([
|
||||
{ type: "text", text: "I need your approval first." },
|
||||
{
|
||||
type: "tool-getWeather",
|
||||
state: "approval-requested",
|
||||
approval: { id: "approval-1" },
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("streams the gated output and approved wording after approval", async () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const chunks = await readTurnChunks(
|
||||
chat.transport({ delayMs: undefined }),
|
||||
[userMessage, respondToApproval(assistantMessage, true)]
|
||||
)
|
||||
|
||||
expect(chunks.map((chunk) => chunk.type)).toEqual([
|
||||
"start",
|
||||
"start-step",
|
||||
"tool-output-available",
|
||||
"text-start",
|
||||
"text-delta",
|
||||
"text-end",
|
||||
"finish",
|
||||
])
|
||||
// Continuations omit the message id so the client updates the paused
|
||||
// assistant message in place instead of forking a duplicate.
|
||||
expect(chunks[0]).toMatchObject({ messageId: undefined })
|
||||
expect(chunks[2]).toMatchObject({
|
||||
toolCallId: "call-1",
|
||||
output: weatherOutput,
|
||||
})
|
||||
expect(chunks[4]).toMatchObject({ delta: "Fetched it." })
|
||||
})
|
||||
|
||||
it("streams the denial and denied wording after denial", async () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const chunks = await readTurnChunks(
|
||||
chat.transport({ delayMs: undefined }),
|
||||
[userMessage, respondToApproval(assistantMessage, false)]
|
||||
)
|
||||
|
||||
expect(chunks.map((chunk) => chunk.type)).toEqual([
|
||||
"start",
|
||||
"start-step",
|
||||
"tool-output-denied",
|
||||
"text-start",
|
||||
"text-delta",
|
||||
"text-end",
|
||||
"finish",
|
||||
])
|
||||
expect(chunks[2]).toMatchObject({ toolCallId: "call-1" })
|
||||
expect(chunks[4]).toMatchObject({ delta: "Okay, skipping." })
|
||||
})
|
||||
|
||||
it("does not retrigger automatic sending after a continuation merges", async () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const respondedMessage = respondToApproval(assistantMessage, true)
|
||||
const chunks = await readTurnChunks(
|
||||
chat.transport({ delayMs: undefined }),
|
||||
[userMessage, respondedMessage]
|
||||
)
|
||||
|
||||
// The client merges these chunks into the prior assistant message as a
|
||||
// new step. Rebuild that merged shape and assert the triggers ignore the
|
||||
// already-resolved tool call behind the step boundary.
|
||||
const mergedMessage = {
|
||||
...respondedMessage,
|
||||
parts: [
|
||||
...respondedMessage.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? ({
|
||||
...part,
|
||||
state: "output-available",
|
||||
output: weatherOutput,
|
||||
approval: { id: "approval-1", approved: true },
|
||||
} as (typeof respondedMessage.parts)[number])
|
||||
: part
|
||||
),
|
||||
{ type: "step-start" } as (typeof respondedMessage.parts)[number],
|
||||
{
|
||||
type: "text",
|
||||
text: "Fetched it.",
|
||||
} as (typeof respondedMessage.parts)[number],
|
||||
],
|
||||
}
|
||||
|
||||
expect(chunks.some((chunk) => chunk.type === "start-step")).toBe(true)
|
||||
expect(
|
||||
lastAssistantMessageIsCompleteWithToolCalls({
|
||||
messages: [userMessage, mergedMessage],
|
||||
})
|
||||
).toBe(false)
|
||||
expect(
|
||||
lastAssistantMessageIsCompleteWithApprovalResponses({
|
||||
messages: [userMessage, mergedMessage],
|
||||
})
|
||||
).toBe(false)
|
||||
})
|
||||
|
||||
it("continues an elicitation with the user-submitted output", async () => {
|
||||
const chat = createChat<TestMessage>()
|
||||
.user("Help me plan the release.")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("createFile", {
|
||||
dynamic: true,
|
||||
input: { filename: "plan.md", content: "" },
|
||||
})
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.name === "createFile" && toolCall.output
|
||||
? `Saved ${toolCall.output.filename}.`
|
||||
: "No file yet.",
|
||||
{ mode: "instant" }
|
||||
)
|
||||
})
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const answeredMessage = {
|
||||
...assistantMessage,
|
||||
parts: assistantMessage.parts.map((part) =>
|
||||
part.type === "dynamic-tool"
|
||||
? ({
|
||||
...part,
|
||||
state: "output-available",
|
||||
output: {
|
||||
filename: "plan.md",
|
||||
url: "https://example.com/plan.md",
|
||||
},
|
||||
} as (typeof assistantMessage.parts)[number])
|
||||
: part
|
||||
),
|
||||
}
|
||||
const chunks = await readTurnChunks(
|
||||
chat.transport({ delayMs: undefined }),
|
||||
[userMessage, answeredMessage]
|
||||
)
|
||||
|
||||
expect(chunks.map((chunk) => chunk.type)).toEqual([
|
||||
"start",
|
||||
"start-step",
|
||||
"text-start",
|
||||
"text-delta",
|
||||
"text-end",
|
||||
"finish",
|
||||
])
|
||||
expect(chunks[3]).toMatchObject({ delta: "Saved plan.md." })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -10,7 +10,12 @@ import type {
|
||||
|
||||
import { assertNever } from "../core"
|
||||
import type { ChatFormat, ChunkEncoder } from "../core"
|
||||
import { eventsFromParts, getMessageText, materializeParts } from "./parts"
|
||||
import {
|
||||
eventsFromParts,
|
||||
getMessageText,
|
||||
getToolCalls,
|
||||
materializeParts,
|
||||
} from "./parts"
|
||||
|
||||
/**
|
||||
* The AI SDK `ChatFormat` plugin. Pass to `createChatRuntime` for advanced
|
||||
@@ -92,6 +97,10 @@ export function createAiSdkFormat<
|
||||
return message.metadata
|
||||
},
|
||||
|
||||
getToolCalls(message) {
|
||||
return getToolCalls(message)
|
||||
},
|
||||
|
||||
encodeChunk(chunk) {
|
||||
switch (chunk.type) {
|
||||
case "start":
|
||||
@@ -172,6 +181,12 @@ export function createAiSdkFormat<
|
||||
dynamic: chunk.dynamic,
|
||||
title: chunk.title,
|
||||
} as Chunk
|
||||
case "tool-approval-request":
|
||||
return {
|
||||
type: "tool-approval-request",
|
||||
approvalId: chunk.approvalId,
|
||||
toolCallId: chunk.toolCallId,
|
||||
} as Chunk
|
||||
case "tool-output-available":
|
||||
return {
|
||||
type: "tool-output-available",
|
||||
@@ -230,8 +245,13 @@ export function createAiSdkFormat<
|
||||
|
||||
createTransport(transportContext, options = {}) {
|
||||
return {
|
||||
async sendMessages({ messages, messageId, abortSignal }) {
|
||||
const turn = transportContext.resolveTurn(messages, messageId)
|
||||
async sendMessages({ messages, messageId, abortSignal, trigger }) {
|
||||
// Automatic sends after tool results and approvals pass the last
|
||||
// assistant message's id; only regeneration may replay a turn by id.
|
||||
const turn = transportContext.resolveTurn(
|
||||
messages,
|
||||
trigger === "regenerate-message" ? messageId : undefined
|
||||
)
|
||||
|
||||
if (!turn) {
|
||||
throw new Error("No assistant response found for this transcript.")
|
||||
|
||||
@@ -2,17 +2,18 @@ import { describe, expect, it } from "vitest"
|
||||
|
||||
import { streamMessageParts, yieldMessageParts } from "./chat"
|
||||
import { createChat } from "./index"
|
||||
import { getToolCalls } from "./parts"
|
||||
import {
|
||||
artifact,
|
||||
weatherLoading,
|
||||
weatherOutput,
|
||||
weatherSuccess,
|
||||
} from "./test-utils"
|
||||
import type { DataParts, Tools } from "./test-utils"
|
||||
import type { DataParts, TestMessage, Tools } from "./test-utils"
|
||||
|
||||
describe("AI SDK parts", () => {
|
||||
it("creates typed messages with explicit parts", () => {
|
||||
const [message] = createChat<unknown, DataParts, Tools>()
|
||||
const [message] = createChat<TestMessage>()
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("Here is the forecast.")
|
||||
writer.data({
|
||||
@@ -56,7 +57,7 @@ describe("AI SDK parts", () => {
|
||||
})
|
||||
|
||||
it("creates rich messages with the fluent writer", () => {
|
||||
const [message] = createChat<unknown, DataParts, Tools>()
|
||||
const [message] = createChat<TestMessage>()
|
||||
.assistant(
|
||||
({ writer }) => {
|
||||
writer.stepStart()
|
||||
@@ -135,7 +136,7 @@ describe("AI SDK parts", () => {
|
||||
})
|
||||
|
||||
it("materializes all writer part variants", () => {
|
||||
const [message] = createChat<unknown, DataParts, Tools>()
|
||||
const [message] = createChat<TestMessage>()
|
||||
.assistant(
|
||||
({ writer }) => {
|
||||
writer.stepStart()
|
||||
@@ -197,3 +198,73 @@ describe("AI SDK parts", () => {
|
||||
)
|
||||
})
|
||||
})
|
||||
|
||||
describe("AI SDK approval parts", () => {
|
||||
function createApprovalMessage() {
|
||||
const [, assistantMessage] = createChat<TestMessage>()
|
||||
.user("Fetch the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", {
|
||||
input: { city: "San Francisco" },
|
||||
needsApproval: true,
|
||||
output: weatherOutput,
|
||||
})
|
||||
})
|
||||
.get(2)
|
||||
|
||||
return assistantMessage
|
||||
}
|
||||
|
||||
it("materializes a needsApproval call as an approval-requested part", () => {
|
||||
expect(createApprovalMessage().parts).toMatchObject([
|
||||
{
|
||||
type: "tool-getWeather",
|
||||
toolCallId: "call-1",
|
||||
state: "approval-requested",
|
||||
input: { city: "San Francisco" },
|
||||
approval: { id: "approval-1" },
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("summarizes tool calls with their approval state", () => {
|
||||
const message = createApprovalMessage()
|
||||
const respondedMessage = {
|
||||
...message,
|
||||
parts: message.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? ({
|
||||
...part,
|
||||
state: "approval-responded",
|
||||
approval: { id: "approval-1", approved: true },
|
||||
} as (typeof message.parts)[number])
|
||||
: part
|
||||
),
|
||||
}
|
||||
|
||||
expect(getToolCalls(respondedMessage)).toMatchObject([
|
||||
{
|
||||
toolCallId: "call-1",
|
||||
name: "getWeather",
|
||||
state: "approval-responded",
|
||||
input: { city: "San Francisco" },
|
||||
approval: { id: "approval-1", approved: true },
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("hydrates approval-requested parts back into approval events", () => {
|
||||
const chat = createChat<TestMessage>({
|
||||
messages: [createApprovalMessage()],
|
||||
})
|
||||
const [message] = chat.get(1)
|
||||
|
||||
expect(message.parts).toMatchObject([
|
||||
{
|
||||
type: "tool-getWeather",
|
||||
state: "approval-requested",
|
||||
approval: { id: "approval-1" },
|
||||
},
|
||||
])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -16,7 +16,7 @@ import type {
|
||||
} from "ai"
|
||||
|
||||
import { assertNever } from "../core"
|
||||
import type { ChatEvent } from "../core"
|
||||
import type { ChatEvent, ToolCallSummary } from "../core"
|
||||
|
||||
/** Concatenates the text of every `text` part in a message. */
|
||||
export function getMessageText(message: Pick<UIMessage, "parts">) {
|
||||
@@ -26,6 +26,56 @@ export function getMessageText(message: Pick<UIMessage, "parts">) {
|
||||
.join("")
|
||||
}
|
||||
|
||||
function toToolCallSummary(
|
||||
name: string,
|
||||
part: ToolUIPart | DynamicToolUIPart,
|
||||
dynamic?: boolean
|
||||
): ToolCallSummary {
|
||||
return {
|
||||
toolCallId: part.toolCallId,
|
||||
name,
|
||||
dynamic,
|
||||
input: "input" in part ? part.input : undefined,
|
||||
output: part.state === "output-available" ? part.output : undefined,
|
||||
errorText: part.state === "output-error" ? part.errorText : undefined,
|
||||
state: part.state,
|
||||
approval:
|
||||
"approval" in part && part.approval
|
||||
? {
|
||||
id: part.approval.id,
|
||||
approved: part.approval.approved,
|
||||
reason: part.approval.reason,
|
||||
}
|
||||
: undefined,
|
||||
}
|
||||
}
|
||||
|
||||
/** Reads a message's typed and dynamic tool calls into framework-neutral summaries. */
|
||||
export function getToolCalls(message: Pick<UIMessage, "parts">) {
|
||||
const summaries: ToolCallSummary[] = []
|
||||
|
||||
for (const part of message.parts) {
|
||||
if (part.type === "dynamic-tool") {
|
||||
const dynamicToolPart = part as DynamicToolUIPart
|
||||
|
||||
summaries.push(
|
||||
toToolCallSummary(dynamicToolPart.toolName, dynamicToolPart, true)
|
||||
)
|
||||
continue
|
||||
}
|
||||
|
||||
if (part.type.startsWith("tool-") && "toolCallId" in part) {
|
||||
const toolPart = part as ToolUIPart
|
||||
|
||||
summaries.push(
|
||||
toToolCallSummary(part.type.replace("tool-", ""), toolPart)
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
return summaries
|
||||
}
|
||||
|
||||
/** Replaces the part with the same `type` + `id` in place, or appends. */
|
||||
export function replaceOrPushPart<
|
||||
DATA_PARTS extends UIDataTypes,
|
||||
@@ -115,6 +165,18 @@ export function materializeParts<
|
||||
} as UIMessagePart<DATA_PARTS, TOOLS>)
|
||||
break
|
||||
}
|
||||
case "tool-approval-request": {
|
||||
const index = findToolPartIndex(parts, event.toolCallId)
|
||||
|
||||
if (index !== -1) {
|
||||
parts[index] = {
|
||||
...parts[index],
|
||||
state: "approval-requested",
|
||||
approval: { id: event.approvalId },
|
||||
} as UIMessagePart<DATA_PARTS, TOOLS>
|
||||
}
|
||||
break
|
||||
}
|
||||
case "tool-output": {
|
||||
const index = findToolPartIndex(parts, event.toolCallId)
|
||||
|
||||
@@ -266,6 +328,17 @@ export function eventsFromParts<
|
||||
input: "input" in toolPart ? toolPart.input : {},
|
||||
})
|
||||
|
||||
if (
|
||||
toolPart.state === "approval-requested" ||
|
||||
toolPart.state === "approval-responded"
|
||||
) {
|
||||
events.push({
|
||||
kind: "tool-approval-request",
|
||||
toolCallId: toolPart.toolCallId,
|
||||
approvalId: toolPart.approval.id,
|
||||
})
|
||||
}
|
||||
|
||||
if (toolPart.state === "output-available") {
|
||||
events.push({
|
||||
kind: "tool-output",
|
||||
@@ -308,6 +381,17 @@ export function eventsFromParts<
|
||||
input: "input" in dynamicToolPart ? dynamicToolPart.input : {},
|
||||
})
|
||||
|
||||
if (
|
||||
dynamicToolPart.state === "approval-requested" ||
|
||||
dynamicToolPart.state === "approval-responded"
|
||||
) {
|
||||
events.push({
|
||||
kind: "tool-approval-request",
|
||||
toolCallId: dynamicToolPart.toolCallId,
|
||||
approvalId: dynamicToolPart.approval.id,
|
||||
})
|
||||
}
|
||||
|
||||
if (dynamicToolPart.state === "output-available") {
|
||||
events.push({
|
||||
kind: "tool-output",
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
import type { UIMessage } from "ai"
|
||||
|
||||
export type TestMessage = UIMessage<unknown, DataParts, Tools>
|
||||
|
||||
export type DataParts = {
|
||||
weather: {
|
||||
city: string
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { describe, expect, it } from "vitest"
|
||||
import { describe, expect, it, vi } from "vitest"
|
||||
|
||||
import { createChatRuntime } from "./chat"
|
||||
import { createTestChat, createTestFormat, readStream } from "./test-utils"
|
||||
@@ -373,3 +373,249 @@ describe("createChatRuntime hydration and fallback", () => {
|
||||
expect(chat.get()).toHaveLength(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe("human-in-the-loop turns", () => {
|
||||
function createApprovalChat() {
|
||||
return createTestChat()
|
||||
.user("Fetch the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", {
|
||||
input: { city: "Paris" },
|
||||
needsApproval: true,
|
||||
output: { temperature: 21 },
|
||||
})
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(toolCall?.approved ? "Fetched it." : "Okay, skipping.")
|
||||
})
|
||||
}
|
||||
|
||||
function withApprovalResponse(message: TestMessage, approved: boolean) {
|
||||
return {
|
||||
...message,
|
||||
parts: message.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? {
|
||||
...part,
|
||||
state: "approval-responded",
|
||||
approval: { id: "approval-1", approved },
|
||||
}
|
||||
: part
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
it("scripts the approval request and pauses the turn", () => {
|
||||
const chat = createApprovalChat()
|
||||
const turn = chat.transport().context.resolveTurn(chat.get(1))
|
||||
|
||||
expect(turn?.events).toMatchObject([
|
||||
{ kind: "tool-input", toolCallId: "call-1" },
|
||||
{
|
||||
kind: "tool-approval-request",
|
||||
toolCallId: "call-1",
|
||||
approvalId: "approval-1",
|
||||
output: { temperature: 21 },
|
||||
},
|
||||
])
|
||||
expect(turn?.message.parts).toMatchObject([
|
||||
{
|
||||
type: "tool-getWeather",
|
||||
state: "approval-requested",
|
||||
approval: { id: "approval-1" },
|
||||
},
|
||||
])
|
||||
})
|
||||
|
||||
it("bounds get() at the first continuation turn", () => {
|
||||
const chat = createApprovalChat()
|
||||
|
||||
expect(chat.get()).toHaveLength(2)
|
||||
expect(() => chat.get(3)).toThrow(/continuation/)
|
||||
})
|
||||
|
||||
it("streams the gated output and approved context after approval", () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const turn = chat
|
||||
.transport()
|
||||
.context.resolveTurn([
|
||||
userMessage,
|
||||
withApprovalResponse(assistantMessage, true),
|
||||
])
|
||||
|
||||
expect(turn?.message.id).toBe("msg-3")
|
||||
expect(turn?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{
|
||||
kind: "tool-output",
|
||||
toolCallId: "call-1",
|
||||
output: { temperature: 21 },
|
||||
},
|
||||
{ kind: "text", text: "Fetched it." },
|
||||
])
|
||||
})
|
||||
|
||||
it("streams the denial and denied context after denial", () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const turn = chat
|
||||
.transport()
|
||||
.context.resolveTurn([
|
||||
userMessage,
|
||||
withApprovalResponse(assistantMessage, false),
|
||||
])
|
||||
|
||||
expect(turn?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{ kind: "tool-denied", toolCallId: "call-1" },
|
||||
{ kind: "text", text: "Okay, skipping." },
|
||||
])
|
||||
})
|
||||
|
||||
it("re-resolves a continuation to the same message", () => {
|
||||
const chat = createApprovalChat()
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const transcript = [
|
||||
userMessage,
|
||||
withApprovalResponse(assistantMessage, true),
|
||||
]
|
||||
const { context } = chat.transport()
|
||||
const first = context.resolveTurn(transcript)
|
||||
const second = context.resolveTurn(transcript)
|
||||
|
||||
expect(second?.message).toEqual(first?.message)
|
||||
})
|
||||
|
||||
it("reads a user-submitted elicitation output into the continuation", () => {
|
||||
const chat = createTestChat()
|
||||
.user("Plan the release?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", { input: { city: "Paris" } })
|
||||
})
|
||||
.assistant(({ writer, toolCall }) => {
|
||||
writer.text(
|
||||
toolCall?.name === "getWeather" && toolCall.output
|
||||
? `Temperature ${toolCall.output.temperature}.`
|
||||
: "No answer."
|
||||
)
|
||||
})
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const answeredMessage = {
|
||||
...assistantMessage,
|
||||
parts: assistantMessage.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? { ...part, state: "output-available", output: { temperature: 72 } }
|
||||
: part
|
||||
),
|
||||
}
|
||||
const turn = chat
|
||||
.transport()
|
||||
.context.resolveTurn([userMessage, answeredMessage])
|
||||
|
||||
expect(turn?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{ kind: "text", text: "Temperature 72." },
|
||||
])
|
||||
})
|
||||
|
||||
it("chains continuations and warns when nothing is pending", () => {
|
||||
const warn = vi.spyOn(console, "warn").mockImplementation(() => {})
|
||||
const chat = createTestChat()
|
||||
.user("Plan the release?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", { input: { city: "Paris" } })
|
||||
})
|
||||
.assistant(({ writer }) => {
|
||||
writer.text("Thanks.")
|
||||
})
|
||||
.assistant(({ writer, toolCalls }) => {
|
||||
writer.text(`Pending ${toolCalls?.length ?? 0}.`)
|
||||
})
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const answeredMessage = {
|
||||
...assistantMessage,
|
||||
parts: assistantMessage.parts.map((part) =>
|
||||
part.type === "tool-getWeather"
|
||||
? { ...part, state: "output-available", output: { temperature: 72 } }
|
||||
: part
|
||||
),
|
||||
}
|
||||
const { context } = chat.transport()
|
||||
const firstContinuation = context.resolveTurn([
|
||||
userMessage,
|
||||
answeredMessage,
|
||||
])
|
||||
const secondContinuation = context.resolveTurn([
|
||||
userMessage,
|
||||
answeredMessage,
|
||||
firstContinuation!.message,
|
||||
])
|
||||
|
||||
expect(firstContinuation?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{ kind: "text", text: "Thanks." },
|
||||
])
|
||||
expect(secondContinuation?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{ kind: "text", text: "Pending 0." },
|
||||
])
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining("without a pending tool call")
|
||||
)
|
||||
warn.mockRestore()
|
||||
})
|
||||
|
||||
it("starts eager continuation turns with a step boundary", () => {
|
||||
const chat = createTestChat()
|
||||
.user("Plan the release?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", { input: { city: "Paris" } })
|
||||
})
|
||||
.assistant("Thanks.")
|
||||
const [userMessage, assistantMessage] = chat.get(2)
|
||||
const turn = chat
|
||||
.transport()
|
||||
.context.resolveTurn([userMessage, assistantMessage])
|
||||
|
||||
expect(turn?.events).toMatchObject([
|
||||
{ kind: "step-start" },
|
||||
{ kind: "text", text: "Thanks." },
|
||||
])
|
||||
})
|
||||
|
||||
it("warns when an approval request has no continuation turn", () => {
|
||||
const warn = vi.spyOn(console, "warn").mockImplementation(() => {})
|
||||
|
||||
createTestChat()
|
||||
.user("Fetch the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
writer.tool("getWeather", {
|
||||
input: { city: "Paris" },
|
||||
needsApproval: true,
|
||||
output: { temperature: 21 },
|
||||
})
|
||||
})
|
||||
.transport()
|
||||
|
||||
expect(warn).toHaveBeenCalledWith(
|
||||
expect.stringContaining("no continuation turn follows")
|
||||
)
|
||||
warn.mockRestore()
|
||||
})
|
||||
|
||||
it("throws when a needsApproval handle is resolved by the script", () => {
|
||||
createTestChat()
|
||||
.user("Fetch the weather?")
|
||||
.assistant(({ writer }) => {
|
||||
const handle = writer.tool("getWeather", {
|
||||
input: { city: "Paris" },
|
||||
needsApproval: true,
|
||||
})
|
||||
|
||||
expect(() => handle.output({ temperature: 21 })).toThrow(
|
||||
/needsApproval/
|
||||
)
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
@@ -10,10 +10,17 @@ import type {
|
||||
ChatTurn,
|
||||
ChatUserOptions,
|
||||
DataTypes,
|
||||
MessageRole,
|
||||
PendingToolCall,
|
||||
ToolSet,
|
||||
TurnStreamOptions,
|
||||
} from "./types"
|
||||
import { cloneValue, DEFAULT_STREAM_DELAY_MS, DEFAULT_TEXT } from "./utils"
|
||||
import {
|
||||
cloneValue,
|
||||
DEFAULT_STREAM_DELAY_MS,
|
||||
DEFAULT_TEXT,
|
||||
devWarn,
|
||||
} from "./utils"
|
||||
import { createEventWriter } from "./writer"
|
||||
import type { EventWriter } from "./writer"
|
||||
|
||||
@@ -50,6 +57,23 @@ export type ChatTransportOptions<
|
||||
fallback?: ChatFallback<MESSAGE, PART, DATA, TOOLS, WRITER>
|
||||
}
|
||||
|
||||
/**
|
||||
* The context handed to assistant writer callbacks. A continuation turn — a
|
||||
* callback turn scripted after a turn that pauses for user input — also
|
||||
* receives the live transcript and the resolved human-in-the-loop tool
|
||||
* calls; eager turns receive only the writer.
|
||||
*/
|
||||
export type AssistantTurnContext<
|
||||
WRITER,
|
||||
MESSAGE,
|
||||
TOOLS extends ToolSet = ToolSet,
|
||||
> = {
|
||||
writer: WRITER
|
||||
messages?: MESSAGE[]
|
||||
toolCall?: PendingToolCall<TOOLS>
|
||||
toolCalls?: Array<PendingToolCall<TOOLS>>
|
||||
}
|
||||
|
||||
/** A deterministic conversation: chain turns, then read them back or stream them through a transport. */
|
||||
export type Chat<
|
||||
MESSAGE,
|
||||
@@ -68,10 +92,16 @@ export type Chat<
|
||||
/**
|
||||
* Scripts an assistant turn from a string (one streamed text part), a
|
||||
* parts array (static parts, streamed instantly), or a writer callback
|
||||
* (full control over parts and timing).
|
||||
* (full control over parts and timing). A callback turn scripted after a
|
||||
* paused turn — an unresolved tool call or `needsApproval` — becomes a
|
||||
* continuation: it materializes when the follow-up request arrives, with
|
||||
* the live transcript and resolved tool calls in its context.
|
||||
*/
|
||||
assistant(
|
||||
input?: string | PART[] | ((context: { writer: WRITER }) => void),
|
||||
input?:
|
||||
| string
|
||||
| PART[]
|
||||
| ((context: AssistantTurnContext<WRITER, MESSAGE, TOOLS>) => void),
|
||||
options?: ChatAssistantOptions<METADATA>
|
||||
): Chat<MESSAGE, PART, TRANSPORT, METADATA, DATA, TOOLS, WRITER>
|
||||
/** Scripts an assistant turn that streams a bare error chunk and ends. */
|
||||
@@ -82,7 +112,12 @@ export type Chat<
|
||||
sleep(
|
||||
delayMs: number
|
||||
): Chat<MESSAGE, PART, TRANSPORT, METADATA, DATA, TOOLS, WRITER>
|
||||
/** Returns clones of the first `count` configured messages, or all messages when omitted. */
|
||||
/**
|
||||
* Returns clones of the first `count` configured messages, or all
|
||||
* materializable messages when omitted. Continuation turns have no message
|
||||
* without a live transcript, so `get` stops before the first one and
|
||||
* throws when `count` reaches past it.
|
||||
*/
|
||||
get(count?: number): MESSAGE[]
|
||||
/**
|
||||
* Returns the next configured user message after the given transcript, or
|
||||
@@ -117,8 +152,26 @@ export function createChatRuntime<
|
||||
messages?: MESSAGE[]
|
||||
} = {}
|
||||
): Chat<MESSAGE, PART, TRANSPORT, METADATA, DATA, TOOLS, WRITER> {
|
||||
type InternalTurn = ChatTurn<MESSAGE, DATA, TOOLS> & {
|
||||
type InternalTurn = {
|
||||
role: MessageRole
|
||||
events: ChatEvent<DATA, TOOLS>[]
|
||||
messageId: string
|
||||
metadata?: METADATA
|
||||
// Continuation turns carry `resolve` instead of a materialized message.
|
||||
message?: MESSAGE
|
||||
resolve?: (context: AssistantTurnContext<WRITER, MESSAGE, TOOLS>) => void
|
||||
// Continuations stream without a message id so the client keeps merging
|
||||
// into the assistant message it is already continuing.
|
||||
continuation?: boolean
|
||||
// The events last streamed for a continuation turn. Later continuations
|
||||
// read pending calls from here so re-runs cannot drift generated ids
|
||||
// away from the transcript.
|
||||
lastEvents?: ChatEvent<DATA, TOOLS>[]
|
||||
}
|
||||
|
||||
type ResolvedTurn = ChatTurn<MESSAGE, DATA, TOOLS> & {
|
||||
metadata?: METADATA
|
||||
continuation?: boolean
|
||||
}
|
||||
|
||||
const ids = createChatIds(options)
|
||||
@@ -134,17 +187,56 @@ export function createChatRuntime<
|
||||
event.kind === "tool-input" ||
|
||||
event.kind === "tool-output" ||
|
||||
event.kind === "tool-error" ||
|
||||
event.kind === "tool-denied"
|
||||
event.kind === "tool-denied" ||
|
||||
event.kind === "tool-approval-request"
|
||||
) {
|
||||
ids.reserveToolCallId(event.toolCallId)
|
||||
}
|
||||
|
||||
if (event.kind === "tool-approval-request") {
|
||||
ids.reserveApprovalId(event.approvalId)
|
||||
}
|
||||
|
||||
if (event.kind === "source-url" || event.kind === "source-document") {
|
||||
ids.reserveSourceId(event.part.sourceId)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function getPendingToolInputs(events: ChatEvent<DATA, TOOLS>[]) {
|
||||
const resolvedToolCallIds = new Set<string>()
|
||||
|
||||
for (const event of events) {
|
||||
if (
|
||||
event.kind === "tool-output" ||
|
||||
event.kind === "tool-error" ||
|
||||
event.kind === "tool-denied"
|
||||
) {
|
||||
resolvedToolCallIds.add(event.toolCallId)
|
||||
}
|
||||
}
|
||||
|
||||
return events.flatMap((event) =>
|
||||
event.kind === "tool-input" && !resolvedToolCallIds.has(event.toolCallId)
|
||||
? [
|
||||
{
|
||||
input: event,
|
||||
approval: events.find(
|
||||
(
|
||||
candidate
|
||||
): candidate is Extract<
|
||||
ChatEvent<DATA, TOOLS>,
|
||||
{ kind: "tool-approval-request" }
|
||||
> =>
|
||||
candidate.kind === "tool-approval-request" &&
|
||||
candidate.toolCallId === event.toolCallId
|
||||
),
|
||||
},
|
||||
]
|
||||
: []
|
||||
)
|
||||
}
|
||||
|
||||
function resolveMessageId(id?: string) {
|
||||
if (id !== undefined) {
|
||||
ids.reserveMessageId(id)
|
||||
@@ -165,6 +257,7 @@ export function createChatRuntime<
|
||||
turns.push({
|
||||
role: format.getMessageRole(message),
|
||||
message,
|
||||
messageId: format.getMessageId(message),
|
||||
events,
|
||||
metadata: format.getMessageMetadata?.(message),
|
||||
})
|
||||
@@ -187,7 +280,7 @@ export function createChatRuntime<
|
||||
let latestChatIndex = -1
|
||||
|
||||
for (let index = 0; index < turns.length; index++) {
|
||||
if (messageIds.has(format.getMessageId(turns[index].message))) {
|
||||
if (messageIds.has(turns[index].messageId)) {
|
||||
latestChatIndex = index
|
||||
}
|
||||
}
|
||||
@@ -198,8 +291,8 @@ export function createChatRuntime<
|
||||
latestChatIndex = turns.findIndex(
|
||||
(turn) =>
|
||||
lastMessage !== undefined &&
|
||||
format.getMessageRole(turn.message) ===
|
||||
format.getMessageRole(lastMessage) &&
|
||||
turn.message !== undefined &&
|
||||
turn.role === format.getMessageRole(lastMessage) &&
|
||||
format.getMessageText(turn.message) ===
|
||||
format.getMessageText(lastMessage)
|
||||
)
|
||||
@@ -208,32 +301,33 @@ export function createChatRuntime<
|
||||
return latestChatIndex
|
||||
}
|
||||
|
||||
function findNextAssistantTurn(messages: MESSAGE[], messageId?: string) {
|
||||
if (messageId) {
|
||||
const index = turns.findIndex(
|
||||
(turn) => format.getMessageId(turn.message) === messageId
|
||||
)
|
||||
const turn = turns[index]
|
||||
function findAssistantTurnIndexAfter(startIndex: number) {
|
||||
for (let index = startIndex; index < turns.length; index++) {
|
||||
if (turns[index].role === "assistant") {
|
||||
return index
|
||||
}
|
||||
}
|
||||
|
||||
if (turn?.role === "assistant") {
|
||||
return turn
|
||||
return -1
|
||||
}
|
||||
|
||||
function findNextAssistantTurnIndex(messages: MESSAGE[], messageId?: string) {
|
||||
if (messageId) {
|
||||
const index = turns.findIndex((turn) => turn.messageId === messageId)
|
||||
|
||||
if (turns[index]?.role === "assistant") {
|
||||
return index
|
||||
}
|
||||
|
||||
// Unknown ids fall through to transcript matching so regenerating a
|
||||
// fallback-produced message falls back again instead of replaying the
|
||||
// first configured response.
|
||||
if (index !== -1) {
|
||||
return turns
|
||||
.slice(index + 1)
|
||||
.find((nextTurn) => nextTurn.role === "assistant")
|
||||
return findAssistantTurnIndexAfter(index + 1)
|
||||
}
|
||||
}
|
||||
|
||||
const latestChatIndex = findLatestChatIndex(messages)
|
||||
|
||||
return turns
|
||||
.slice(latestChatIndex + 1)
|
||||
.find((turn) => turn.role === "assistant")
|
||||
return findAssistantTurnIndexAfter(findLatestChatIndex(messages) + 1)
|
||||
}
|
||||
|
||||
function findNextUserTurn(messages: readonly MESSAGE[]) {
|
||||
@@ -243,7 +337,10 @@ export function createChatRuntime<
|
||||
}
|
||||
|
||||
function materializeAssistantInput(
|
||||
input: string | PART[] | ((context: { writer: WRITER }) => void),
|
||||
input:
|
||||
| string
|
||||
| PART[]
|
||||
| ((context: AssistantTurnContext<WRITER, MESSAGE, TOOLS>) => void),
|
||||
events: ChatEvent<DATA, TOOLS>[]
|
||||
): PART[] {
|
||||
if (typeof input === "string") {
|
||||
@@ -281,7 +378,7 @@ export function createChatRuntime<
|
||||
function createFallbackTurn(
|
||||
fallback: ChatFallback<MESSAGE, PART, DATA, TOOLS, WRITER>,
|
||||
messages: MESSAGE[]
|
||||
): InternalTurn {
|
||||
): ResolvedTurn {
|
||||
const events: ChatEvent<DATA, TOOLS>[] = []
|
||||
const input =
|
||||
typeof fallback === "function"
|
||||
@@ -303,6 +400,185 @@ export function createChatRuntime<
|
||||
}
|
||||
}
|
||||
|
||||
function findPreviousAssistantIndex(turnIndex: number) {
|
||||
for (let index = turnIndex - 1; index >= 0; index--) {
|
||||
if (turns[index].role === "assistant") {
|
||||
return index
|
||||
}
|
||||
}
|
||||
|
||||
return -1
|
||||
}
|
||||
|
||||
function isPreviousTurnPausedAssistant() {
|
||||
const previousTurn = turns[turns.length - 1]
|
||||
|
||||
if (previousTurn?.role !== "assistant") {
|
||||
return false
|
||||
}
|
||||
|
||||
// A continuation may itself pause, so turns after one stay continuations.
|
||||
return (
|
||||
previousTurn.resolve !== undefined ||
|
||||
getPendingToolInputs(previousTurn.events).length > 0
|
||||
)
|
||||
}
|
||||
|
||||
function resolvePendingToolCalls(turnIndex: number, messages: MESSAGE[]) {
|
||||
const previousIndex = findPreviousAssistantIndex(turnIndex)
|
||||
const previousTurn = turns[previousIndex]
|
||||
|
||||
if (!previousTurn) {
|
||||
return []
|
||||
}
|
||||
|
||||
const previousEvents = previousTurn.resolve
|
||||
? (previousTurn.lastEvents ??
|
||||
materializeDeferredTurn(previousIndex, messages).events)
|
||||
: previousTurn.events
|
||||
const pendingInputs = getPendingToolInputs(previousEvents)
|
||||
|
||||
if (!pendingInputs.length) {
|
||||
return []
|
||||
}
|
||||
|
||||
const previousMessage = messages.find(
|
||||
(message) => format.getMessageId(message) === previousTurn.messageId
|
||||
)
|
||||
const summaries =
|
||||
previousMessage && format.getToolCalls
|
||||
? format.getToolCalls(previousMessage)
|
||||
: []
|
||||
|
||||
return pendingInputs.map((pendingInput) => ({
|
||||
...pendingInput,
|
||||
summary: summaries.find(
|
||||
(summary) => summary.toolCallId === pendingInput.input.toolCallId
|
||||
),
|
||||
}))
|
||||
}
|
||||
|
||||
function toPendingToolCall(
|
||||
call: ReturnType<typeof resolvePendingToolCalls>[number]
|
||||
) {
|
||||
const approved = call.approval
|
||||
? call.summary?.approval?.approved
|
||||
: undefined
|
||||
|
||||
return {
|
||||
name: call.input.name,
|
||||
toolCallId: call.input.toolCallId,
|
||||
input: call.summary?.input ?? call.input.input,
|
||||
output: call.approval
|
||||
? approved === true
|
||||
? call.approval.output
|
||||
: undefined
|
||||
: call.summary?.state === "output-available"
|
||||
? call.summary.output
|
||||
: undefined,
|
||||
...(call.approval
|
||||
? {
|
||||
approved: approved === true,
|
||||
denied: approved === false,
|
||||
}
|
||||
: {}),
|
||||
} as PendingToolCall<TOOLS>
|
||||
}
|
||||
|
||||
function materializeDeferredTurn(
|
||||
turnIndex: number,
|
||||
messages: MESSAGE[]
|
||||
): ResolvedTurn {
|
||||
const turn = turns[turnIndex]
|
||||
const resolve = turn.resolve
|
||||
|
||||
if (!resolve) {
|
||||
throw new Error("Only continuation turns materialize from a transcript.")
|
||||
}
|
||||
|
||||
const events = [...turn.events]
|
||||
|
||||
// The client merges a continuation into the prior assistant message as a
|
||||
// new step. The step boundary keeps the resolved tool call out of the
|
||||
// latest step so automatic sending does not retrigger on it.
|
||||
events.push({ kind: "step-start" })
|
||||
|
||||
const pending = resolvePendingToolCalls(turnIndex, messages)
|
||||
|
||||
if (!pending.length) {
|
||||
devWarn(
|
||||
"A continuation turn resolved without a pending tool call. Its toolCall context is empty."
|
||||
)
|
||||
}
|
||||
|
||||
for (const call of pending) {
|
||||
if (!call.approval || call.summary?.state !== "approval-responded") {
|
||||
continue
|
||||
}
|
||||
|
||||
if (call.summary.approval?.approved) {
|
||||
if (call.approval.errorText !== undefined) {
|
||||
events.push({
|
||||
kind: "tool-error",
|
||||
toolCallId: call.input.toolCallId,
|
||||
errorText: call.approval.errorText,
|
||||
providerExecuted: call.input.providerExecuted,
|
||||
toolMetadata: call.input.toolMetadata,
|
||||
dynamic: call.input.dynamic,
|
||||
})
|
||||
} else {
|
||||
events.push({
|
||||
kind: "tool-output",
|
||||
toolCallId: call.input.toolCallId,
|
||||
output: call.approval.output,
|
||||
providerExecuted: call.input.providerExecuted,
|
||||
toolMetadata: call.input.toolMetadata,
|
||||
dynamic: call.input.dynamic,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
events.push({
|
||||
kind: "tool-denied",
|
||||
toolCallId: call.input.toolCallId,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const toolCalls = pending.map(toPendingToolCall)
|
||||
// Same cast point as materializeAssistantInput: the full writer passes
|
||||
// through the adapter's narrower callback type.
|
||||
const writer = createEventWriter(events, {
|
||||
ids,
|
||||
payloads,
|
||||
}) as unknown as WRITER
|
||||
|
||||
resolve({
|
||||
writer,
|
||||
messages,
|
||||
toolCall: toolCalls[toolCalls.length - 1],
|
||||
toolCalls,
|
||||
})
|
||||
reserveEventIds(events)
|
||||
turn.lastEvents = events
|
||||
|
||||
const parts = format.materializeParts(events)
|
||||
const turnMetadata =
|
||||
turn.metadata === undefined ? metadata() : cloneValue(turn.metadata)
|
||||
|
||||
return {
|
||||
role: "assistant",
|
||||
message: format.createMessage({
|
||||
id: turn.messageId,
|
||||
role: "assistant",
|
||||
metadata: turnMetadata,
|
||||
parts,
|
||||
}),
|
||||
events,
|
||||
metadata: turnMetadata,
|
||||
continuation: true,
|
||||
}
|
||||
}
|
||||
|
||||
const api: Chat<MESSAGE, PART, TRANSPORT, METADATA, DATA, TOOLS, WRITER> = {
|
||||
user(text = DEFAULT_TEXT, userOptions: ChatUserOptions<METADATA> = {}) {
|
||||
const events = takePendingEvents()
|
||||
@@ -334,8 +610,9 @@ export function createChatRuntime<
|
||||
userOptions.metadata === undefined
|
||||
? metadata()
|
||||
: cloneValue(userOptions.metadata)
|
||||
const messageId = resolveMessageId(userOptions.id)
|
||||
const userMessage = format.createMessage({
|
||||
id: resolveMessageId(userOptions.id),
|
||||
id: messageId,
|
||||
role: "user",
|
||||
metadata: turnMetadata,
|
||||
parts,
|
||||
@@ -344,6 +621,7 @@ export function createChatRuntime<
|
||||
return pushTurn({
|
||||
role: "user",
|
||||
message: userMessage,
|
||||
messageId,
|
||||
events,
|
||||
metadata: turnMetadata,
|
||||
})
|
||||
@@ -353,17 +631,40 @@ export function createChatRuntime<
|
||||
input:
|
||||
| string
|
||||
| PART[]
|
||||
| ((context: { writer: WRITER }) => void) = DEFAULT_TEXT,
|
||||
| ((
|
||||
context: AssistantTurnContext<WRITER, MESSAGE, TOOLS>
|
||||
) => void) = DEFAULT_TEXT,
|
||||
assistantOptions: ChatAssistantOptions<METADATA> = {}
|
||||
) {
|
||||
const events = takePendingEvents()
|
||||
const continuation = isPreviousTurnPausedAssistant()
|
||||
|
||||
if (typeof input === "function" && continuation) {
|
||||
return pushTurn({
|
||||
role: "assistant",
|
||||
messageId: resolveMessageId(assistantOptions.id),
|
||||
events,
|
||||
metadata:
|
||||
assistantOptions.metadata === undefined
|
||||
? undefined
|
||||
: cloneValue(assistantOptions.metadata),
|
||||
resolve: input,
|
||||
continuation,
|
||||
})
|
||||
}
|
||||
|
||||
if (continuation) {
|
||||
events.push({ kind: "step-start" })
|
||||
}
|
||||
|
||||
const messageParts = materializeAssistantInput(input, events)
|
||||
const turnMetadata =
|
||||
assistantOptions.metadata === undefined
|
||||
? metadata()
|
||||
: cloneValue(assistantOptions.metadata)
|
||||
const messageId = resolveMessageId(assistantOptions.id)
|
||||
const assistantMessage = format.createMessage({
|
||||
id: resolveMessageId(assistantOptions.id),
|
||||
id: messageId,
|
||||
role: "assistant",
|
||||
metadata: turnMetadata,
|
||||
parts: messageParts,
|
||||
@@ -372,8 +673,10 @@ export function createChatRuntime<
|
||||
return pushTurn({
|
||||
role: "assistant",
|
||||
message: assistantMessage,
|
||||
messageId,
|
||||
events,
|
||||
metadata: turnMetadata,
|
||||
continuation: continuation || undefined,
|
||||
})
|
||||
},
|
||||
|
||||
@@ -386,15 +689,17 @@ export function createChatRuntime<
|
||||
})
|
||||
|
||||
const turnMetadata = metadata()
|
||||
const messageId = resolveMessageId()
|
||||
|
||||
return pushTurn({
|
||||
role: "assistant",
|
||||
message: format.createMessage({
|
||||
id: resolveMessageId(),
|
||||
id: messageId,
|
||||
role: "assistant",
|
||||
metadata: turnMetadata,
|
||||
parts: [],
|
||||
}),
|
||||
messageId,
|
||||
events,
|
||||
metadata: turnMetadata,
|
||||
})
|
||||
@@ -410,18 +715,35 @@ export function createChatRuntime<
|
||||
return api
|
||||
},
|
||||
|
||||
get(count = turns.length) {
|
||||
get(count) {
|
||||
const deferredIndex = turns.findIndex(
|
||||
(turn) => turn.resolve !== undefined
|
||||
)
|
||||
const limit = deferredIndex === -1 ? turns.length : deferredIndex
|
||||
|
||||
if (count === undefined) {
|
||||
count = limit
|
||||
}
|
||||
|
||||
if (!Number.isInteger(count) || count < 0) {
|
||||
throw new RangeError("count must be a non-negative integer.")
|
||||
}
|
||||
|
||||
return turns.slice(0, count).map((turn) => cloneValue(turn.message))
|
||||
if (deferredIndex !== -1 && count > deferredIndex) {
|
||||
throw new Error(
|
||||
"get() cannot materialize a continuation turn without a live transcript."
|
||||
)
|
||||
}
|
||||
|
||||
return turns
|
||||
.slice(0, count)
|
||||
.map((turn) => cloneValue(turn.message as MESSAGE))
|
||||
},
|
||||
|
||||
next(messages) {
|
||||
const turn = findNextUserTurn(messages)
|
||||
|
||||
return turn ? cloneValue(turn.message) : null
|
||||
return turn?.message ? cloneValue(turn.message) : null
|
||||
},
|
||||
|
||||
transport(
|
||||
@@ -433,13 +755,35 @@ export function createChatRuntime<
|
||||
WRITER
|
||||
> = {}
|
||||
) {
|
||||
const lastAssistantIndex = findPreviousAssistantIndex(turns.length)
|
||||
const lastAssistant = turns[lastAssistantIndex]
|
||||
|
||||
if (
|
||||
lastAssistant &&
|
||||
!lastAssistant.resolve &&
|
||||
getPendingToolInputs(lastAssistant.events).some(
|
||||
(pendingInput) => pendingInput.approval
|
||||
)
|
||||
) {
|
||||
devWarn(
|
||||
"The last scripted assistant turn requests approval but no continuation turn follows. The decision will have no response."
|
||||
)
|
||||
}
|
||||
|
||||
return format.createTransport(
|
||||
{
|
||||
resolveTurn(messages, messageId) {
|
||||
const turn = findNextAssistantTurn(messages, messageId)
|
||||
const turnIndex = findNextAssistantTurnIndex(messages, messageId)
|
||||
const turn = turnIndex === -1 ? undefined : turns[turnIndex]
|
||||
|
||||
if (turn || transportOptions.fallback === undefined) {
|
||||
return turn
|
||||
if (turn) {
|
||||
return turn.resolve
|
||||
? materializeDeferredTurn(turnIndex, messages)
|
||||
: (turn as ChatTurn<MESSAGE, DATA, TOOLS>)
|
||||
}
|
||||
|
||||
if (transportOptions.fallback === undefined) {
|
||||
return undefined
|
||||
}
|
||||
|
||||
return createFallbackTurn(transportOptions.fallback, messages)
|
||||
@@ -449,8 +793,13 @@ export function createChatRuntime<
|
||||
const steps = lowerEvents<METADATA, DATA, TOOLS>(turn.events, {
|
||||
delayMs: DEFAULT_STREAM_DELAY_MS,
|
||||
...streamOptions,
|
||||
messageId: format.getMessageId(turn.message),
|
||||
messageMetadata: (turn as InternalTurn).metadata,
|
||||
// A continuation stream carries no message id: the client is
|
||||
// already continuing the paused assistant message, and a new
|
||||
// id would fork it into a duplicate instead of updating it.
|
||||
messageId: (turn as ResolvedTurn).continuation
|
||||
? undefined
|
||||
: format.getMessageId(turn.message),
|
||||
messageMetadata: (turn as ResolvedTurn).metadata,
|
||||
})
|
||||
|
||||
return createTurnStream(steps, encodeChunk, abortSignal)
|
||||
|
||||
@@ -10,6 +10,7 @@ describe("createChatIds", () => {
|
||||
expect(ids.nextMessageId()).toBe("msg-2")
|
||||
expect(ids.nextToolCallId()).toBe("call-1")
|
||||
expect(ids.nextSourceId()).toBe("source-1")
|
||||
expect(ids.nextApprovalId()).toBe("approval-1")
|
||||
})
|
||||
|
||||
it("supports custom prefixes", () => {
|
||||
@@ -30,9 +31,11 @@ describe("createChatIds", () => {
|
||||
ids.reserveMessageId("msg-2")
|
||||
ids.reserveToolCallId("call-4")
|
||||
ids.reserveSourceId("source-3")
|
||||
ids.reserveApprovalId("approval-2")
|
||||
|
||||
expect(ids.nextMessageId()).toBe("msg-3")
|
||||
expect(ids.nextToolCallId()).toBe("call-5")
|
||||
expect(ids.nextSourceId()).toBe("source-4")
|
||||
expect(ids.nextApprovalId()).toBe("approval-3")
|
||||
})
|
||||
})
|
||||
|
||||
@@ -9,6 +9,7 @@ export function createChatIds(options: ChatIdsOptions = {}): ChatIds {
|
||||
const messageIdPrefix = options.messageIdPrefix ?? "msg"
|
||||
const toolCallIdPrefix = options.toolCallIdPrefix ?? "call"
|
||||
const sourceIdPrefix = options.sourceIdPrefix ?? "source"
|
||||
const approvalIdPrefix = options.approvalIdPrefix ?? "approval"
|
||||
|
||||
function createSequence(prefix: string) {
|
||||
const reserved = new Set<string>()
|
||||
@@ -55,6 +56,7 @@ export function createChatIds(options: ChatIdsOptions = {}): ChatIds {
|
||||
const messageIds = createSequence(messageIdPrefix)
|
||||
const toolCallIds = createSequence(toolCallIdPrefix)
|
||||
const sourceIds = createSequence(sourceIdPrefix)
|
||||
const approvalIds = createSequence(approvalIdPrefix)
|
||||
|
||||
return {
|
||||
nextMessageId() {
|
||||
@@ -69,6 +71,10 @@ export function createChatIds(options: ChatIdsOptions = {}): ChatIds {
|
||||
return sourceIds.next()
|
||||
},
|
||||
|
||||
nextApprovalId() {
|
||||
return approvalIds.next()
|
||||
},
|
||||
|
||||
reserveMessageId(id) {
|
||||
messageIds.reserve(id)
|
||||
},
|
||||
@@ -80,5 +86,9 @@ export function createChatIds(options: ChatIdsOptions = {}): ChatIds {
|
||||
reserveSourceId(id) {
|
||||
sourceIds.reserve(id)
|
||||
},
|
||||
|
||||
reserveApprovalId(id) {
|
||||
approvalIds.reserve(id)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -154,6 +154,18 @@ export function lowerEvents<
|
||||
})
|
||||
break
|
||||
}
|
||||
case "tool-approval-request": {
|
||||
// The gated output stays script-side; only the request goes on the wire.
|
||||
steps.push({
|
||||
kind: "chunk",
|
||||
chunk: {
|
||||
type: "tool-approval-request",
|
||||
approvalId: event.approvalId,
|
||||
toolCallId: event.toolCallId,
|
||||
},
|
||||
})
|
||||
break
|
||||
}
|
||||
case "tool-output": {
|
||||
steps.push({
|
||||
kind: "chunk",
|
||||
|
||||
@@ -6,6 +6,7 @@ import type {
|
||||
MessageRole,
|
||||
NeutralChunk,
|
||||
StreamStep,
|
||||
ToolCallSummary,
|
||||
ToolSet,
|
||||
TransportContext,
|
||||
TurnStreamOptions,
|
||||
@@ -97,6 +98,20 @@ export function createTestFormat<
|
||||
})
|
||||
}
|
||||
|
||||
if (event.kind === "tool-approval-request") {
|
||||
const index = parts.findIndex(
|
||||
(part) => part.toolCallId === event.toolCallId
|
||||
)
|
||||
|
||||
if (index !== -1) {
|
||||
parts[index] = {
|
||||
...parts[index],
|
||||
state: "approval-requested",
|
||||
approval: { id: event.approvalId },
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (event.kind === "tool-output") {
|
||||
const index = parts.findIndex(
|
||||
(part) => part.toolCallId === event.toolCallId
|
||||
@@ -161,6 +176,19 @@ export function createTestFormat<
|
||||
return message.parts
|
||||
},
|
||||
|
||||
getToolCalls(message) {
|
||||
return message.parts
|
||||
.filter((part) => part.type.startsWith("tool-"))
|
||||
.map((part) => ({
|
||||
toolCallId: part.toolCallId as string,
|
||||
name: part.type.replace("tool-", ""),
|
||||
input: part.input,
|
||||
output: part.state === "output-available" ? part.output : undefined,
|
||||
state: part.state as ToolCallSummary["state"],
|
||||
approval: part.approval as ToolCallSummary["approval"],
|
||||
}))
|
||||
},
|
||||
|
||||
createTransport(context, options) {
|
||||
return { context, options }
|
||||
},
|
||||
|
||||
@@ -30,6 +30,52 @@ export type ToolWriterOptions<
|
||||
errorText?: string
|
||||
/** Script the call as a `dynamic-tool` part instead of a typed `tool-<name>` part. */
|
||||
dynamic?: boolean
|
||||
/**
|
||||
* Pause the turn behind a user approval. `output` (or `errorText`) then
|
||||
* means "stream this after approval" instead of "resolve immediately";
|
||||
* denial streams `tool-output-denied` automatically.
|
||||
*/
|
||||
needsApproval?: boolean
|
||||
approvalId?: string
|
||||
}
|
||||
|
||||
/**
|
||||
* A human-in-the-loop tool call as a continuation turn sees it: the scripted
|
||||
* input, the user-submitted output (elicitation) or gated output (approval),
|
||||
* and the approval decision when one was requested.
|
||||
*/
|
||||
export type PendingToolCall<TOOLS extends ToolSet> = {
|
||||
[NAME in keyof TOOLS & string]: {
|
||||
name: NAME
|
||||
toolCallId: string
|
||||
input: TOOLS[NAME]["input"]
|
||||
output: TOOLS[NAME]["output"] | undefined
|
||||
approved?: boolean
|
||||
denied?: boolean
|
||||
}
|
||||
}[keyof TOOLS & string]
|
||||
|
||||
/**
|
||||
* A framework-neutral summary of one tool call read back from a transcript
|
||||
* message. Adapters produce these through `ChatFormat.getToolCalls`.
|
||||
*/
|
||||
export type ToolCallSummary = {
|
||||
toolCallId: string
|
||||
name: string
|
||||
dynamic?: boolean
|
||||
input: unknown
|
||||
output?: unknown
|
||||
errorText?: string
|
||||
state:
|
||||
| "input-streaming"
|
||||
| "input-available"
|
||||
| "approval-requested"
|
||||
| "approval-responded"
|
||||
| "output-available"
|
||||
| "output-error"
|
||||
| "output-denied"
|
||||
| "undefined-input"
|
||||
approval?: { id: string; approved?: boolean; reason?: string }
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -125,6 +171,13 @@ export type ChatEvent<
|
||||
dynamic?: boolean
|
||||
input: unknown
|
||||
}
|
||||
| {
|
||||
kind: "tool-approval-request"
|
||||
toolCallId: string
|
||||
approvalId: string
|
||||
output?: unknown
|
||||
errorText?: string
|
||||
}
|
||||
| {
|
||||
kind: "tool-output"
|
||||
toolCallId: string
|
||||
@@ -188,6 +241,11 @@ export type NeutralChunk<
|
||||
dynamic?: boolean
|
||||
title?: string
|
||||
}
|
||||
| {
|
||||
type: "tool-approval-request"
|
||||
approvalId: string
|
||||
toolCallId: string
|
||||
}
|
||||
| {
|
||||
type: "tool-output-available"
|
||||
toolCallId: string
|
||||
@@ -237,9 +295,11 @@ export type ChatIds = {
|
||||
nextMessageId(): string
|
||||
nextToolCallId(): string
|
||||
nextSourceId(): string
|
||||
nextApprovalId(): string
|
||||
reserveMessageId(id: string): void
|
||||
reserveToolCallId(id: string): void
|
||||
reserveSourceId(id: string): void
|
||||
reserveApprovalId(id: string): void
|
||||
}
|
||||
|
||||
/** Prefixes used by the deterministic chat id generators. */
|
||||
@@ -247,6 +307,7 @@ export type ChatIdsOptions = {
|
||||
messageIdPrefix?: string
|
||||
toolCallIdPrefix?: string
|
||||
sourceIdPrefix?: string
|
||||
approvalIdPrefix?: string
|
||||
}
|
||||
|
||||
/** Chat-wide options: id prefixes plus the fixed clock used for default metadata. */
|
||||
@@ -321,6 +382,8 @@ export type ChatFormat<
|
||||
getMessageParts(message: MESSAGE): PART[]
|
||||
/** Optional: extract a message's metadata when hydrating chats from existing messages. */
|
||||
getMessageMetadata?(message: MESSAGE): METADATA | undefined
|
||||
/** Optional: read a message's tool calls so continuation turns can resolve human-in-the-loop state. */
|
||||
getToolCalls?(message: MESSAGE): ToolCallSummary[]
|
||||
createTransport(
|
||||
context: TransportContext<MESSAGE, CHUNK, METADATA, DATA, TOOLS>,
|
||||
options?: TurnStreamOptions
|
||||
|
||||
@@ -46,3 +46,10 @@ export function splitTextDeltas(text: string) {
|
||||
export function getDataPartName(type: `data-${string}`) {
|
||||
return type.slice("data-".length)
|
||||
}
|
||||
|
||||
/** Logs a development-only warning. */
|
||||
export function devWarn(message: string) {
|
||||
if (process.env.NODE_ENV !== "production") {
|
||||
console.warn(`[helpers] ${message}`)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -157,6 +157,14 @@ export function createEventWriter<
|
||||
input: clonedOptions.input ?? {},
|
||||
})
|
||||
|
||||
function assertResolvable(method: string) {
|
||||
if (clonedOptions.needsApproval) {
|
||||
throw new Error(
|
||||
`A needsApproval tool call resolves from the user's decision; remove the ${method}() call.`
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
const handle: ToolHandle<TOOLS[NAME]["output"]> = {
|
||||
sleep(delayMs: number) {
|
||||
events.push({
|
||||
@@ -169,6 +177,7 @@ export function createEventWriter<
|
||||
},
|
||||
|
||||
output(output: TOOLS[NAME]["output"]) {
|
||||
assertResolvable("output")
|
||||
events.push({
|
||||
kind: "tool-output",
|
||||
toolCallId,
|
||||
@@ -182,6 +191,7 @@ export function createEventWriter<
|
||||
},
|
||||
|
||||
error(errorText = "Tool call failed.") {
|
||||
assertResolvable("error")
|
||||
events.push({
|
||||
kind: "tool-error",
|
||||
toolCallId,
|
||||
@@ -195,6 +205,7 @@ export function createEventWriter<
|
||||
},
|
||||
|
||||
denied() {
|
||||
assertResolvable("denied")
|
||||
events.push({
|
||||
kind: "tool-denied",
|
||||
toolCallId,
|
||||
@@ -204,6 +215,25 @@ export function createEventWriter<
|
||||
},
|
||||
}
|
||||
|
||||
if (clonedOptions.needsApproval) {
|
||||
const approvalId =
|
||||
clonedOptions.approvalId ?? context.ids.nextApprovalId()
|
||||
|
||||
if (clonedOptions.approvalId !== undefined) {
|
||||
context.ids.reserveApprovalId(clonedOptions.approvalId)
|
||||
}
|
||||
|
||||
events.push({
|
||||
kind: "tool-approval-request",
|
||||
toolCallId,
|
||||
approvalId,
|
||||
output: clonedOptions.output,
|
||||
errorText: clonedOptions.errorText,
|
||||
})
|
||||
|
||||
return handle
|
||||
}
|
||||
|
||||
if (clonedOptions.output !== undefined) {
|
||||
handle.output(clonedOptions.output)
|
||||
}
|
||||
|
||||
@@ -162,6 +162,7 @@ export function createRunEncoder<
|
||||
value: chunk.part,
|
||||
})
|
||||
case "abort":
|
||||
case "tool-approval-request":
|
||||
case "tool-output-denied":
|
||||
case "start-step":
|
||||
case "file":
|
||||
|
||||
@@ -224,6 +224,7 @@ export function materializeParts<
|
||||
case "reasoning-file":
|
||||
case "custom":
|
||||
case "step-start":
|
||||
case "tool-approval-request":
|
||||
// These events have no TanStack message part.
|
||||
break
|
||||
default:
|
||||
|
||||
Generated
+6
-51
@@ -425,8 +425,8 @@ importers:
|
||||
specifier: 0.20.0
|
||||
version: 0.20.0(@opentelemetry/api@1.9.0)
|
||||
ai:
|
||||
specifier: 7.0.22
|
||||
version: 7.0.22(zod@4.4.3)
|
||||
specifier: 7.0.0-canary.159
|
||||
version: 7.0.0-canary.159(zod@3.25.76)
|
||||
rimraf:
|
||||
specifier: ^6.0.1
|
||||
version: 6.0.1
|
||||
@@ -439,6 +439,9 @@ importers:
|
||||
vitest:
|
||||
specifier: ^3.2.6
|
||||
version: 3.2.6(@types/debug@4.1.12)(@types/node@20.19.10)(@vitest/browser@3.2.6)(jiti@2.7.0)(jsdom@28.1.0(@noble/hashes@1.8.0))(lightningcss@1.32.0)(msw@2.10.4(@types/node@20.19.10)(typescript@5.9.2))(tsx@4.20.3)(yaml@2.8.1)
|
||||
zod:
|
||||
specifier: 3.25.76
|
||||
version: 3.25.76
|
||||
|
||||
packages/react:
|
||||
dependencies:
|
||||
@@ -664,12 +667,6 @@ packages:
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/gateway@4.0.16':
|
||||
resolution: {integrity: sha512-9iYxdOLquoOFk5e+02P9qfBIA+EJU0FNJvyjGxi4iXJabt2+GlbaCg7TX0sTtxOsBVkdabkatqWM6+kvp53tBg==}
|
||||
engines: {node: '>=22'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/mcp@2.0.0-canary.56':
|
||||
resolution: {integrity: sha512-4YteXhAIJNmhpG+Tab+iyDyw53oXceLH/Xa88FutBGFPJanqgQyB+Dp2kgRbq8xxcQLp6ufuVwxYXYuFq3exUw==}
|
||||
engines: {node: '>=22'}
|
||||
@@ -694,12 +691,6 @@ packages:
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/provider-utils@5.0.7':
|
||||
resolution: {integrity: sha512-OSm5/5kdrHa11WIOo5LYgDKnxYWp5aB/wx5EXRHi0jpUGduMDeB6oht9U6p+UNNWIP3F/EqPpV8d7vdP/iRnqg==}
|
||||
engines: {node: '>=22'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
'@ai-sdk/provider@4.0.0-canary.17':
|
||||
resolution: {integrity: sha512-m/rtalImeIt7deuQGkEHehqlIPOM8Sjb0cF1b1SJ3B6Re+WYgbUkOK9gY2SSro1YO8jYBRgTMPz2qYzPtIzAxQ==}
|
||||
engines: {node: '>=22'}
|
||||
@@ -708,10 +699,6 @@ packages:
|
||||
resolution: {integrity: sha512-+XBwXEPkN+l/90bb7TaSK15jAq9ScsGrauJ/NLSGip0KX5Mf1pfEfqBWDuJMIYVRYHNdV1dDBy7JCoKUCQe/tA==}
|
||||
engines: {node: '>=22'}
|
||||
|
||||
'@ai-sdk/provider@4.0.3':
|
||||
resolution: {integrity: sha512-e0CpNWJUY7OxAFAnCZkw+ri9QOHWwTs1tXP42782KFGCU07qt8NiXCrCVowyCB5dP2r5/Uls+g2oPd8kOJn9dw==}
|
||||
engines: {node: '>=22'}
|
||||
|
||||
'@ai-sdk/react@4.0.0-canary.162':
|
||||
resolution: {integrity: sha512-OSiEDRsgJVviCajzxbRgxLGxDj+jAsAPlp5Uo4QOA0UrvZ3Aecg+78XZK5YJIazX4J9wNam6Uj481uEdIXtR/w==}
|
||||
engines: {node: '>=22'}
|
||||
@@ -4711,12 +4698,6 @@ packages:
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
ai@7.0.22:
|
||||
resolution: {integrity: sha512-iAXYwQtR18qN7GXwNmGVAQM4uSJOh2+nZ5RP9NtDXfH5d4tlYyYzAM133O3U+eVcyWrvNRzecN9yW58xpCdfMg==}
|
||||
engines: {node: '>=22'}
|
||||
peerDependencies:
|
||||
zod: ^3.25.76 || ^4.1.8
|
||||
|
||||
ajv-formats@3.0.1:
|
||||
resolution: {integrity: sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ==}
|
||||
peerDependencies:
|
||||
@@ -9802,13 +9783,6 @@ snapshots:
|
||||
'@vercel/oidc': 3.2.0
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/gateway@4.0.16(zod@4.4.3)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 4.0.3
|
||||
'@ai-sdk/provider-utils': 5.0.7(zod@4.4.3)
|
||||
'@vercel/oidc': 3.2.0
|
||||
zod: 4.4.3
|
||||
|
||||
'@ai-sdk/mcp@2.0.0-canary.56(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 4.0.0-canary.17
|
||||
@@ -9838,14 +9812,6 @@ snapshots:
|
||||
eventsource-parser: 3.1.0
|
||||
zod: 3.25.76
|
||||
|
||||
'@ai-sdk/provider-utils@5.0.7(zod@4.4.3)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 4.0.3
|
||||
'@standard-schema/spec': 1.1.0
|
||||
'@workflow/serde': 4.1.0
|
||||
eventsource-parser: 3.1.0
|
||||
zod: 4.4.3
|
||||
|
||||
'@ai-sdk/provider@4.0.0-canary.17':
|
||||
dependencies:
|
||||
json-schema: 0.4.0
|
||||
@@ -9854,10 +9820,6 @@ snapshots:
|
||||
dependencies:
|
||||
json-schema: 0.4.0
|
||||
|
||||
'@ai-sdk/provider@4.0.3':
|
||||
dependencies:
|
||||
json-schema: 0.4.0
|
||||
|
||||
'@ai-sdk/react@4.0.0-canary.162(react@19.2.3)(zod@3.25.76)':
|
||||
dependencies:
|
||||
'@ai-sdk/mcp': 2.0.0-canary.56(zod@3.25.76)
|
||||
@@ -11185,7 +11147,7 @@ snapshots:
|
||||
cors: 2.8.6
|
||||
cross-spawn: 7.0.6
|
||||
eventsource: 3.0.7
|
||||
eventsource-parser: 3.0.6
|
||||
eventsource-parser: 3.1.0
|
||||
express: 5.2.1
|
||||
express-rate-limit: 7.5.1(express@5.2.1)
|
||||
jose: 6.1.3
|
||||
@@ -13773,13 +13735,6 @@ snapshots:
|
||||
'@ai-sdk/provider-utils': 5.0.0-canary.44(zod@3.25.76)
|
||||
zod: 3.25.76
|
||||
|
||||
ai@7.0.22(zod@4.4.3):
|
||||
dependencies:
|
||||
'@ai-sdk/gateway': 4.0.16(zod@4.4.3)
|
||||
'@ai-sdk/provider': 4.0.3
|
||||
'@ai-sdk/provider-utils': 5.0.7(zod@4.4.3)
|
||||
zod: 4.4.3
|
||||
|
||||
ajv-formats@3.0.1(ajv@8.17.1):
|
||||
optionalDependencies:
|
||||
ajv: 8.17.1
|
||||
|
||||
Reference in New Issue
Block a user