assistant-ui Streaming

Always consult assistant-ui.com/llms.txt for latest API.

The assistant-stream package handles streaming from AI backends.

References

• ./references/data-stream.md -- AI SDK data stream format
• ./references/assistant-transport.md -- Native assistant-ui format
• ./references/encoders.md -- Encoders and decoders

When to Use

Using Vercel AI SDK?
├─ Yes → toUIMessageStreamResponse() (no assistant-stream needed)
└─ No → assistant-stream for custom backends

Installation

bash
1npm install assistant-stream

Custom Streaming Response

ts
1import { createAssistantStreamResponse } from "assistant-stream";
2
3export async function POST(req: Request) {
4  return createAssistantStreamResponse(async (stream) => {
5    stream.appendText("Hello ");
6    stream.appendText("world!");
7
8    // Tool call example
9    const tool = stream.addToolCallPart({ toolCallId: "1", toolName: "get_weather" });
10    tool.argsText.append('{"city":"NYC"}');
11    tool.argsText.close();
12    tool.setResponse({ result: { temperature: 22 } });
13
14    stream.close();
15  });
16}

With useLocalRuntime

useLocalRuntime expects ChatModelRunResult chunks. Yield content parts for streaming:

tsx
1import { useLocalRuntime } from "@assistant-ui/react";
2
3const runtime = useLocalRuntime({
4  model: {
5    async *run({ messages, abortSignal }) {
6      const response = await fetch("/api/chat", {
7        method: "POST",
8        body: JSON.stringify({ messages }),
9        signal: abortSignal,
10      });
11
12      const reader = response.body?.getReader();
13      const decoder = new TextDecoder();
14      let buffer = "";
15
16      while (reader) {
17        const { done, value } = await reader.read();
18        if (done) break;
19
20        buffer += decoder.decode(value, { stream: true });
21        const parts = buffer.split("\n");
22        buffer = parts.pop() ?? "";
23
24        for (const chunk of parts.filter(Boolean)) {
25          yield { content: [{ type: "text", text: chunk }] };
26        }
27      }
28    },
29  },
30});

Debugging Streams

ts
1import { AssistantStream, DataStreamDecoder } from "assistant-stream";
2
3const stream = AssistantStream.fromResponse(response, new DataStreamDecoder());
4for await (const event of stream) {
5  console.log("Event:", JSON.stringify(event, null, 2));
6}

Stream Event Types

• part-start with part.type = "text" | "reasoning" | "tool-call" | "source" | "file"
• text-delta with streamed text
• result with tool results
• step-start, step-finish, message-finish
• error strings

Common Gotchas

Stream not updating UI

• Check Content-Type is text/event-stream
• Check for CORS errors

Tool calls not rendering

• addToolCallPart needs both toolCallId and toolName
• Register tool UI with makeAssistantToolUI

Partial text not showing

• Use text-delta events for streaming