How to Stream AI Responses

Chainabit streams AI work through Server-Sent Events (SSE). Use the stream to append assistant text, render Chao tool cards, handle approval prompts, and detect terminal run states.

Environment Variables

export BASE_URL="https://api.chainabit.com/api/v1"
export TOKEN="your-access-token"

Streaming Endpoints

Endpoint	Purpose
GET /ai/sessions/:sessionId/messages/:messageId/stream	Stream a conversation response
GET /ai/runs/:runId/stream	Stream an AI feature run

Both endpoints use the same event envelope.

Create a Session and Send a Message

cURL/Bash

SESSION=$(curl -s -X POST "$BASE_URL/ai/sessions" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"title": "Planning session", "assistantType": "chao"}')

SESSION_ID=$(echo "$SESSION" | jq -r '.data.id')

MESSAGE=$(curl -s -X POST "$BASE_URL/ai/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"content": "Help me plan my week"}')

MESSAGE_ID=$(echo "$MESSAGE" | jq -r '.data.assistantMessage.id')
RUN_ID=$(echo "$MESSAGE" | jq -r '.data.run.id')

JavaScript

const sessionResponse = await fetch(`${BASE_URL}/ai/sessions`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({ title: "Planning session", assistantType: "chao" }),
});
const { data: session } = await sessionResponse.json();

const messageResponse = await fetch(
  `${BASE_URL}/ai/sessions/${session.id}/messages`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ content: "Help me plan my week" }),
  }
);
const { data } = await messageResponse.json();
const messageId = data.assistantMessage.id;
const runId = data.run.id;

Python

import os
import requests

BASE_URL = os.environ["BASE_URL"]
TOKEN = os.environ["TOKEN"]
headers = {"Authorization": f"Bearer {TOKEN}", "Content-Type": "application/json"}

session = requests.post(
    f"{BASE_URL}/ai/sessions",
    headers=headers,
    json={"title": "Planning session", "assistantType": "chao"},
).json()["data"]

message = requests.post(
    f"{BASE_URL}/ai/sessions/{session['id']}/messages",
    headers=headers,
    json={"content": "Help me plan my week"},
).json()["data"]

message_id = message["assistantMessage"]["id"]
run_id = message["run"]["id"]

Connect to the Stream

Tip: The POST endpoint returns {runId, finishReason: "running"} within milliseconds. Open the stream immediately after receiving the response — you'll see run.started right away, then text begins streaming as Chao works.

cURL

curl -N "$BASE_URL/ai/sessions/$SESSION_ID/messages/$MESSAGE_ID/stream" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Accept: text/event-stream"

JavaScript

import EventSource from "eventsource";

const url = `${BASE_URL}/ai/sessions/${session.id}/messages/${messageId}/stream`;
const es = new EventSource(url, {
  headers: { Authorization: `Bearer ${TOKEN}` },
});

let fullContent = "";
const toolCards = new Map();

const normalizeToolPayload = (envelope) => {
  const payload = envelope.payload ?? {};
  return {
    toolKey: payload.toolKey ?? payload.key,
    callId: payload.callId ?? payload.toolCallId ?? envelope.stepId,
    input: payload.input ?? payload.args,
    output: payload.data ?? payload.output,
    activity: payload.activity,
    status: payload.activity?.status,
  };
};

const upsertToolCard = (event) => {
  const envelope = JSON.parse(event.data);
  const card = normalizeToolPayload(envelope);
  if (card.callId) toolCards.set(card.callId, card);
};

["tool.started", "tool.progress", "tool.completed", "tool.failed", "tool.degraded",
 "tool.approval_required", "tool.approval_response", "tool.approval_timeout",
 "plan.step_added", "capability.resolved"].forEach((type) => {
  es.addEventListener(type, upsertToolCard);
});

es.addEventListener("message.delta", (event) => {
  const { payload } = JSON.parse(event.data);
  fullContent += payload.delta ?? "";
  process.stdout.write(payload.delta ?? "");
});

es.addEventListener("message.completed", () => {
  console.log("\nAssistant message completed.");
});

es.addEventListener("run.heartbeat", (event) => {
  const { payload } = JSON.parse(event.data);
  // Show a status card between tool results and the next inference cycle.
  if (payload.activity) {
    console.log(`[${payload.activity.label}] ${payload.activity.detail}`);
  }
});

const closeOnTerminal = (event) => {
  if (event.type === "run.error") {
    const { payload } = JSON.parse(event.data);
    console.error("Run error:", payload.error);
  }
  es.close();
};

["message.partially_completed", "run.failed", "run.error", "run.settlement.completed"]
  .forEach((type) => es.addEventListener(type, closeOnTerminal));

Python

import json
import requests

with requests.get(
    f"{BASE_URL}/ai/sessions/{session['id']}/messages/{message_id}/stream",
    headers={"Authorization": f"Bearer {TOKEN}", "Accept": "text/event-stream"},
    stream=True,
) as response:
    for line in response.iter_lines(decode_unicode=True):
        if line and line.startswith("data: "):
            envelope = json.loads(line[6:])
            event_type = envelope["type"]
            payload = envelope.get("payload", {})
            if event_type == "message.delta":
                print(payload.get("delta", ""), end="", flush=True)
            elif event_type in {"run.error", "run.failed", "run.settlement.completed"}:
                break

Event Format

id: 1
retry: 3000
event: message.delta
data: {"eventId":1,"type":"message.delta","runId":"...","timestamp":"...","payload":{"delta":"Here is"}}

Common events:

Event	Description
run.started	Inference has begun. Show an active indicator.
run.status.changed	Status transition. payload.status is "running" or "cancelled". Show "Preparing…" until message.delta starts.
message.delta	Append payload.delta to the assistant text
message.completed	Full assistant content is available in payload.content
run.heartbeat	The assistant is between steps in a multi-tool run. payload.reason is 'planning_next_step' (reviewing tool results, deciding next action) or 'continuation' (response was truncated, requesting continuation). payload.activity.detail contains a user-safe description — use it to show a status card or thinking indicator.
tool.started / tool.progress / tool.completed	Render or update a tool card. payload.activity.label (e.g. "Web Search") and payload.activity.detail are safe to show users.
tool.failed	Mark a tool card failed; the run may still continue
tool.degraded	Render a graceful fallback state
tool.approval_required	Show approve/reject controls
run.thinking.started	Extended reasoning phase began. Show a thinking indicator using payload.activity.label.
run.thinking.completed	Extended reasoning phase ended. Hide the thinking indicator.
plan.step_added	Add a planned action card
capability.resolved	Show selected capability/tool route when useful
cot.step	A reasoning step (observation, thought, action, or reflection) emitted by Chao before the final reply. Only present when chain-of-thought tracing is active on the session.
message.partially_completed	The assistant produced partial content
run.failed / run.error	Terminal error. payload.error contains a machine-readable error key.
run.settlement.completed	Terminal successful settlement

Chain-of-Thought Reasoning Steps

When a session is created with CoT tracing enabled, Chao emits structured reasoning steps as cot.step events before the final message.delta stream begins. These events are distinct from message content — they describe Chao's internal reasoning process, not the reply to the user.

event: cot.step
data: {"eventId":4,"type":"cot.step","runId":"...","payload":{"stepType":"observation","content":"User is asking about...","traceId":"..."}}

event: cot.step
data: {"eventId":5,"type":"cot.step","runId":"...","payload":{"stepType":"thought","content":"I should approach this by..."}}

event: message.delta
data: {"eventId":6,"type":"message.delta","runId":"...","payload":{"delta":"Here is my answer..."}}

cot.step payload fields:

Field	Type	Description
stepType	"observation" | "thought" | "action" | "reflection"	Category of reasoning step
content	string	The reasoning text
confidence	number (optional)	0–1 confidence score when present
traceId	string	Groups all steps from one run
stepIndex	number	Monotonically increasing ordering hint

Steps are never included in message.delta — payload.delta always contains only the user-facing response text.

Extended Thinking Events

When Chao enters an extended reasoning phase (Pro plans and above), three events bracket the thinking period:

Event	Key payload fields	What to show
run.thinking.started	activity.label, activity.detail	Thinking indicator (e.g. "Thinking…")
message.thinking.delta	delta: string	Optional: stream reasoning text into a collapsible panel
run.thinking.completed	activity.label, thinkingTokens?: number	Hide the indicator

event: run.thinking.started
data: {"type":"run.thinking.started","payload":{"activity":{"label":"Thinking","detail":"The model is reasoning through the next response."}}}

event: message.thinking.delta
data: {"type":"message.thinking.delta","payload":{"delta":"Let me consider the options..."}}

event: run.thinking.completed
data: {"type":"run.thinking.completed","payload":{"activity":{"label":"Thinking complete"},"thinkingTokens":342}}

Show payload.activity.label as a status indicator while run.thinking.started through run.thinking.completed are active. Hide it when run.thinking.completed fires. The thinking content is separate from message.delta — payload.delta always contains only the user-facing reply.

Approval Buttons

When you receive tool.approval_required, submit the decision using the toolCallId from the event:

curl -X POST "$BASE_URL/ai/runs/$RUN_ID/tools/$TOOL_CALL_ID/approve" \
  -H "Authorization: Bearer $TOKEN"

curl -X POST "$BASE_URL/ai/runs/$RUN_ID/tools/$TOOL_CALL_ID/reject" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"reason": "Not now"}'

Reconnection

If the connection drops before a terminal event, reconnect to the same stream. When your client can set headers, pass Last-Event-ID with the latest received SSE id:

curl -N "$BASE_URL/ai/runs/$RUN_ID/stream" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Accept: text/event-stream" \
  -H "Last-Event-ID: 5"

After page reload, fetch message history and rebuild tool cards from message.toolCalls. Then reconnect to the stream if the assistant message still has an active runId.

Tips

• Use curl -N or --no-buffer while testing.
• Listen to named events with addEventListener; do not rely only on the default message event.
• cot.step events carry Chao's real-time reasoning steps. Render them as a collapsible "thinking" panel or ignore them entirely — they never appear in message.delta.
• Treat other cot.* events (cot.run.started, cot.run.completed, cot.tool.reasoning) as optional debug metadata; build public reasoning UI from payload.activity and message.toolCalls.
• Do not close the run on tool.failed; wait for message.completed, message.partially_completed, run.failed, run.error, or run.settlement.completed.