AI Audio Generation

Generate short music clips (up to 30 seconds) from a text description. The platform uses Google Lyria to produce audio and returns a time-limited URL to the generated file.

Two entry points are available:

Path	When to use
Chao tool (media.audio.generate)	Inside an AI chat session — let the model decide when to generate based on the conversation
REST API (POST /ai/generative-media/context-runs)	Direct, programmatic invocation — optionally enriched with personal context

Generating audio requires the ai.audio.generate entitlement on the active plan.

---

Chao Tool Path

Description

Send a user message to a session. Set capabilityKey to ai.audio.generate to skip intent detection and route directly to the audio generation tool.

Endpoints

Method	Path	Description	Auth	Rate Limit
POST	/ai/sessions/:sessionId/messages	Send a message that triggers audio generation	JWT + Entitlement	20/min
GET	/ai/sessions/:sessionId/messages/:messageId/stream	SSE stream of progress and completion events	JWT + Entitlement	20/min

Request

Request Body

Field	Type	Required	Description
content	string	Yes	Free-text description of the music to generate
capabilityKey	string	Yes (for deterministic routing)	Set to ai.audio.generate
durationSeconds	number	No	Target clip length in seconds (5–30). Default: 20. Clamped to 30; ignored if ≤ 0
bpm	number	No	Beats per minute (40–220). Clamped to range; omitted from the request if missing or ≤ 0
styleTokens	string[]	No	Genre, instrument, or mood hints (e.g. ["jazz", "piano", "late night"]). Max 10 items
effortMode	string	No	basic, thinking, or pro
idempotencyKey	string	No	Client-generated key to deduplicate retries

The authenticated request supplies the actor. Request bodies must never include accountId, chainerId, or workspaceId.

Response

{
  "data": {
    "sessionId": "550e8400-e29b-41d4-a716-446655440000",
    "userMessage": {
      "id": "550e8400-e29b-41d4-a716-446655440010",
      "role": "user",
      "content": "A calm jazz piano piece for late-night focus sessions.",
      "createdAt": "2026-05-22T10:00:00Z",
      "toolCalls": []
    },
    "assistantMessage": {
      "id": "550e8400-e29b-41d4-a716-446655440011",
      "role": "assistant",
      "content": "",
      "createdAt": "2026-05-22T10:00:00Z",
      "runId": "550e8400-e29b-41d4-a716-446655440012",
      "status": "pending",
      "toolCalls": [],
      "modelName": "Lyria",
      "providerKey": "google"
    },
    "run": {
      "id": "550e8400-e29b-41d4-a716-446655440012",
      "status": "running"
    }
  }
}

SSE Event Sequence

event: run.started
event: tool.started      toolKey=media.audio.generate
event: tool.progress     message="Generating audio..."      activityType=audio_generation
event: tool.completed    toolKey=media.audio.generate  data={ audioUrl, durationSeconds, ... }
event: message.delta     (assistant prose summarising the result)
event: message.completed
event: run.settlement.completed

SSE — tool.completed Payload

{
  "eventId": 4,
  "type": "tool.completed",
  "runId": "550e8400-e29b-41d4-a716-446655440012",
  "stepId": "tc_abc123",
  "timestamp": "2026-05-22T10:00:28Z",
  "payload": {
    "toolKey": "media.audio.generate",
    "callId": "tc_abc123",
    "executionMs": 22400,
    "success": true,
    "data": {
      "success": true,
      "audioUrl": "https://media.chainabit.com/ai-generated/audio/...wav",
      "durationSeconds": 20,
      "mimeType": "audio/wav"
    }
  }
}

Error Responses

HTTP	Meaning
400	Schema validation failed (e.g. bpm outside 40–220, durationSeconds > 30, sending image-capability fields like aspectRatio)
401	Missing or invalid JWT
402	Insufficient credits to reserve the requested duration
403	Plan does not include ai.audio.generate
404	sessionId not found
429	Rate limit exceeded

Capability-aware 400 example — sending an image field with the audio capability:

{
  "error": {
    "code": "bad_request",
    "message": "aspectRatio is only valid for ai.image.generate",
    "details": {
      "fields": [
        { "field": "aspectRatio", "message": "aspectRatio is only valid for ai.image.generate", "capabilityKey": "ai.image.generate" }
      ]
    }
  }
}

Parameters can also be sent nested under parameters (preferred for new code) — see Messages: Capability Parameters.

Code Examples

Sending a Message

Use the id from Create a Session response as $SESSION_ID.

cURL — prompt only

curl -X POST "$BASE_URL/ai/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "A calm jazz piano piece for late-night focus sessions.",
    "capabilityKey": "ai.audio.generate"
  }'

cURL — with parameters

curl -X POST "$BASE_URL/ai/sessions/$SESSION_ID/messages" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "content": "Upbeat pop background music for a workout video.",
    "capabilityKey": "ai.audio.generate",
    "durationSeconds": 25,
    "bpm": 128,
    "styleTokens": ["guitar", "synth", "energetic"]
  }'

JavaScript

const sessionId = process.env.SESSION_ID; // id of the AI session to send the message to

const res = await fetch(
  `${BASE_URL}/ai/sessions/${sessionId}/messages`,
  {
    method: "POST",
    headers: {
      Authorization: `Bearer ${TOKEN}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({
      content: "Ambient electronic music for deep focus.",
      capabilityKey: "ai.audio.generate",
      durationSeconds: 30,
      bpm: 80,
      styleTokens: ["ambient", "electronic", "minimal"],
    }),
  },
);
const { data } = await res.json();
const { assistantMessage, run } = data;
console.log("Run id:", run.id, "stream:", assistantMessage.id);

Python

import os
import requests

session_id = os.environ["SESSION_ID"]  # id of the AI session to send the message to

res = requests.post(
    f"{BASE_URL}/ai/sessions/{session_id}/messages",
    headers={
        "Authorization": f"Bearer {TOKEN}",
        "Content-Type": "application/json",
    },
    json={
        "content": "A cinematic orchestral piece with rising tension.",
        "capabilityKey": "ai.audio.generate",
        "durationSeconds": 20,
        "bpm": 90,
        "styleTokens": ["cinematic", "orchestral", "strings"],
    },
)
body = res.json()["data"]
assistant_id = body["assistantMessage"]["id"]
run_id = body["run"]["id"]

Streaming the Response

Use the same $SESSION_ID as above, and the assistantMessage.id from the message-send response (data.assistantMessage.id) as $MESSAGE_ID.

JavaScript — EventSource

const sessionId = process.env.SESSION_ID; // same session used to send the message
const assistantId = process.env.MESSAGE_ID; // assistantMessage.id from the send-message response

const stream = new EventSource(
  `${BASE_URL}/ai/sessions/${sessionId}/messages/${assistantId}/stream`,
  { withCredentials: true },
);

stream.addEventListener("tool.progress", (event) => {
  const { payload } = JSON.parse(event.data);
  if (payload.activityType === "audio_generation") {
    console.log("progress:", payload.message);
  }
});

stream.addEventListener("tool.completed", (event) => {
  const { payload } = JSON.parse(event.data);
  if (payload.toolKey === "media.audio.generate") {
    console.log("Audio ready:", payload.data.audioUrl);
    stream.close();
  }
});

stream.addEventListener("run.settlement.completed", () => stream.close());

Python — sseclient

import os
import sseclient, requests, json

session_id = os.environ["SESSION_ID"]  # same session used to send the message
assistant_id = os.environ["MESSAGE_ID"]  # assistantMessage.id from the send-message response

response = requests.get(
    f"{BASE_URL}/ai/sessions/{session_id}/messages/{assistant_id}/stream",
    headers={"Authorization": f"Bearer {TOKEN}"},
    stream=True,
)
client = sseclient.SSEClient(response)
for event in client.events():
    if event.event == "tool.completed":
        payload = json.loads(event.data)["payload"]
        if payload["toolKey"] == "media.audio.generate":
            print("audio:", payload["data"]["audioUrl"])
            break

---

Direct REST Path

Description

Use POST /ai/generative-media/context-runs when you want programmatic control or to combine audio generation with personal context (e.g. the user's Chainies or memory).

POST /ai/generative-media/context-runs
Authorization: Bearer $TOKEN
Content-Type: application/json

Requires ai.contextMedia.generate and ai.audio.generate.

Request

Request Body

Field	Type	Required	Description
prompt	string	Yes	Music description — max 4000 chars
mediaType	string	Yes	Must be "audio"
generationMode	string	No	context_aware (default), prompt_only
requestedContextSources	string[]	No	Personal context sources to include (see Generative Media API)
durationSeconds	number	No	1–30 seconds
bpm	number	No	40–220 BPM
styleTokens	string[]	No	Max 10 items, max 60 chars each. Merged with context-derived style tokens
sessionId	UUID	No	Links the generation job to a chat session
providerPreference	string	No	Override provider routing (max 40 chars)
modelPreference	string	No	Override model within the provider (max 80 chars)

Response

{
  "data": {
    "jobId": "550e8400-e29b-41d4-a716-446655440010",
    "status": "completed",
    "mediaType": "audio",
    "generationMode": "prompt_only",
    "permissionRequired": false,
    "allowedContextSources": [],
    "blockedContextSources": [],
    "resultMediaId": "550e8400-e29b-41d4-a716-446655440020",
    "errorCode": null,
    "createdAt": "2026-05-22T10:00:00Z",
    "updatedAt": "2026-05-22T10:00:25Z"
  }
}

Use resultMediaId to retrieve the audio file via the Files API.

Job Status Values

Status	Meaning
pending	Job accepted, generation not yet started
permission_required	No consented context sources; proceed with prompt_only or grant permissions
running	Generation in progress
completed	Audio generated and stored; resultMediaId is set
generation_failed	Lyria returned an error or empty result
storage_failed	Generation succeeded but R2 upload failed
cancelled	Cancelled by the user

Error Responses

HTTP	Meaning
400	Validation failed — e.g. bpm not in 40–220, styleTokens item exceeds 60 chars
401	Missing or invalid JWT
403	Plan does not include ai.contextMedia.generate or ai.audio.generate
429	Rate limit exceeded

Code Examples

cURL — prompt only

curl -X POST "$BASE_URL/ai/generative-media/context-runs" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "A reflective lo-fi hip hop beat for studying.",
    "mediaType": "audio",
    "generationMode": "prompt_only",
    "durationSeconds": 25
  }'

cURL — with context and style

curl -X POST "$BASE_URL/ai/generative-media/context-runs" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "Create background music that fits my mood today.",
    "mediaType": "audio",
    "generationMode": "context_aware",
    "requestedContextSources": ["preferences", "chainies"],
    "durationSeconds": 20,
    "bpm": 90,
    "styleTokens": ["calm", "acoustic", "morning"]
  }'

JavaScript

const res = await fetch(`${BASE_URL}/ai/generative-media/context-runs`, {
  method: "POST",
  headers: {
    Authorization: `Bearer ${TOKEN}`,
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    prompt: "Upbeat electronic music for a productivity sprint.",
    mediaType: "audio",
    generationMode: "prompt_only",
    durationSeconds: 15,
    bpm: 120,
    styleTokens: ["electronic", "upbeat", "synth"],
  }),
});
const { data } = await res.json();
console.log("Job:", data.jobId, "Status:", data.status);
if (data.status === "completed") {
  // Fetch the presigned audio URL via the files API using data.resultMediaId
}

Python

import requests

res = requests.post(
    f"{BASE_URL}/ai/generative-media/context-runs",
    headers={
        "Authorization": f"Bearer {TOKEN}",
        "Content-Type": "application/json",
    },
    json={
        "prompt": "A jazz trio improvisation, late night bar setting.",
        "mediaType": "audio",
        "generationMode": "prompt_only",
        "durationSeconds": 30,
        "bpm": 75,
        "styleTokens": ["jazz", "piano", "bass", "brushed-drums"],
    },
)
data = res.json()["data"]
print(f"Job {data['jobId']}: {data['status']}")

---

Parameter Behaviour

Duration

• Default: 20 seconds when durationSeconds is omitted.
• Maximum: 30 seconds. Values above 30 are clamped to 30.
• Invalid: Values ≤ 0 (including zero and negative numbers) fall back to the default (20 s).
• The actual duration returned in the response reflects what the model produced, which may differ slightly from the requested value. Billing is calculated from the actual duration.

BPM

• Range: 40–220. Values outside this range are clamped to the nearest bound.
• Invalid: Non-numeric values, zero, and negative numbers are silently omitted from the generation request. The model infers its own tempo.
• When BPM is omitted, Lyria selects a tempo appropriate to the prompt and style tokens.

Style Tokens

• Free-form strings that describe genre, instrument, or mood (e.g. "jazz", "piano", "cinematic", "upbeat").
• Passed to Lyria as weighted prompts alongside the main prompt.
• In the context-aware REST path, user-provided tokens are merged with context-derived tokens built from the user's personal data. User tokens are appended and carry equal weight.
• No vocabulary constraint applies — any string is accepted, subject to the length limit (60 chars per token, max 10 tokens via REST; max 10 tokens implicit via the Chao tool schema).

---

Billing

Audio generation is billed by duration:

• Unit: minutes (durationSeconds / 60)
• Credits are reserved before the generation request is sent.
• Credits are committed using the actual duration returned by Lyria (or the requested duration if Lyria does not report one).
• If generation produces no audio, the reserved credits are released and no charge is applied.

---

Behaviour Notes

• Backward compatibility: Sending { content, capabilityKey: "ai.audio.generate" } without any optional parameters works exactly as before. All structured parameters are optional.
• Storage: Generated audio is uploaded to the workspace's R2 bucket and returned as a time-limited presigned URL. Fetch a fresh URL via the Files API once the original expires.
• Empty result: If Lyria produces no audio chunks, the tool returns success: false with an empty audioUrl. No charge is applied.
• Approval gate: The Chao tool is requiresApproval: true — the user must confirm before audio generation executes inside a chat session.

---

Related

• AI Messages — generic message / streaming pipeline used by all AI capabilities.
• AI Sessions — create the session that hosts a generation run.
• Generative Media API — full context-aware media pipeline with per-source consent controls.
• AI Video Generation — equivalent docs for video generation.
• Files API — retrieve generated media by resultMediaId.