Pick the API that matches the interaction
For a live voice assistant, use xAI’s Speech to Speech API. For a spoken script, use its Text to Speech API. Keep these as separate provider operations in your code: one is a long-lived audio conversation; the other produces an audio result from text.
A small server-owned interface helps you change providers without rebuilding the page:
type VoiceInput = { text: string; language: 'en' | 'zh' };
type VoiceJob = { ticket: string; status: 'queued' };
interface NarrationProvider {
submit(input: VoiceInput, requestId: string): Promise<VoiceJob>;
}
This is an application interface, not an xAI SDK. Keep the provider name, internal job identifier, and permanent credentials out of browser responses.
Build the request boundary first
Your browser sends text to your own server. The server checks origin, request size, supported language, and usage limits. It then submits one task with a stable request identifier and returns a scoped ticket. The browser polls your status endpoint and receives an audio URL only after success.
Treat a retry as the same request until its outcome is known. Disabling a button improves usability but does not replace server-side idempotency. A double click, lost response, or network retry must not create two billable jobs.
Connect a real-time conversation
Use the official Speech to Speech session protocol for microphone audio and response events. Your server should issue a short-lived client credential using the documented ephemeral-token flow. The permanent key stays on the server. Enforce a session-duration limit and close the connection when the user stops.
Keep audio capture, audio encoding, transport, and playback as separate modules. Record event names, timings, and error categories for diagnosis; avoid recording raw conversations by default.
Before enabling public traffic
- Test empty input, unsupported language, oversized requests, and repeated submissions.
- Verify that one browser cannot fetch another browser’s result.
- Reject arbitrary upstream URLs; the server owns endpoint selection.
- Set bounded connection and generation timeouts.
- Return actionable public errors without provider bodies or credentials.
- Check audio playback and download, not just a successful JSON response.
Use the WebSocket guide for the real-time state machine and the troubleshooting guide for failure isolation. This tutorial is an architecture walkthrough; current request fields and events remain defined by the official documentation.