Stream audio

Audio format


Encoding	Raw 16-bit PCM (PCM16), little-endian
Channels	Mono
Sample rate	`8000`, `12000`, `16000`, `24000` (default) or `48000` Hz
Accepted as	`Int16Array`, `Buffer`, `Uint8Array`, `ArrayBuffer`, or a base64 string
Chunk size	Any. A chunk may even split a sample.

Set the sample rate when you start the session. It must match your audio:

const avatar = await LiveAvatar.start({ apiKey, avatar: "ava4", sampleRate: 16000 });

You can also set it per chunk: avatar.pushAudio(pcm, { sampleRate: 16000 }).

Audio can arrive faster than real time. It’s queued and spoken at its natural pace, one chunk after another.

Don’t send compressed audio. If your TTS only returns MP3, Opus or WAV, decode it to PCM first, and strip WAV headers.

Don’t play the audio yourself. The avatar plays it.

Provider settings

OpenAI Realtime (WebSocket, server side)

Realtime audio deltas are base64 PCM16 at 24 kHz, so pass them straight through:

realtime.on("message", (event) => {
  if (event.type === "response.output_audio.delta") {   // "response.audio.delta" on older versions
    avatar.pushAudio(event.delta);
  }
  if (event.type === "input_audio_buffer.speech_started") {
    interrupt();                                         // see Interruptions
  }
});

If your page talks to OpenAI Realtime directly over WebRTC, the voice never reaches your server. Move the Realtime session to your server (WebSocket) to use Realistic Avatars, or use the Open Source avatar in the browser.

OpenAI TTS

Ask for response_format: "pcm": 24 kHz PCM16 mono. Stream the response body into pushAudio:

const response = await openai.audio.speech.create({
  model: "gpt-4o-mini-tts",
  voice: "coral",
  input: text,
  response_format: "pcm",
});
for await (const chunk of response.body) avatar.pushAudio(chunk);

ElevenLabs

Use output_format: "pcm_24000" with sampleRate: 24000, or pcm_16000 with sampleRate: 16000.

Azure Speech

Use the output format Raw24Khz16BitMonoPcm with sampleRate: 24000.

Cartesia

Use { container: "raw", encoding: "pcm_s16le", sample_rate: 24000 } with sampleRate: 24000.

Any other TTS

Ask for raw 16-bit mono PCM at one of the supported sample rates, and pass that rate as sampleRate.

Get started

Your account

Voice Agents (no code)

Realistic Avatars (API)

Open Source (3D, free)

Help

Audio format

Provider settings

Know when it’s speaking

​Audio format

​Provider settings

​Know when it’s speaking

Audio format

Provider settings

Know when it’s speaking