Interruptions and speech events - Talking Avatar

Interrupt the avatar

When the user starts talking over the avatar, call clear(). It drops the audio that hasn’t played yet, stops the avatar mid-sentence, and resolves with how many milliseconds of the interrupted reply the user heard.

const heardMs = await avatar.clear();

Call it wherever your app already handles barge-in, such as your voice activity detector’s speech started event.

Keep the transcript accurate

Your agent’s model generated the whole reply, but the user only heard part of it. Use heardMs to tell the model what was actually said. With OpenAI Realtime, truncate the assistant’s item to what was heard:

realtime.on("message", async (event) => {
  if (event.type === "input_audio_buffer.speech_started") {
    const ms = await avatar.clear();
    realtime.send({
      type: "conversation.item.truncate",
      item_id: currentAssistantItemId, // the assistant item that was playing
      content_index: 0,
      audio_end_ms: ms,
    });
  }
});

Speech events

Event	Fires when
`speechStarted`	The avatar starts speaking (a reply’s first audio is playing)
`speechEnded(playedMs)`	Everything queued has played. `playedMs` is how much of this reply was heard.

avatar.on("speechStarted", () => showSpeakingIndicator());
avatar.on("speechEnded", (playedMs) => hideSpeakingIndicator());

on() returns a function that removes the listener:

const off = avatar.on("speechEnded", handler);
off();

Use these events to drive your UI (a speaking indicator, or a thinking state), or to know when the avatar has finished a goodbye before you close the session. See the full list of events in the LiveAvatar reference.