> ## Documentation Index
> Fetch the complete documentation index at: https://docs.talking-avatar.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Audio sources

> Lip sync audio clips, live streams, microphones and WebRTC voice agents.

The avatar lip syncs any audio you give it, and plays that audio itself. There are two inputs.

## Audio clips

`speak` takes a URL, an `ArrayBuffer` or a `Blob`, in any format the browser can decode (MP3, WAV, Ogg, AAC…). The clip is analysed in full before it starts (in roughly 1% of its duration), then played with no added delay.

```js theme={null}
await avatar.speak("/greeting.mp3");          // resolves when the clip ends or is stopped
await avatar.speak(await ttsResponse.blob()); // e.g. straight from a TTS API
avatar.stopSpeaking();
```

Calling `speak` again stops the current clip.

## Live audio streams

`attachStream` takes any `MediaStream` with an audio track: a WebRTC remote stream, `getUserMedia`, or `element.captureStream()`.

```js theme={null}
await avatar.attachStream(stream); // plays it and lip syncs it
avatar.detachStream();             // stop
```

Attaching a new stream replaces the previous one. `detachStream` doesn't stop the source track; whoever owns it does.

### Stream delay

The stream plays `streamDelayMs` (default **150 ms**) late. The model hears each sound that long before the listener does, so the mouth can move into it on time. Lower it for less latency, at the cost of rougher lip sync:

```js theme={null}
new TalkingAvatar({ container, avatarUrl, streamDelayMs: 80 });
```

## Voice agents

Give the agent's incoming audio to `attachStream` **instead of** playing it in an `<audio>` element.

**OpenAI Realtime (WebRTC)**

```js theme={null}
const pc = new RTCPeerConnection();
pc.ontrack = (event) => avatar.attachStream(event.streams[0]);
```

**LiveKit**

With `livekit-client`, for example with LiveKit Agents:

```js theme={null}
import { RoomEvent, Track } from "livekit-client";

room.on(RoomEvent.TrackSubscribed, (track) => {
  if (track.kind === Track.Kind.Audio) {
    avatar.attachStream(new MediaStream([track.mediaStreamTrack])); // don't call track.attach()
  }
});
```

**Daily**

With `@daily-co/daily-js`, for example with Pipecat:

```js theme={null}
call.on("track-started", ({ track, participant }) => {
  if (track.kind === "audio" && !participant?.local) avatar.attachStream(new MediaStream([track]));
});
```

**Any WebRTC stack**

Find the remote audio `MediaStreamTrack` and pass `new MediaStream([track])`:

```js theme={null}
function onTrack(event) {
  if (event.track.kind !== "audio") return;
  avatar.attachStream(new MediaStream([event.track])).catch(console.error);
}
pc.addEventListener("track", onTrack);

// When leaving the conversation:
// pc.removeEventListener("track", onTrack);
// avatar.detachStream();
// avatar.dispose();
```

## Microphone

```js theme={null}
const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
await avatar.attachStream(mic);
```

The avatar plays the microphone back, so use headphones to avoid echo. Live audio needs HTTPS (or `localhost`).

## Listening and thinking

The avatar knows when it's speaking from its own audio. In between, it looks like it's listening: attentive brows and a relaxed blink rate. While your agent is working on a reply, show it thinking:

```js theme={null}
avatar.setStatus("thinking");  // e.g. after the user stops talking
avatar.setStatus("listening"); // back to the default
```

Speech detected in the audio always takes priority over the status.

## Best results

> [!TIP]
>
> **Use TTS voices for the best lip sync.** The model was trained on clean, clearly articulated read speech, which is what TTS engines produce. Real voices, such as from a microphone, also work, but with background noise, echo or unclear speech the lip sync is less accurate.

* **One audio source at a time** per avatar: attaching a stream replaces the previous one. A clip and a stream can overlap.
* **The avatar plays the audio.** It can't lip sync audio that's played somewhere else without also playing it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.