> ## Documentation Index
> Fetch the complete documentation index at: https://docs.talking-avatar.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Set up with a coding agent

> A ready prompt for Claude Code, Codex, Cursor or any coding agent to add a Talking Avatar to your project.

Paste this prompt into Claude Code, Codex, Cursor or any coding agent, inside your project. It looks at your voice provider, server and page, tells you its plan, then wires the SDK into your own setup. Your API key stays in your environment (`TALKING_AVATAR_API_KEY`), never in the chat.

The same prompt is on the dashboard's **Quickstart** page, under **Copy prompt**.

> [!TIP]
>
> The prompt proposes a small Node.js service when your backend is in another language. To skip it, add a line asking the agent to use the [realtime avatar API](https://docs.talking-avatar.dev/api-reference/realtime/messages.md) over WebSocket instead of the SDK, with the link to that page.

````markdown Prompt theme={null}
# Task: add a Talking Avatar to this project

Add a realistic, lip-synced avatar to this app's voice agent with the Talking Avatar SDK
(`@interviewflowai/talking-avatar`). The avatar speaks the agent's voice in real time on the page. Fit it into the
existing code, framework and conventions. Don't build a separate demo app.

## Step 1: Explore first, then tell me your plan
Before changing anything, find out and report back in a few lines:
1. **The voice.** Which TTS or speech-to-speech provider produces the agent's voice (OpenAI TTS, OpenAI Realtime,
   ElevenLabs, Azure, Cartesia, Deepgram, Fish Audio…), in which file, and where that audio is played today.
2. **Where the audio lives.** On the server, or only in the browser (e.g. OpenAI Realtime over WebRTC straight from
   the page)? The avatar needs the voice on the server.
3. **The server.** Language and framework. The SDK's server part needs Node.js (22+, or 18/20 with `WebSocket`
   from the "ws" package).
4. **The page.** Which component shows the conversation, and the UI framework (React, Next.js, Vue, Svelte…).
5. **LiveKit.** Does the project already use LiveKit rooms?

Then state which mode you'll use (Step 2) and the files you'll touch, and implement.
**Stop and ask me first** if you'd need to move the voice from the browser to the server, add a Node.js service
next to a non-Node backend, or change authentication or other features that play audio.

## Step 2: Pick the mode
- **The project uses LiveKit** → LiveKit mode: the avatar joins the existing room as a participant.
- **Otherwise** → Direct mode: the server starts the avatar, and the page shows it with `LiveAvatarView`.
- **The voice agent runs in Python** (LiveKit Agents, Pipecat…) or another non-Node backend → propose a small
  Node.js service that runs the avatar session and receives the agent's audio from it, and ask me before building it.

## Step 3: Install and configure
- `npm install @interviewflowai/talking-avatar` in the Node.js server, and in the web app (Direct mode).
- Read the API key from the env var `TALKING_AVATAR_API_KEY`, and add it to `.env.example` and any config
  validation. Keys look like `tav_…` and are created at https://app.talking-avatar.dev → API keys.
- **Never** ask me to paste the key into chat, hard-code it, commit it, or send it to the browser.

## Step 4: Server (`@interviewflowai/talking-avatar/server`)
```js
import { LiveAvatar, LiveAvatarError } from "@interviewflowai/talking-avatar/server"

// One avatar per conversation: start it when the conversation starts.
const avatar = await LiveAvatar.start({
  apiKey: process.env.TALKING_AVATAR_API_KEY,
  avatar: "ava6",     // any avatar ID from the Avatars page, e.g. ava6 (casual, home study) or ava4 (professional, office)
  sampleRate: 24000,  // the TTS audio's real rate: 8000 | 12000 | 16000 | 24000 | 48000
  // LiveKit mode only: livekit: { url, token } (see Step 5b)
})

avatar.pushAudio(chunk)                // every chunk of the agent's voice as it streams (PCM16 mono)
const heardMs = await avatar.clear()   // the user interrupted: drops unplayed audio, returns ms of the reply heard
avatar.on("speechStarted", () => {})
avatar.on("speechEnded", (playedMs) => {})
avatar.on("close", (code, reason) => {})  // 1000 closed by you · 4001 bad key · 4002 balance under 1 min · 4003 3-hour limit
avatar.close()                         // the conversation is over: billing stops
```
`pushAudio` takes PCM16 mono as an `Int16Array`, `Buffer` / `Uint8Array` / `ArrayBuffer` (chunks may split a
sample) or a base64 string. Audio can arrive faster than real time: it's queued and spoken at its natural pace.

Direct mode also needs **one route** for the page (adapt it to the framework; protect it with the app's existing auth):
```js
app.post("/avatar", async (req, res) => {
  try { res.json(await avatar.handleView(req.body)) }
  catch (error) { res.status(503).json({ error: error.message }) }
})
```

Rules:
- **Lifecycle.** Exactly one `LiveAvatar` per conversation. `close()` it when the conversation ends, the user
  leaves, or anything fails, so billing stops. Several conversations at once → one route each, e.g.
  `/avatar/:conversationId` looking up that conversation's avatar.
- **Play the voice once.** The avatar plays the voice itself, in sync with the lips. Remove or disable the old
  playback of the agent's audio (an `<audio>` element, a speaker stream, `track.attach()`…) for this agent only.
- **Raw PCM.** Ask the TTS for raw 16-bit mono PCM and pass the real sample rate. If it can only return MP3, Opus or
  WAV, decode to PCM first (strip WAV headers).
- **Interruptions.** Call `clear()` wherever the app already handles barge-in.
- **Errors.** `LiveAvatar.start` rejects with a `LiveAvatarError` (`.code` as above) if a session can't start.
  Show a clear message (e.g. 4002 → "add credits"); never crash the voice agent because the avatar failed.

Provider recipes (use the one that matches):
- **OpenAI Realtime (WebSocket, server side):** for `response.output_audio.delta` events (`response.audio.delta` on
  older versions) call `avatar.pushAudio(event.delta)` (base64 PCM16, 24 kHz). On
  `input_audio_buffer.speech_started`: `const ms = await avatar.clear()`, then send
  `{ type: "conversation.item.truncate", item_id: <the assistant item playing>, content_index: 0, audio_end_ms: ms }`
  so the model only remembers what the user heard.
- **OpenAI TTS:** `response_format: "pcm"` → 24 kHz PCM16 mono; stream the response body into `pushAudio`.
- **ElevenLabs:** `output_format: "pcm_24000"` (or `pcm_16000`, matching `sampleRate`).
- **Azure Speech:** `Raw24Khz16BitMonoPcm`.
- **Cartesia:** `{ container: "raw", encoding: "pcm_s16le", sample_rate: 24000 }`.

## Step 5a: Page (Direct mode, `@interviewflowai/talking-avatar/live`)
```js
import { LiveAvatarView } from "@interviewflowai/talking-avatar/live"

const view = new LiveAvatarView({
  container: element,   // the video fills it: 16:9, never cropped; or pass your own <video>
  endpoint: "/avatar",  // or (body) => fetch("/avatar", { method: "POST", headers, body: JSON.stringify(body) }).then(r => r.json())
  onStatus: (status) => {},  // connecting | live | muted (call view.unmute() from a click) | reconnecting | closed
  onError: (error) => {},
})
startButton.onclick = () => view.connect()  // from a user click, so the browser allows the voice
// view.disconnect() when the conversation UI goes away
```
- Use the app's framework: create the view on mount (e.g. React `useEffect` with a ref), `disconnect()` on unmount.
- Connect from a user gesture (the existing "start call" button is ideal).
- It reconnects by itself after network drops; show its status if the UI has a place for it.

## Step 5b: LiveKit mode
- On the server, mint a **publish-only** LiveKit token for the avatar: identity `avatar-<room>`, grant
  `{ room, roomJoin: true, canPublish: true, canSubscribe: false }`, and pass `livekit: { url, token }` to
  `LiveAvatar.start`. Your LiveKit secrets stay on your server.
- The avatar joins with a video track named `avatar` and its voice as `avatar-audio`. Render them with the
  existing LiveKit client, and stop playing the agent's original voice track so it isn't heard twice.
- Skip `LiveAvatarView` and the `/avatar` route.

## Step 6: Verify, then report
- Run the project's own type-check, lint, tests and build.
- If you can, run it: the avatar appears and moves while listening, the lips match the voice, the voice is heard once
  (not doubled), interrupting stops it, and ending the conversation calls `close()`.
- Report: what you changed (files), the env var to set, how to try it, and anything you couldn't verify.

Pricing: $0.02 per minute of live session, billed per second from the dashboard's prepaid credits.
Docs: https://www.npmjs.com/package/@interviewflowai/talking-avatar
````


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.