Browser playback

With direct output, each browser receives the avatar’s video and voice over WebRTC. Your API key and the session’s WebSocket stay on your server; the browser only talks to one route of yours, which relays the WebRTC handshake to the session.

1. Your route

One POST route, protected by your app’s own authentication. It takes two kinds of body:

Request body	Your server does	Response
`{}`	Sends `{"type": "view", "id"}` and waits for the `view_offer` with that `id`	`{"viewId", "sdp"}` from the `view_offer`
`{"viewId", "sdp"}`	Sends `{"type": "view_answer", "id", "viewId", "sdp"}` and waits for `view_ready`	`{"viewId"}`

If the session replies with an error (or doesn’t reply within about 15 seconds), answer with an HTTP error, such as 503. One route serves any number of browsers. With several conversations at once, give each its own route, such as /avatar/{conversation_id}. This is exactly the contract of the SDK’s handleView, so the browser side can be the SDK’s player or your own code. See the Python example for a complete route.

2. The page

With LiveAvatarView (recommended)
Plain WebRTC

The SDK’s browser player works with any backend that implements the route above. It fits the video to its container, plays the voice, and reconnects by itself after network drops or a republished stream.

<div id="avatar" style="width: 640px; aspect-ratio: 16 / 9"></div>
<button id="start">Start</button>

<script type="module">
  import { LiveAvatarView } from "https://cdn.jsdelivr.net/npm/@interviewflowai/talking-avatar@0.2.0/dist/live.js";

  const view = new LiveAvatarView({
    container: document.getElementById("avatar"),
    endpoint: "/avatar", // your route
    onStatus: (status) => console.log(status),
  });
  document.getElementById("start").onclick = () => view.connect();
</script>

With a bundler, npm install @interviewflowai/talking-avatar and import from @interviewflowai/talking-avatar/live. See the LiveAvatarView reference.

No SDK at all: the browser receives the offer, answers it, and plays the tracks.

<video id="avatar" playsinline autoplay style="width: 640px; aspect-ratio: 16 / 9; background: #000"></video>
<button id="start">Start</button>

<script type="module">
  const post = (body) =>
    fetch("/avatar", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify(body) })
      .then((r) => (r.ok ? r.json() : Promise.reject(new Error(`HTTP ${r.status}`))));

  async function watch(video) {
    const pc = new RTCPeerConnection({ iceServers: [{ urls: "stun:stun.l.google.com:19302" }] });
    const stream = new MediaStream();
    pc.ontrack = (event) => {
      stream.addTrack(event.track);
      if (video.srcObject !== stream) video.srcObject = stream;
    };
    const offer = await post({});                                   // 1. the session's offer
    await pc.setRemoteDescription({ type: "offer", sdp: offer.sdp });
    await pc.setLocalDescription(await pc.createAnswer());          // 2. the browser's answer
    await post({ viewId: offer.viewId, sdp: pc.localDescription.sdp });
    return pc;
  }

  const video = document.getElementById("avatar");
  document.getElementById("start").onclick = async () => {
    await watch(video);
    await video.play(); // from the click, so the browser lets the voice play
  };
</script>

Your code then handles what LiveAvatarView does for you:

Reconnect when pc.connectionState becomes failed, or when video frames stop arriving for a couple of seconds (watch framesReceived in pc.getStats()), by running watch() again with a new RTCPeerConnection.
After republished, every browser must run watch() again.
Autoplay: if video.play() is rejected, play it muted and show a button that unmutes it.

Networks

The avatar’s stream works for most networks with a public STUN server. For users behind strict corporate firewalls, add your own TURN server to iceServers (in LiveAvatarView, its iceServers option).

Sound

Browsers block sound until the user interacts with the page. Start watching from a click, such as your “Start call” button.

Overview

Realtime avatar API

Private agent links

Requests we send you

1. Your route

2. The page

Networks

Sound

​1. Your route

​2. The page

​Networks

​Sound

1. Your route

2. The page

Networks

Sound