RTVI events

Full catalog of events emitted by a Breeze Buddy Daily session, plus client → server messages like tts-speak.

Breeze Buddy streams the entire session — transport state, user speech, bot speech, LLM tokens, function calls, pipeline metrics — to the browser over the Pipecat RTVI protocol. The backend uses Pipecat’s RTVIObserver + RTVIProcessor, so every standard RTVI event reaches the client, and Breeze Buddy adds four custom server messages on top.

Enabling events

RTVI events are gated by a single environment flag:

ENABLE_BREEZE_BUDDY_DAILY_EVENTS=true

Events dispatch automatically for every Daily session (both DAILY agent mode and DAILY_STREAM stream mode). Register listeners via the callbacks object on PipecatClient — no per-event opt-in is required.

import { PipecatClient } from '@pipecat-ai/client-js';
import { DailyTransport } from '@pipecat-ai/daily-transport';
//
const client = new PipecatClient({
  transport: new DailyTransport(),
  callbacks: { onBotConnected: () => console.log('bot connected') }
});
await client.initDevices();
await client.startBotAndConnect({
  endpoint: `${baseUrl}/agent/voice/breeze-buddy/connect`,
  headers: new Headers({ Authorization: `Bearer ${apiToken}` }),
  requestData: { lead_id: leadId }
});

Mode availability at a glance

Two execution modes serve a Daily session; not every event fires in both.

Connection & transport

Participants

Bot lifecycle

User speech

let interim = '';
client.on('userTranscript', (d) => {
  if (d.final) {
    appendToTranscript('user', d.text);
    interim = '';
  } else {
    interim = d.text;
    renderInterim(interim);
  }
});

Bot speech (TTS)

client.on('botTtsText', (d) => appendToStream('bot', d.text));
client.on('botStoppedSpeaking', () => finalizeStream('bot'));

Bot LLM (agent mode only)

Function calls (agent mode only)

client.on('llmFunctionCallStopped', (data) => {
  if (data.function_name === 'appointment_confirmed') {
    showConfirmation(data.arguments.date, data.arguments.time);
  }
});

Audio levels & metrics

Device management

Standard Daily device events reach the client unchanged: onAvailableMicsUpdated, onAvailableSpeakersUpdated, onAvailableCamsUpdated, onMicUpdated, onSpeakerUpdated, onCamUpdated, onDeviceError, onTrackStarted, onTrackStopped. Use them to build device pickers and mic-level visualizers; see the Daily SDK docs for payloads.

Custom server messages

Breeze Buddy sends four application-level messages on top of RTVI. They arrive as serverMessage events with a typed type field.

client.on('serverMessage', (msg) => {
  switch (msg.type) {
    case 'conversation-end':
      toast(`Call ended: ${msg.data.reason}`);
      break;
    case 'pipeline-error':
      reportError(msg.data.processor, msg.data.error);
      break;
  }
});

Client-to-server messages

The client can push messages back to the bot via client.sendClientMessage(type, data). Today one type is wired end-to-end: tts-speak, available in DAILY_STREAM mode.

tts-speak — client-driven bot utterance

In stream mode (no LLM, no template flow), the client decides what the bot says. Send a tts-speak message and the backend queues a TTSSpeakFrame into the pipeline; it plays through the normal TTS → transport path and fires onBotTts* / onBotStartedSpeaking events.

// Stream mode only — requires execution_mode: "DAILY_STREAM" on the lead
await client.sendClientMessage('tts-speak', {
  text: 'Hello! How can I help you today?'
});

Behaviour

• Mode gate: only handled when the lead’s execution_mode is DAILY_STREAM. Full agent modes do not expose this path.
• Queued, not interrupting: the utterance appends to the FIFO frame queue — if the bot is already speaking, the new text plays after the current utterance finishes. For barge-in style interruption, interrupt the bot via the user speaking (VAD will handle it).
• Max length: text over 2000 characters is silently truncated.
• Auth: no per-message auth — any client connected to the room can trigger tts-speak. Treat room access as the security boundary.
• Observability: each utterance fires the normal onBotTtsStarted, onBotTtsText, onBotTtsStopped, onBotStartedSpeaking, onBotStoppedSpeaking, and onBotOutput events — you can reflect spoken text in your UI from the same listeners you already have.

Error handling

Pipeline and transport errors arrive on two channels — choose based on which you want to surface.

client.on('error', (e) => {
  console.error('[rtvi]', e.type, e.message);
  if (e.type === 'connection-failed') showRetry();
});
client.on('serverMessage', (m) => {
  if (m.type === 'pipeline-error') {
    reportError(m.data.processor, m.data.error);
  }
});

Patterns

type State = 'idle' | 'listening' | 'thinking' | 'speaking';
let state: State = 'idle';
//
client.on('userStartedSpeaking', () => (state = 'listening'));
client.on('userTranscript', (d) => { if (d.final) state = 'thinking'; });
client.on('botStartedSpeaking', () => (state = 'speaking'));
client.on('botStoppedSpeaking', () => (state = 'idle'));
//
client.on('metrics', (m) => trackLatency(m.total_latency_ms));