WebSocket API

​

End-to-end flow

Client ──► Connect /ws
          └─► send {"type":"config", ...}
Sayna  ──► validate + spin up providers + optional LiveKit join
Sayna  ──► send {"type":"ready", ...}
Client ◄── start streaming audio / send commands
Client & Sayna ── keep exchanging JSON control frames + binary audio
Either side closes socket → Sayna tears down providers + LiveKit room

​

Message taxonomy

​

Configuration message

{
  "type": "config",
  "stream_id": "support-call-123",
  "audio": true,
  "stt_config": {
    "provider": "deepgram",
    "language": "en-US",
    "sample_rate": 16000,
    "channels": 1,
    "punctuation": true,
    "encoding": "linear16",
    "model": "nova-2"
  },
  "tts_config": {
    "provider": "deepgram",
    "model": "aura-asteria-en",
    "voice_id": "aura-asteria-en",
    "speaking_rate": 1.0,
    "audio_format": "linear16",
    "sample_rate": 24000
  },
  "livekit": {
    "room_name": "conversation-room-123",
    "enable_recording": true,
    "sayna_participant_identity": "sayna-ai",
    "sayna_participant_name": "Sayna AI",
    "listen_participants": []
  }
}

• audio=false skips STT/TTS initialization but still allows LiveKit control + data channel relays.
• stream_id is optional; provide one to control recording paths and correlate sessions, or let Sayna generate a UUID v4 and return it in the ready message.
• stt_config.sample_rate/channels/encoding must exactly match the binary frames you send—Sayna will not resample.
• tts_config values become session defaults; individual speak messages can override most fields or add id for correlation.
• The livekit block is optional but required for send_message and mirrored playback/recording.

​

Streaming audio best practices

• Chunk size: ~100–200 ms of audio per binary frame (e.g., 3 200 bytes for 16 kHz mono linear16).
• Maintain consistent cadence; bursty uploads increase STT latency.
• Enable punctuation + turn-detect (default features) when you need is_speech_final cues to know when a speaker stopped.
• If the client detects a barge-in (human interrupts AI), send {"type":"clear"} before queueing new speak text.

​

Client commands in detail

​

speak

{
  "type": "speak",
  "text": "Hello! How can I help you today?",
  "flush": true,
  "allow_interruption": true,
  "tts_config": {"voice_id": "aura-luna-en"},
  "id": "greeting-1"
}

• flush=true clears queued audio before speaking; set false to build playlists.
• allow_interruption=false protects critical prompts even if the client sends clear.
• tts_config may override any default except provider credentials.
• Finish signal arrives via tts_playback_complete with the same id.

​

send_message

{
  "type": "send_message",
  "message": "{\"action\":\"typing\"}",
  "role": "assistant",
  "topic": "status",
  "debug": {"source": "sayna-ai", "version": "1.0"}
}

​

Server events in detail

​

ready

​

stt_result

{
  "type": "stt_result",
  "transcript": "Hello, how are you today?",
  "is_final": true,
  "is_speech_final": true,
  "confidence": 0.94
}

• Interim updates set is_final=false and keep streaming as long as audio flows.
• The final chunk for an utterance flips is_final=true.
• is_speech_final=true indicates the turn detector heard silence—ideal for triggering a response.

​

message and participant_disconnected

{"type":"message","label":"livekit.joined","details":{"identity":"user-123"}}
{"type":"participant_disconnected","participant":{"identity":"user-123","name":"Alex"}}

​

Binary TTS payloads

socket.onmessage = (event) => {
  if (event.data instanceof Blob) playAudioChunk(event.data);
  else handleControl(JSON.parse(event.data));
};

​

LiveKit interplay

• Sayna manages the agent-side participant automatically; humans still fetch tokens via POST /livekit/token.
• listen_participants limits which identities can feed audio back into Sayna.
• Recordings respect your LiveKit/S3 configuration when enable_recording=true; files are stored at {server_prefix}/{stream_id}/audio.ogg using the stream_id from your config or the ready message.
• send_message and clear affect both the WebSocket client and LiveKit room so playback remains in sync.

​

Troubleshooting