Paxa Labs
Sign in

LiveKit text to speech for voice agents

A LiveKit voice agent whose users speak more than one language needs a voice that reads mixed sentences. Point LiveKit's OpenAI plugin at Paxa with a base URL and an API key, and the agent speaks English, Thai, and Mandarin in any Paxa TTS Flash v2 voice. The same plugin adds Paxa transcription.

Start freeRead the docs

One plugin, pointed at Paxa#

The speech and transcription objects come from LiveKit's OpenAI plugin, set with a Paxa base URL, key, model, and voice. The notes on each argument are the setup guide's own.

import os

from livekit.plugins import openai

paxa_tts = openai.TTS(
    base_url="https://api.paxalabs.com/v1",
    api_key=os.environ["PAXA_API_KEY"],
    model="paxa-tts-flash-v2",
    voice="khanomkrok",
    response_format="pcm",
)

Give the agent a voice

  • Set model to paxa-tts-flash-v2, or keep OpenAI's model names, which run on it. The plugin asks for an event stream on most model names, and Paxa answers with raw audio, which the plugin plays as it arrives.
  • Set voice to any voice id from GET /v1/voices whose model is paxa-tts-flash-v2. OpenAI's voice names also work (accepted voice names).
  • Keep response_format="pcm". It returns 16-bit mono samples at 24 kHz, the rate the plugin plays, with no decoding step.
  • Set speed between 0.5 and 1.5. A value outside that range answers 400.

Add transcription

  • The plugin transcribes each finished turn as one recording. The session needs a VAD to find where each turn ends.
  • language passes a hint, such as th. Set detect_language=True to send none and let the model detect the language.
  • Paxa STT Lite is a Research preview.

From a key to a speaking agent#

The agent keeps its LLM and its session code. Paxa takes the speech side, and the transcription side when you add it.

  1. Install the plugin

    Install livekit-agents with its openai extra, and set PAXA_API_KEY to a key from the dashboard.

  2. Point the TTS at Paxa

    Create openai.TTS with base_url set to https://api.paxalabs.com/v1, model set to paxa-tts-flash-v2, a Paxa voice id, and response_format set to pcm, the format the plugin plays without decoding.

  3. Add transcription

    Create openai.STT with the same base URL and model paxa-stt-lite-v1-preview, and pass both to AgentSession with a VAD to find where each turn ends.

Paxa TTS Flash v2 lists at $10 per 1M characters. Paxa STT Lite lists at $0.50 per hour of audio, billed by the millisecond.

Questions#

What is LiveKit Agents?

LiveKit describes Agents as a realtime framework for voice, video, and physical AI agents. A voice agent streams audio through speech to text, an LLM, and text to speech, and Paxa can serve the speech to text and text to speech stages.

Which voices can the agent use?

The agent can use any of the 50 Paxa TTS Flash v2 voices, and each one speaks English, Thai, and Mandarin. GET /v1/voices lists their ids, and the voice library plays them. OpenAI's voice names also work and map to Paxa voices.

When does a long Thai reply start speaking?

The agent speaks a reply one sentence at a time. Thai written without punctuation counts as one sentence, and a long Thai reply then starts speaking only after the LLM finishes it.

How does transcription work in LiveKit?

openai.STT sends each finished turn to Paxa as one recording, and the session's VAD finds where each turn ends. Pass language as a hint, such as th, or set detect_language to True to let the model detect the language. Paxa STT Lite is a Research preview.

Start with free credits

Sign in with Google, GitHub, or Hugging Face and spend 100 one-time free credits on your own text.

Start free

Products

Use cases

Integrations