Paxa Labs
Sign in

Live speech

WSwss://api.paxalabs.com/v1/tts/live

WS /v1/tts/live holds one WebSocket for many syntheses: text frames in, audio frames out. Billed cumulatively per character, one minimum per connection.

View as Markdown
// Runs on Bun as-is; on Node, `import WebSocket from "ws"` accepts
// the same constructor options. Browsers cannot send this header, and
// API keys never ship to browsers anyway.
const socket = new WebSocket("wss://api.paxalabs.com/v1/tts/live", {
  headers: { Authorization: `Bearer ${process.env.PAXA_API_KEY}` },
});
socket.binaryType = "arraybuffer";

socket.onopen = () => {
  socket.send(JSON.stringify({
    type: "start",
    model: "paxa-tts-flash-v2",
    voice: "khanomkrok",
    timestamps: "word",
  }));
  // Send text whenever it becomes available, token by token if you like.
  socket.send(JSON.stringify({ type: "text", text: "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs" }));
  socket.send(JSON.stringify({ type: "flush" }));
  socket.send(JSON.stringify({ type: "end" }));
};

socket.onmessage = (event) => {
  if (typeof event.data === "string") {
    const message = JSON.parse(event.data);
    if (message.type === "done") socket.close();
    return;
  }
  player.feed(new Uint8Array(event.data)); // binary frames are audio
};
{"type":"started","connection":"7d9c2b7e-4a41-4c8e-9a35-1f26d8c04b7a","model":"paxa-tts-flash-v2","voice":"khanomkrok","format":"mp3","sample_rate":44100,"timestamps":"word"}
{"type":"charged","seq":1,"chars":36,"credits":0.45,"total_credits":0.45}
{"type":"timing","seq":1,"spans":[{"text":"สวัสดี","start":0,"end":0.51},{"text":"ครับ","start":0.51,"end":0.85}]}
{"type":"flushed","seq":1}
{"type":"done","total_chars":36,"total_credits":0.45}

Connect once, send start, then interleave text, flush, cancel, and end. An open connection holds one of the plan's concurrent speech slots until it closes. The Live streaming guide covers the lifecycle and billing.

Headers#

authorizationBearer pxa_...Required

Authorization: Bearer pxa_... (recommended).

x-api-keypxa_...

Alternative to the Authorization header. Ignored when Authorization is present.

Authentication

Connect#

wss://api.paxalabs.com/v1/tts/live Binary frames carry audio in the session's format. JSON text frames carry control, billing, and timing events.

Client messages#

startConfigures the session. Must be the first frame; exactly one per connection.

type"start"Required
modelstringRequired

TTS model id, for example paxa-tts-flash-v2.

voicestringRequired

Voice id, resolved as on POST /v1/tts. A designed voice serves the connection until it closes, even if it is deleted meanwhile.

format"mp3" or "opus" or "wav" or "pcm" or "mulaw" or "alaw"

Audio format for the session's binary frames, as format on POST /v1/tts.

Default "mp3"
sample_rate8000 or 16000 or 22050 or 24000 or 32000 or 44100 or 48000

Sample rate of pcm and wav, in Hz. Paxa TTS Flash v2 serves 8000, 16000, 22050, 24000, 32000, 44100, 48000, and Paxa TTS Flash serves 24000 only. For any other format the value may only restate that format's own rate, which is 8000 for mulaw and alaw. Fixed for the connection, like format. A rate the model doesn't serve answers unsupported_format. A request costs the same at every rate.

Example 16000
timestamps"word" or "sentence" or "utterance"

Adds timing events to the session at 1.25x the standard rate, applied to the connection's cumulative character count.

speednumber

Speaking rate, as a multiplier on the voice's own design speed. It runs from 0.5 to 1.5, and 1 is the voice as designed. Rate changes how the audio sounds. It does not change how many characters were sent. A request costs the same at every rate. Values round to the nearest 0.05. Fixed for the connection, like model and voice.

Default 1min 0.5max 1.5Example 1.1
language"auto" or "th" or "en" or "zh"

Sets how codes, digits, and symbols are read. Words keep the language of their script, and a number inside Thai or Chinese text keeps that text's language. Elsewhere, "th" reads them in Thai (AB2039 with Thai letter names and digits) and "en" in English. "zh" reads codes in English, a number next to Chinese characters in Mandarin, and other digits and symbols in Thai. "auto", the default, takes the language most of the text is written in, counting English only in words of three or more letters, and Thai on a tie. Under "auto", a bare number, AB2039, or 250 ml reads as Thai, and ISO9001 or 100 USD as English. Text with no Thai, Chinese, digits, or counted English word reads as English when a two-letter word has a lowercase letter, such as Hi or I'm OK. Pin "th" or "en" to choose how a short number or code reads. A request costs the same in every language. Fixed for the connection, like model and voice. "auto" resolves on each leg's own text, and a leg that is only a number reads as Thai.

Default "auto"Example "en"
temperaturenumber

Sets how much the pacing, pauses, breaths, and melody of a reading vary from one request to the next. It runs from 0 to 2, and 0.5 is the default. They vary least at 0 and most at 2. Paxa TTS Flash v2 reads it. Any other model accepts only a value that rounds to 0.5, and any other answers validation. A request costs the same at every temperature. Values round to the nearest 0.1. Fixed for the connection, like model and voice. A refused value answers an in-band validation error, and the connection stays open for a corrected start frame.

Default 0.5min 0max 2Example 0.5

textAppends text. The server auto-flushes on a newline, sentence-ending punctuation, or a buffered-length threshold, holding back a tag still open at the end of the buffer for up to that threshold.

type"text"Required
textstringRequired

Text appended to the synthesis buffer. A frame longer than the model's max_chars answers an in-band text_too_long error. Every Paxa TTS Flash v2 voice reads tags, as does a Paxa TTS Flash (paxa-tts-flash-v1) voice whose emotion_tags list is not empty. On those voices, tags that no words follow yet wait in the buffer, and each leg opens with the tags the previous leg ended in. A frame that would take the waiting tags and the opening tags past max_chars also answers text_too_long, as does one that would leave opening tags longer than half of max_chars. A frame whose tag the voice cannot read answers an in-band tag_invalid error and leaves the buffer as it was. Billing is cumulative per connection: the total charged equals what one POST /v1/tts call carrying the text of every charged leg would cost.

at least 1 character

flushSynthesizes the buffered text now. A flush with nothing to speak in the buffer (empty, or tags alone) answers flushed immediately.

type"flush"Required

cancelStops the leg being spoken and discards the buffered text. Use it when the speaker interrupts. The leg in flight was charged before its first byte and stays charged. The discarded buffer was never charged. A session that sends one sentence per frame pays only for the sentence it interrupted. The server answers cancelled, and the session stays open for more text.

type"cancel"Required

endFlushes any remaining text, then the server sends done and closes with code 1000.

type"end"Required

Server events#

Audio frames: Raw audio bytes in the session's format for the current leg, in delivery order between that leg's charged and flushed events.

startedThe session is configured and ready for text.

type"started"Required
connectionstringRequired

Connection id; quote it in support requests (the x-request-id analog).

modelstringRequired

The model serving this session.

voicestringRequired

Canonical voice id the session resolved to.

format"mp3" or "opus" or "wav" or "pcm" or "mulaw" or "alaw"Required

The audio format every binary frame of the session carries.

sample_rateintegerRequired

The sample rate of the session's audio in Hz, which is the rate the start frame asked for or else the format's own. Check it before playing raw pcm.

timestamps"word" or "sentence" or "utterance"

Echoed when the session requested timing.

speednumber

Echoed when the session asked for a rate of its own.

language"th" or "en" or "zh"

Echoed when the session pinned a language.

temperaturenumber

Echoed when the session asked for a temperature of its own.

chargedBilling receipt, sent before the leg's audio. Receipts replace the x-credits-charged header.

type"charged"Required
seqintegerRequired

The synthesis leg this receipt covers, starting at 1.

charsintegerRequired

Characters synthesized in this leg.

creditsnumberRequired

Credits charged for this leg, exact to a hundredth. The 0.1-credit request floor applies once per connection, on the first charged leg.

total_creditsnumberRequired

Total credits charged on this connection so far.

timingTiming spans; sent only when the session set timestamps.

type"timing"Required
seqintegerRequired

The leg these spans belong to.

spansarray of objectsRequired

Times are relative to the start of this leg's audio (each leg is an independent audio stream).

Show properties
spans[].textstringRequired

The spoken input text this span covers. An emotion tag takes no span.

spans[].startnumberRequired

Where the span begins in the audio, in seconds.

spans[].endnumberRequired

Where the span ends in the audio, in seconds.

flushedThe buffer is drained; safe to end or keep sending.

type"flushed"Required
seqintegerRequired

All audio and timing through leg seq have been sent and the buffer is empty. 0 when no leg has run yet.

cancelledA cancel frame was applied. The stopped leg keeps the charge its receipt reported; the discarded buffer was never charged. The session stays open, and the next text frame starts a new leg.

type"cancelled"Required
seqintegerRequired

The leg that stopped. No further audio or timing arrives for it, and any already received is safe to discard. 0 when no leg was in flight and only the buffer was discarded.

errorSomething failed. Recoverable errors leave the connection open; after a fatal one the server closes with the code listed below.

type"error"Required
codestringRequired

Stable machine-readable code from the error catalog. Branch on this field; the errors reference documents every code.

seqinteger

The leg the error belongs to, when it belongs to one.

retry_afterinteger

Seconds to wait before re-flushing; sent with rate_limited.

doneEvery leg is delivered; the server closes with code 1000 after this event. A connection that closes without done did not finish delivering.

type"done"Required
total_charsintegerRequired

Characters synthesized on this connection.

total_creditsnumberRequired

Credits charged on this connection, exact to a hundredth.

Close codes#

CodeDescription
1000Normal close after done, or client-initiated.
1001The server is restarting. Reconnect and resume.
1009A frame exceeded 64 KiB.
4400A frame violated the session protocol, or the start frame named an unknown model or voice, or a format or sample rate the model does not serve. A frame that fails validation answers an error event, and the connection stays open.
4401The API key stopped being valid mid-session.
4402Insufficient credits for the next leg.
4403The key's spending cap was reached.
4408No frame arrived and no audio was delivered for 300 seconds.
4413The client read too slowly and the outbound buffer overflowed.
4429Too many concurrent live connections for the account. Close an existing connection and reconnect.
4500Internal server error.
4503The start frame's model is not served right now. Nothing was charged; retry once GET /health reports it served.

Errors#

400validationThe request body, query, or headers failed validation against the endpoint schema, or broke a rule the endpoint's reference states. A field the schema refused adds two fields to the problem body. path names the first failing field by location and name in dot form, such as body.voice or headers.idempotency-key. A body that could not be read gives the path body alone. reason is one stable code from the validation reasons list.
400live_protocolA frame on a live WebSocket violated the session protocol. The frame arrived before start, repeated start, or followed end.
400unknown_modelThe model field does not name a served model.
400unsupported_formatThe model does not serve this format, or this format at the requested sample rate.
400unknown_voiceThe voice id names no voice on the model's roster and no designed voice this account holds.
400text_too_longThe text exceeds the model's per-request character ceiling.
400tag_invalidThe text holds a tag the model refuses. On Paxa TTS Flash v2, the bracket asks for what the model does not do, such as an accent, singing, or a sound outside the supported list. It can also hold no letters or carry a weight that is not a number. On a Paxa TTS Flash voice that takes emotion tags, the bracketed word is outside the tag list, or its weight is not a number. Nothing was charged.
401unauthorizedThe request carried no API key, or the key is invalid or disabled.
402insufficient_creditsThe account does not have enough credits for this request. Nothing was charged.
403key_limitThis API key reached its spending cap. Nothing was charged.
422unspeakable_textThe text contains nothing the model can voice: only tags, emoji, punctuation, or a script it does not speak. A text of tags alone is refused before the charge; any other is charged and refunded.
429rate_limitedRequests per minute for the plan are exhausted. One window covers the whole account, across every product and every key.
429concurrency_limitedThe account holds the plan's full count of concurrent requests for this product. Nothing was charged. Each product is limited separately, and an open live connection holds one slot of its product for its lifetime.
500internalRequest state was inconsistent on the server.
502provider_errorModel inference failed after the request was charged.
503provider_unavailableThe model behind this endpoint is not available right now. Nothing was charged.
PreviousSpeech (OpenAI-compatible)NextDesign a voice