# Live speech

`WS wss://api.paxalabs.com/v1/tts/live`

WS /v1/tts/live holds one WebSocket for many syntheses: text frames in, audio frames out. Billed cumulatively per character, one minimum per connection.

Connect once, send `start`, then interleave `text`, `flush`, `cancel`, and `end`. An open connection holds one of the plan's [concurrent speech slots](https://paxalabs.com/docs/rate-limits#concurrency) until it closes. The [Live streaming guide](https://paxalabs.com/docs/live-streaming) covers the lifecycle and billing.

## Headers

- `authorization` (Bearer pxa_..., required): Authorization: Bearer pxa_... (recommended).
- `x-api-key` (pxa_..., optional): Alternative to the Authorization header. Ignored when Authorization is present.

See authentication at https://paxalabs.com/docs/authentication.

## Connect

`wss://api.paxalabs.com/v1/tts/live`. Binary frames carry audio in the session's format. JSON text frames carry control, billing, and timing events.

## Client messages

**start**: Configures the session. Must be the first frame; exactly one per connection.

- `type` ("start", required)
- `model` (string, required): TTS model id, for example paxa-tts-flash-v2.
- `voice` (string, required): Voice id, resolved as on POST /v1/tts. A designed voice serves the connection until it closes, even if it is deleted meanwhile.
- `format` ("mp3" or "opus" or "wav" or "pcm" or "mulaw" or "alaw", optional, default "mp3"): Audio format for the session's binary frames, as format on POST /v1/tts.
- `sample_rate` (8000 or 16000 or 22050 or 24000 or 32000 or 44100 or 48000, optional, example 16000): Sample rate of pcm and wav, in Hz. Paxa TTS Flash v2 serves 8000, 16000, 22050, 24000, 32000, 44100, 48000, and Paxa TTS Flash serves 24000 only. For any other format the value may only restate that format's own rate, which is 8000 for mulaw and alaw. Fixed for the connection, like format. A rate the model doesn't serve answers unsupported_format. A request costs the same at every rate.
- `timestamps` ("word" or "sentence" or "utterance", optional): Adds timing events to the session at 1.25x the standard rate, applied to the connection's cumulative character count.
- `speed` (number, optional, default 1, min 0.5, max 1.5, example 1.1): Speaking rate, as a multiplier on the voice's own design speed. It runs from 0.5 to 1.5, and 1 is the voice as designed. Rate changes how the audio sounds. It does not change how many characters were sent. A request costs the same at every rate. Values round to the nearest 0.05. Fixed for the connection, like model and voice.
- `language` ("auto" or "th" or "en" or "zh", optional, default "auto", example "en"): Sets how codes, digits, and symbols are read. Words keep the language of their script, and a number inside Thai or Chinese text keeps that text's language. Elsewhere, "th" reads them in Thai (AB2039 with Thai letter names and digits) and "en" in English. "zh" reads codes in English, a number next to Chinese characters in Mandarin, and other digits and symbols in Thai. "auto", the default, takes the language most of the text is written in, counting English only in words of three or more letters, and Thai on a tie. Under "auto", a bare number, AB2039, or 250 ml reads as Thai, and ISO9001 or 100 USD as English. Text with no Thai, Chinese, digits, or counted English word reads as English when a two-letter word has a lowercase letter, such as Hi or I'm OK. Pin "th" or "en" to choose how a short number or code reads. A request costs the same in every language. Fixed for the connection, like model and voice. "auto" resolves on each leg's own text, and a leg that is only a number reads as Thai.
- `temperature` (number, optional, default 0.5, min 0, max 2, example 0.5): Sets how much the pacing, pauses, breaths, and melody of a reading vary from one request to the next. It runs from 0 to 2, and 0.5 is the default. They vary least at 0 and most at 2. Paxa TTS Flash v2 reads it. Any other model accepts only a value that rounds to 0.5, and any other answers validation. A request costs the same at every temperature. Values round to the nearest 0.1. Fixed for the connection, like model and voice. A refused value answers an in-band validation error, and the connection stays open for a corrected start frame.

**text**: Appends text. The server auto-flushes on a newline, sentence-ending punctuation, or a buffered-length threshold, holding back a tag still open at the end of the buffer for up to that threshold.

- `type` ("text", required)
- `text` (string, required, at least 1 character): Text appended to the synthesis buffer. A frame longer than the model's max_chars answers an in-band text_too_long error. Every Paxa TTS Flash v2 voice reads tags, as does a Paxa TTS Flash (paxa-tts-flash-v1) voice whose emotion_tags list is not empty. On those voices, tags that no words follow yet wait in the buffer, and each leg opens with the tags the previous leg ended in. A frame that would take the waiting tags and the opening tags past max_chars also answers text_too_long, as does one that would leave opening tags longer than half of max_chars. A frame whose tag the voice cannot read answers an in-band tag_invalid error and leaves the buffer as it was. Billing is cumulative per connection: the total charged equals what one POST /v1/tts call carrying the text of every charged leg would cost.

**flush**: Synthesizes the buffered text now. A flush with nothing to speak in the buffer (empty, or tags alone) answers flushed immediately.

- `type` ("flush", required)

**cancel**: Stops the leg being spoken and discards the buffered text. Use it when the speaker interrupts. The leg in flight was charged before its first byte and stays charged. The discarded buffer was never charged. A session that sends one sentence per frame pays only for the sentence it interrupted. The server answers cancelled, and the session stays open for more text.

- `type` ("cancel", required)

**end**: Flushes any remaining text, then the server sends done and closes with code 1000.

- `type` ("end", required)

## Server events

Audio frames: Raw audio bytes in the session's format for the current leg, in delivery order between that leg's charged and flushed events.

**started**: The session is configured and ready for text.

- `type` ("started", required)
- `connection` (string, required): Connection id; quote it in support requests (the x-request-id analog).
- `model` (string, required): The model serving this session.
- `voice` (string, required): Canonical voice id the session resolved to.
- `format` ("mp3" or "opus" or "wav" or "pcm" or "mulaw" or "alaw", required): The audio format every binary frame of the session carries.
- `sample_rate` (integer, required): The sample rate of the session's audio in Hz, which is the rate the start frame asked for or else the format's own. Check it before playing raw pcm.
- `timestamps` ("word" or "sentence" or "utterance", optional): Echoed when the session requested timing.
- `speed` (number, optional): Echoed when the session asked for a rate of its own.
- `language` ("th" or "en" or "zh", optional): Echoed when the session pinned a language.
- `temperature` (number, optional): Echoed when the session asked for a temperature of its own.

**charged**: Billing receipt, sent before the leg's audio. Receipts replace the x-credits-charged header.

- `type` ("charged", required)
- `seq` (integer, required): The synthesis leg this receipt covers, starting at 1.
- `chars` (integer, required): Characters synthesized in this leg.
- `credits` (number, required): Credits charged for this leg, exact to a hundredth. The 0.1-credit request floor applies once per connection, on the first charged leg.
- `total_credits` (number, required): Total credits charged on this connection so far.

**timing**: Timing spans; sent only when the session set timestamps.

- `type` ("timing", required)
- `seq` (integer, required): The leg these spans belong to.
- `spans` (array of objects, required): Times are relative to the start of this leg's audio (each leg is an independent audio stream).
  - `spans[].text` (string, required): The spoken input text this span covers. An emotion tag takes no span.
  - `spans[].start` (number, required): Where the span begins in the audio, in seconds.
  - `spans[].end` (number, required): Where the span ends in the audio, in seconds.

**flushed**: The buffer is drained; safe to end or keep sending.

- `type` ("flushed", required)
- `seq` (integer, required): All audio and timing through leg seq have been sent and the buffer is empty. 0 when no leg has run yet.

**cancelled**: A cancel frame was applied. The stopped leg keeps the charge its receipt reported; the discarded buffer was never charged. The session stays open, and the next text frame starts a new leg.

- `type` ("cancelled", required)
- `seq` (integer, required): The leg that stopped. No further audio or timing arrives for it, and any already received is safe to discard. 0 when no leg was in flight and only the buffer was discarded.

**error**: Something failed. Recoverable errors leave the connection open; after a fatal one the server closes with the code listed below.

- `type` ("error", required)
- `code` (string, required): Stable machine-readable code from the error catalog. Branch on this field; the errors reference documents every code.
- `seq` (integer, optional): The leg the error belongs to, when it belongs to one.
- `retry_after` (integer, optional): Seconds to wait before re-flushing; sent with rate_limited.

**done**: Every leg is delivered; the server closes with code 1000 after this event. A connection that closes without done did not finish delivering.

- `type` ("done", required)
- `total_chars` (integer, required): Characters synthesized on this connection.
- `total_credits` (number, required): Credits charged on this connection, exact to a hundredth.

## Close codes

| Code | Meaning |
| --- | --- |
| 1000 | Normal close after done, or client-initiated. |
| 1001 | The server is restarting. Reconnect and resume. |
| 1009 | A frame exceeded 64 KiB. |
| 4400 | A frame violated the session protocol, or the start frame named an unknown model or voice, or a format or sample rate the model does not serve. A frame that fails validation answers an error event, and the connection stays open. |
| 4401 | The API key stopped being valid mid-session. |
| 4402 | Insufficient credits for the next leg. |
| 4403 | The key's spending cap was reached. |
| 4408 | No frame arrived and no audio was delivered for 300 seconds. |
| 4413 | The client read too slowly and the outbound buffer overflowed. |
| 4429 | Too many concurrent live connections for the account. Close an existing connection and reconnect. |
| 4500 | Internal server error. |
| 4503 | The start frame's model is not served right now. Nothing was charged; retry once GET /health reports it served. |

## Errors

- `validation` (400): The request body, query, or headers failed validation against the endpoint schema, or broke a rule the endpoint's reference states. A field the schema refused adds two fields to the problem body. path names the first failing field by location and name in dot form, such as body.voice or headers.idempotency-key. A body that could not be read gives the path body alone. reason is one stable code from the validation reasons list.
- `live_protocol` (400): A frame on a live WebSocket violated the session protocol. The frame arrived before start, repeated start, or followed end.
- `unknown_model` (400): The model field does not name a served model.
- `unsupported_format` (400): The model does not serve this format, or this format at the requested sample rate.
- `unknown_voice` (400): The voice id names no voice on the model's roster and no designed voice this account holds.
- `text_too_long` (400): The text exceeds the model's per-request character ceiling.
- `tag_invalid` (400): The text holds a tag the model refuses. On Paxa TTS Flash v2, the bracket asks for what the model does not do, such as an accent, singing, or a sound outside the supported list. It can also hold no letters or carry a weight that is not a number. On a Paxa TTS Flash voice that takes emotion tags, the bracketed word is outside the tag list, or its weight is not a number. Nothing was charged.
- `unauthorized` (401): The request carried no API key, or the key is invalid or disabled.
- `insufficient_credits` (402): The account does not have enough credits for this request. Nothing was charged.
- `key_limit` (403): This API key reached its spending cap. Nothing was charged.
- `unspeakable_text` (422): The text contains nothing the model can voice: only tags, emoji, punctuation, or a script it does not speak. A text of tags alone is refused before the charge; any other is charged and refunded.
- `rate_limited` (429): Requests per minute for the plan are exhausted. One window covers the whole account, across every product and every key.
- `concurrency_limited` (429): The account holds the plan's full count of concurrent requests for this product. Nothing was charged. Each product is limited separately, and an open live connection holds one slot of its product for its lifetime.
- `internal` (500): Request state was inconsistent on the server.
- `provider_error` (502): Model inference failed after the request was charged.
- `provider_unavailable` (503): The model behind this endpoint is not available right now. Nothing was charged.

## Example

TypeScript:

```typescript
// Runs on Bun as-is; on Node, `import WebSocket from "ws"` accepts
// the same constructor options. Browsers cannot send this header, and
// API keys never ship to browsers anyway.
const socket = new WebSocket("wss://api.paxalabs.com/v1/tts/live", {
  headers: { Authorization: `Bearer ${process.env.PAXA_API_KEY}` },
});
socket.binaryType = "arraybuffer";

socket.onopen = () => {
  socket.send(JSON.stringify({
    type: "start",
    model: "paxa-tts-flash-v2",
    voice: "khanomkrok",
    timestamps: "word",
  }));
  // Send text whenever it becomes available, token by token if you like.
  socket.send(JSON.stringify({ type: "text", text: "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs" }));
  socket.send(JSON.stringify({ type: "flush" }));
  socket.send(JSON.stringify({ type: "end" }));
};

socket.onmessage = (event) => {
  if (typeof event.data === "string") {
    const message = JSON.parse(event.data);
    if (message.type === "done") socket.close();
    return;
  }
  player.feed(new Uint8Array(event.data)); // binary frames are audio
};
```

Python:

```python
import json
import os

from websockets.sync.client import connect  # pip install websockets

with connect(
    "wss://api.paxalabs.com/v1/tts/live",
    additional_headers={"Authorization": f"Bearer {os.environ['PAXA_API_KEY']}"},
) as socket:
    socket.send(json.dumps({"type": "start", "model": "paxa-tts-flash-v2", "voice": "khanomkrok"}))
    socket.send(json.dumps({"type": "text", "text": "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"}))
    socket.send(json.dumps({"type": "flush"}))
    socket.send(json.dumps({"type": "end"}))

    with open("speech.mp3", "wb") as f:
        for frame in socket:
            if isinstance(frame, bytes):
                f.write(frame)  # binary frames are audio
                continue
            event = json.loads(frame)
            if event["type"] == "done":
                break
```

Server events:

```json
{"type":"started","connection":"7d9c2b7e-4a41-4c8e-9a35-1f26d8c04b7a","model":"paxa-tts-flash-v2","voice":"khanomkrok","format":"mp3","sample_rate":44100,"timestamps":"word"}
{"type":"charged","seq":1,"chars":36,"credits":0.45,"total_credits":0.45}
{"type":"timing","seq":1,"spans":[{"text":"สวัสดี","start":0,"end":0.51},{"text":"ครับ","start":0.51,"end":0.85}]}
{"type":"flushed","seq":1}
{"type":"done","total_chars":36,"total_credits":0.45}
```