# Two-way live speech over WebSocket

Live streaming is a two-way WebSocket for a whole conversation: send text as it becomes available and audio frames stream back, with billing receipts on the same connection.

[The live endpoint](https://paxalabs.com/docs/api/live) holds one WebSocket open for a whole conversation: send text whenever it becomes available and receive audio frames as soon as they are ready, in both directions at once. Use it when text arrives incrementally, for example from an LLM token stream or a dialogue turn under composition. When each request's text is already complete, [HTTP streaming](https://paxalabs.com/docs/streaming) delivers the same low time to first byte with no connection to manage.

|  | HTTP streaming | Live connection |
| --- | --- | --- |
| Transport | One `POST /v1/tts` per request | One WebSocket for many syntheses |
| Text | Complete per request | Incremental fragments |
| Billing | Per request, 0.1 credit minimum each | Cumulative per connection, one minimum |
| Timing | `timestamps` per request | Set once in `start`, timing events per leg |

## Open a session

1. Connect: Open the WebSocket to the live endpoint with the API key in the `Authorization` header, exactly as on HTTP requests. A refused upgrade answers ordinary problem+json.
2. start: The first frame configures the connection: `model` and `voice`, plus optional `format`, `sample_rate`, `timestamps`, `speed`, `language`, and [`temperature`](https://paxalabs.com/docs/temperature). One configuration per connection. Each leg is its own synthesis. Under the default `language`, `auto`, a leg that is only a number reads as Thai. Pin `th` or `en` to read every leg's numbers and codes the same way.
3. text: Send text fragments as they become available. The server accumulates them and synthesizes at a natural boundary: a newline, sentence-ending punctuation, or a length threshold. Thai text carries no terminal punctuation. A Thai client reaches the length threshold, or sends a flush frame at the end of a turn. On a voice that reads [emotion tags](https://paxalabs.com/docs/emotion-tags#live) or [direction tags](https://paxalabs.com/docs/directions#live), a tag still open at a boundary waits until it closes. One still open past the length threshold is read as text and is not checked as a tag. Each leg opens in the delivery the last one ended in.
4. flush: Force synthesis of everything buffered so far, for example at the end of a dialogue turn. Flushing an empty buffer is a safe no-op.
5. cancel: Stop the leg being spoken and discard the buffered text, for example when the speaker talks over the reply. The connection stays open for the next turn.
6. end: Signal that no more text is coming. The server finishes synthesizing, sends `done`, and closes the connection.

TypeScript:

```typescript
// Runs on Bun as-is; on Node, `import WebSocket from "ws"` accepts
// the same constructor options. Browsers cannot send this header, and
// API keys never ship to browsers anyway.
const socket = new WebSocket("wss://api.paxalabs.com/v1/tts/live", {
  headers: { Authorization: `Bearer ${process.env.PAXA_API_KEY}` },
});
socket.binaryType = "arraybuffer";

socket.onopen = () => {
  socket.send(JSON.stringify({
    type: "start",
    model: "paxa-tts-flash-v2",
    voice: "khanomkrok",
    timestamps: "word",
  }));
  // Send text whenever it becomes available, token by token if you like.
  socket.send(JSON.stringify({ type: "text", text: "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs" }));
  socket.send(JSON.stringify({ type: "flush" }));
  socket.send(JSON.stringify({ type: "end" }));
};

socket.onmessage = (event) => {
  if (typeof event.data === "string") {
    const message = JSON.parse(event.data);
    if (message.type === "done") socket.close();
    return;
  }
  player.feed(new Uint8Array(event.data)); // binary frames are audio
};
```

Python:

```python
import json
import os

from websockets.sync.client import connect  # pip install websockets

with connect(
    "wss://api.paxalabs.com/v1/tts/live",
    additional_headers={"Authorization": f"Bearer {os.environ['PAXA_API_KEY']}"},
) as socket:
    socket.send(json.dumps({"type": "start", "model": "paxa-tts-flash-v2", "voice": "khanomkrok"}))
    socket.send(json.dumps({"type": "text", "text": "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"}))
    socket.send(json.dumps({"type": "flush"}))
    socket.send(json.dumps({"type": "end"}))

    with open("speech.mp3", "wb") as f:
        for frame in socket:
            if isinstance(frame, bytes):
                f.write(frame)  # binary frames are audio
                continue
            event = json.loads(frame)
            if event["type"] == "done":
                break
```

## Server events

Binary frames carry audio in the format chosen at `start`. Every other frame is a JSON event. A `charged` event is a billing receipt. A `timing` event carries spans when [timestamps](https://paxalabs.com/docs/timestamps) are on. An `error` event carries the same stable codes as [Errors](https://paxalabs.com/docs/errors). A `flushed` event acknowledges a drained buffer, and a `cancelled` event acknowledges an interruption. A `done` event closes the exchange. The [API reference](https://paxalabs.com/docs/api/live) documents every message shape, and the [text to speech page](https://paxalabs.com/text-to-speech) traces a live session frame by frame.

Server events:

```json
{"type":"started","connection":"7d9c2b7e-4a41-4c8e-9a35-1f26d8c04b7a","model":"paxa-tts-flash-v2","voice":"khanomkrok","format":"mp3","sample_rate":44100,"timestamps":"word"}
{"type":"charged","seq":1,"chars":36,"credits":0.45,"total_credits":0.45}
{"type":"timing","seq":1,"spans":[{"text":"สวัสดี","start":0,"end":0.51},{"text":"ครับ","start":0.51,"end":0.85}]}
{"type":"flushed","seq":1}
{"type":"done","total_chars":36,"total_credits":0.45}
```

## Interrupt the speaker

A `cancel` frame stops the leg being spoken and discards whatever text is still buffered. Audio for that leg ends, a `cancelled` event names the leg that stopped, and the connection stays open. The next `text` frame starts a new leg on the same session. This is how an assistant handles a speaker talking over it. Closing the connection would work too, but it gives up the concurrency slot and costs a full upgrade to get back.

An interrupted leg keeps the charge its `charged` receipt reported, because that text had already been synthesized when the interruption arrived. The buffered text discarded alongside it was never charged at all. Sending one sentence per `text` frame therefore bounds what an interruption costs to the sentence being spoken, which is the same shape that keeps time to first audio low.

Barge-in:

```typescript
// Stop the reply the moment the speaker starts. The sentence being
// spoken ends where it is, whatever was queued behind it is dropped,
// and the same socket carries the next turn.
speech.onSpeechStart = () => {
  socket.send(JSON.stringify({ type: "cancel" }));
  player.discard(); // drop the audio already buffered on the client
};

socket.onmessage = (event) => {
  if (typeof event.data !== "string") {
    player.feed(new Uint8Array(event.data));
    return;
  }
  const message = JSON.parse(event.data);
  if (message.type === "cancelled") {
    // No more audio arrives for message.seq. That leg keeps the charge
    // its receipt reported; the discarded text was never charged.
    return;
  }
  if (message.type === "done") socket.close();
};
```

## Billing

A connection is billed cumulatively at the same rate as every other request. Speech costs 10 credits per 1,000 characters, or 1.25 times that when timestamps are on, billed per character and rounded up to the next hundredth of a credit. The 0.1 credit minimum applies once per connection, however many messages it carries. Short fragments sent on one connection therefore pay the minimum once:

| Delivery | Billed characters | Credits |
| --- | --- | --- |
| One live connection | 15 | 0.15 |
| 3 separate HTTP requests | 4, 5, 6 | 0.3 |

Each `charged` event is a receipt for one increment. The connection's total is the sum of its receipts, and [GET /v1/me](https://paxalabs.com/docs/api/me) reflects it live. A WebSocket carries no `x-credits-charged` header, and the receipts report the charge in its place.

## Handle errors and close

Failures before the connection upgrades are ordinary HTTP `application/problem+json` responses. After the upgrade, failures arrive as `error` events whose `code` matches the [error catalog](https://paxalabs.com/docs/errors). Recoverable failures leave the connection open for a retry: [rate_limited](https://paxalabs.com/docs/errors#rate_limited), a failed leg reporting [provider_error](https://paxalabs.com/docs/errors#provider_error), which is refunded automatically, and [tag_invalid](https://paxalabs.com/docs/errors#tag_invalid) for a tag the voice cannot read. After `tag_invalid` the buffer keeps what it held before the `text` frame. Credit exhaustion and protocol violations such as [live_protocol](https://paxalabs.com/docs/errors#live_protocol) close the connection with the code listed in the [reference](https://paxalabs.com/docs/api/live). A connection that closes without a `done` event did not finish delivering. Treat the tail as missing.

> Each synthesis leg counts as one request against the plan's [rate limit](https://paxalabs.com/docs/rate-limits), exactly like a `POST /v1/tts` call. A `rate_limited` error keeps the buffer and the connection and pauses auto-flush. Send `flush` after `retry_after` seconds, or send `cancel` to drop the buffered text and start a new turn, which auto-flushes again. The connection itself holds one of the plan's [concurrent speech slots](https://paxalabs.com/docs/rate-limits#concurrency) for as long as it stays open. An upgrade past the limit is refused with `429 concurrency_limited`. Close connections you no longer need.