# Realtime transcription

`WS wss://api.paxalabs.com/v1/stt/live`

WS /v1/stt/live transcribes speech as it is spoken and detects the end of each turn on its own. Built for voice agents. Send the microphone and receive one final transcript per turn. Charged in credits by audio received.

Connect once, send `start`, then stream raw audio as binary frames. The connection reports `speech_started` when the user begins talking. It reports `speech_ended` and `transcript` once the turn is over. An agent interrupts its own reply on the first event and answers on the last. An open connection holds one of the plan's [concurrent transcription slots](https://paxalabs.com/docs/rate-limits#concurrency) until it closes. The [turn detection section](https://paxalabs.com/docs/speech-to-text#realtime) of the transcription guide covers the events and billing.

## Headers

- `authorization` (Bearer pxa_..., required): Authorization: Bearer pxa_... (recommended).
- `x-api-key` (pxa_..., optional): Alternative to the Authorization header. Ignored when Authorization is present.

See authentication at https://paxalabs.com/docs/authentication.

## Connect

`wss://api.paxalabs.com/v1/stt/live`. Binary frames carry audio in. JSON text frames carry control in, and turn, transcript, and billing events out.

## Client messages

**start**: Configures the session. Must be the first frame; exactly one per connection.

- `type` ("start", required)
- `model` (string, required, example "paxa-stt-lite-realtime-v1-preview"): Realtime transcription model id, for example paxa-stt-lite-realtime-v1-preview. GET /v1/models lists the served catalog; a recording model answers unknown_model here.
- `audio` (object, required): What the binary frames carry, fixed for the connection. The bytes are read exactly as declared. A wrong declaration transcribes noise.
  - `audio.encoding` ("pcm_s16le" or "mulaw", required): "pcm_s16le" is signed 16-bit little-endian PCM, "mulaw" is G.711 mu-law as telephony delivers it. Mono.
  - `audio.sample_rate` (16000 or 48000 or 8000, required): Samples per second of the binary frames: 16000 or 48000 for pcm_s16le, 8000 for mulaw. Any other pairing answers validation.
- `language` (string, optional, up to 16 characters, pattern ^[A-Za-z]{2,3}(-[A-Za-z0-9]{2,8})*$, example "th"): The language the speech is expected to be in, as a BCP 47 tag such as "th" or "en". The hint filters nothing. Speech in any other language is still transcribed as spoken, and Thai that switches into English mid-sentence needs no setting. Omitted, the model detects the language itself. Set the tag when detection has picked the wrong language.
- `timestamps` ("word", optional): Adds a words array to every final transcript, one span per word with its start and end in seconds on the connection's clock. Thai words are segmented for you. Costs nothing extra.
- `style` ("verbatim" or "clean", optional, default "verbatim"): How faithfully the transcript follows the speech. "verbatim", the default, writes exactly what was said: fillers, false starts, and self-corrections included. "clean" drops fillers and merges broken phrases into one, for a transcript that is easier to read. It is a readability choice. A clean transcript departs further from what was said, and our own measurement found it less exact. Any other value answers 400 validation.
- `convention` ("spoken" or "written", optional, default "spoken"): How the transcript writes what was said. "spoken", the default, writes numbers as they were said (สามสิบเปอร์เซ็นต์) and a word said twice as two words, which is what a verbatim transcript, a subtitle, or a language model reading the text needs. "written" keeps display conventions: digits and units (30%), the repetition mark ๆ, for dictated text that will be read on a page. The word timestamps are the same under both.
- `vocabulary` (array of strings, optional, example ["Paxa Labs","กรุงเทพฯ","quarterly review"]): Terms the speech is likely to contain that the model should recognize as written: product names, people, places, jargon, in Thai or any other language. Up to 50 terms of 50 characters each, and a longer list or term answers 400 validation. The terms bias recognition and are never inserted; a term that was not said does not appear. Costs nothing extra and combines with every other option.

**playback**: Tells the turn detector when your agent is speaking. Its own voice reaching the microphone then cannot open a turn. While speaking is true, opening a turn takes more sustained speech from the user; a real interruption still opens one and arrives as speech_started. Send it on every change. A session that never sends it treats the room as quiet.

- `type` ("playback", required)
- `speaking` (boolean, required): true while your agent's audio is playing into the same room as the microphone, false once it stops.

**end**: Closes the turn in progress, if any, and delivers its transcript, then the server sends done and closes with code 1000. Audio after end is ignored.

- `type` ("end", required)

## Server events

Audio frames: Audio bytes in the encoding and sample rate the start frame declared, in any chunking and at any cadence, up to 64 KiB a frame. Sending faster than real time is fine.

**started**: The session is configured and listening.

- `type` ("started", required)
- `connection` (string, required): Connection id; quote it in support requests (the x-request-id analog).
- `model` (string, required): The model serving this session.

**speech_started**: The user started speaking. This is the interruption signal. Stop your agent's audio and cancel the reply it was generating. It arrives once enough speech has been heard to be sure. The time it reports is therefore earlier than the moment it arrives.

- `type` ("speech_started", required)
- `turn` (integer, required, min 1): The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.
- `time` (number, required): Where the user started speaking. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

**transcript**: A transcript of one turn. The final transcript is the turn: send its text to your language model. An interim gives an earlier look at the same turn and never needs handling on its own.

- `type` ("transcript", required)
- `turn` (integer, required, min 1): The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.
- `is_final` (boolean, required): true for the turn's finished transcript, sent exactly once per turn. false for an interim transcript of the turn so far, sent at a natural pause and replaced by the final.
- `text` (string, required): The verbatim transcript of the turn, or of the turn so far. Empty on a final when the turn held no speech the model could read.
- `start` (number, required): Where the transcribed audio starts. In seconds of audio received since the connection's first binary frame, exact to a thousandth.
- `end` (number, required): Where the transcribed audio ends. In seconds of audio received since the connection's first binary frame, exact to a thousandth.
- `words` (array of objects, optional): Word spans on the connection's clock, present on a final when the session set timestamps.
  - `words[].text` (string, required): The spoken input text this span covers. An emotion tag takes no span.
  - `words[].start` (number, required): Where the span begins in the audio, in seconds.
  - `words[].end` (number, required): Where the span ends in the audio, in seconds.

**speech_ended**: The user stopped speaking and the turn is closed. Its final transcript follows. Use it to show that the agent is thinking.

- `type` ("speech_ended", required)
- `turn` (integer, required, min 1): The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.
- `time` (number, required): Where the turn ended. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

**charged**: Billing receipt. The connection is charged for the audio it receives, silence included. A period closes at the end of each turn, after a minute of audio with no turn, and at the end of the session. Receipts replace the x-credits-charged header.

- `type` ("charged", required)
- `seq` (integer, required): The billing period this receipt covers, starting at 1.
- `seconds` (number, required): Audio received in this period, in seconds, exact to a thousandth.
- `credits` (number, required): Credits charged for this period, exact to a hundredth. The 0.1-credit request floor applies once per connection, on the first receipt.
- `total_credits` (number, required): Credits charged on this connection so far, the audio of failed turns excluded, exact to a hundredth.

**error**: Something failed. An error with a turn leaves the connection open; after a fatal one the server closes with the code listed below.

- `type` ("error", required)
- `code` (string, required): Stable machine-readable code from the error catalog. Branch on this field; the errors reference documents every code.
- `turn` (integer, optional): The turn the error belongs to, when it belongs to one. That turn delivers no transcript; the session keeps listening.
- `not_billed` (number, optional): Credits this turn's audio is held back from the bill, exact to a hundredth. It comes off what the periods after this turn owe. A turn that fails with little audio behind it returns less than this. Sent with a turn.

**done**: Every turn is delivered; the server closes with code 1000 after this event. A connection that closes without done did not finish delivering.

- `type` ("done", required)
- `total_seconds` (number, required): Audio received on this connection, in seconds, exact to a thousandth.
- `turns` (integer, required): Turns the connection delivered or failed.
- `total_credits` (number, required): Credits charged on this connection, the audio of failed turns excluded, exact to a hundredth. Zero when no turn was delivered, which returns the connection's charge in full.

## Close codes

| Code | Meaning |
| --- | --- |
| 1000 | Normal close after done, or client-initiated. |
| 1001 | The server is restarting. Reconnect and resume. |
| 1009 | A binary frame exceeded 64 KiB. |
| 4400 | A frame violated the session protocol, or the start frame named an unknown model or a sample rate its encoding does not take. Another frame that fails validation answers an error event, and the connection stays open. |
| 4401 | The API key stopped being valid mid-session. |
| 4402 | The account cannot cover the request floor at start, or the next billing period. |
| 4403 | The key's spending cap was reached. |
| 4408 | No frame arrived and no event was sent for 300 seconds. |
| 4413 | The client read too slowly and the outbound buffer overflowed. |
| 4429 | Too many concurrent live connections for the account. Close an existing connection and reconnect. |
| 4500 | Internal server error. |
| 4502 | The transcription backend failed for the whole connection. What the delivered periods charged stays charged; a connection that delivered no turn is refunded in full. Reconnect. |
| 4503 | The transcription backend could not be reached at start. Nothing was charged; retry. |

## Errors

- `validation` (400): The request body, query, or headers failed validation against the endpoint schema, or broke a rule the endpoint's reference states. A field the schema refused adds two fields to the problem body. path names the first failing field by location and name in dot form, such as body.voice or headers.idempotency-key. A body that could not be read gives the path body alone. reason is one stable code from the validation reasons list.
- `live_protocol` (400): A frame on a live WebSocket violated the session protocol. The frame arrived before start, repeated start, or followed end.
- `unknown_model` (400): The model field does not name a served model.
- `unauthorized` (401): The request carried no API key, or the key is invalid or disabled.
- `insufficient_credits` (402): The account does not have enough credits for this request. Nothing was charged.
- `key_limit` (403): This API key reached its spending cap. Nothing was charged.
- `concurrency_limited` (429): The account holds the plan's full count of concurrent requests for this product. Nothing was charged. Each product is limited separately, and an open live connection holds one slot of its product for its lifetime.
- `internal` (500): Request state was inconsistent on the server.
- `provider_error` (502): Model inference failed after the request was charged.
- `provider_unavailable` (503): The model behind this endpoint is not available right now. Nothing was charged.

## Example

TypeScript:

```typescript
import { open } from "node:fs/promises";

// Runs on Bun as-is; on Node, `import WebSocket from "ws"` accepts
// the same constructor options. Browsers cannot send this header, and
// API keys never ship to browsers anyway: relay the audio through your
// server.
const socket = new WebSocket("wss://api.paxalabs.com/v1/stt/live", {
  headers: { Authorization: `Bearer ${process.env.PAXA_API_KEY}` },
});

socket.onopen = async () => {
  socket.send(JSON.stringify({
    type: "start",
    model: "paxa-stt-lite-realtime-v1-preview",
    audio: { encoding: "pcm_s16le", sample_rate: 16000 },
    language: "th",
  }));
  // 16 kHz signed 16-bit mono is 32,000 bytes a second. A microphone
  // delivers frames at this pace on its own; a file has to be paced.
  const file = await open("caller.pcm");
  const frame = new Uint8Array(640); // 20 ms
  for (;;) {
    const { bytesRead } = await file.read(frame, 0, frame.length);
    if (bytesRead === 0) break;
    socket.send(frame.subarray(0, bytesRead));
    await new Promise((resolve) => setTimeout(resolve, 20));
  }
  socket.send(JSON.stringify({ type: "end" }));
};

socket.onmessage = (event) => {
  const message = JSON.parse(String(event.data));
  switch (message.type) {
    case "speech_started":
      // The user is talking: stop the agent's audio, cancel its reply.
      break;
    case "transcript":
      if (message.is_final) console.log(`turn ${message.turn}: ${message.text}`);
      break;
    case "done":
      socket.close();
  }
};
```

Python:

```python
import json
import os
import time

from websockets.sync.client import connect  # pip install websockets

with connect(
    "wss://api.paxalabs.com/v1/stt/live",
    additional_headers={"Authorization": f"Bearer {os.environ['PAXA_API_KEY']}"},
) as socket:
    socket.send(json.dumps({
        "type": "start",
        "model": "paxa-stt-lite-realtime-v1-preview",
        "audio": {"encoding": "pcm_s16le", "sample_rate": 16000},
        "language": "th",
    }))
    # 16 kHz signed 16-bit mono is 32,000 bytes a second; a file has to
    # be paced the way a microphone paces itself.
    with open("caller.pcm", "rb") as audio:
        while frame := audio.read(640):  # 20 ms
            socket.send(frame)
            time.sleep(0.02)
    socket.send(json.dumps({"type": "end"}))

    for message in socket:
        event = json.loads(message)
        if event["type"] == "transcript" and event["is_final"]:
            print(event["turn"], event["text"])
        elif event["type"] == "done":
            break
```

Server events:

```json
{"type":"started","connection":"7d9c2b7e-4a41-4c8e-9a35-1f26d8c04b7a","model":"paxa-stt-lite-realtime-v1-preview"}
{"type":"speech_started","turn":1,"time":0.3}
{"type":"speech_ended","turn":1,"time":12.252}
{"type":"charged","seq":1,"seconds":12.252,"credits":2.56,"total_credits":2.56}
{"type":"transcript","turn":1,"is_final":true,"text":"สวัสดีค่ะคุณสมชายยอดชำระของท่านจำนวน พันสองร้อยห้าสิบ บาทครบกำหนดวันที่ สิบห้า สิงหาคมนี้ชำระผ่านแอปได้ตลอด ยี่สิบสี่ ชั่วโมงค่ะ","start":0.3,"end":12.252}
{"type":"done","total_seconds":12.252,"turns":1,"total_credits":2.56}
```