Paxa Labs

ถอดเสียงเรียลไทม์แบบ Multiplex

WSwss://api.paxalabs.com/v1/stt/multi

WS /v1/stt/multi รองรับการถอดเสียงเรียลไทม์หลายเซสชันบน WebSocket เดียว แต่ละเซสชันคือ context ที่มี id ของตัวเอง คิดค่าบริการและนับลิมิตเหมือนการเชื่อมต่อ /v1/stt/live หนึ่งการเชื่อมต่อ

ดูเป็น Markdown
import { open } from "node:fs/promises";

// Runs on Bun as-is; on Node, `import WebSocket from "ws"` accepts
// the same constructor options.
const socket = new WebSocket("wss://api.paxalabs.com/v1/stt/multi", {
  headers: { Authorization: `Bearer ${process.env.PAXA_API_KEY}` },
});

const send = (frame: object) => socket.send(JSON.stringify(frame));

const ascii = new TextEncoder();
// One byte counts the id, then the id in ASCII, then the audio.
function audioFrame(contextId: string, audio: Uint8Array) {
  const id = ascii.encode(contextId);
  const frame = new Uint8Array(1 + id.length + audio.length);
  frame[0] = id.length;
  frame.set(id, 1);
  frame.set(audio, 1 + id.length);
  return frame;
}

// One context per call, named by your own call id. Audio may follow
// start at once.
async function listen(contextId: string) {
  send({
    type: "start",
    context_id: contextId,
    model: "paxa-stt-lite-realtime-v1-preview",
    audio: { encoding: "pcm_s16le", sample_rate: 16000 },
  });
  // 16 kHz signed 16-bit mono is 32,000 bytes a second. A call delivers
  // frames at this pace on its own; a file has to be paced.
  const file = await open(`${contextId}.pcm`);
  const chunk = new Uint8Array(640); // 20 ms
  for (;;) {
    const { bytesRead } = await file.read(chunk, 0, chunk.length);
    if (bytesRead === 0) break;
    socket.send(audioFrame(contextId, chunk.subarray(0, bytesRead)));
    await new Promise((resolve) => setTimeout(resolve, 20));
  }
  send({ type: "end", context_id: contextId });
}

const calls = new Set(["call-1042", "call-1043"]);
socket.onopen = () => {
  for (const contextId of calls) void listen(contextId);
};

socket.onmessage = (event) => {
  const message = JSON.parse(String(event.data));
  switch (message.type) {
    case "transcript":
      if (message.is_final) console.log(`${message.context_id}: ${message.text}`);
      break;
    case "draining":
      // The server is restarting: start new contexts on another socket
      // and move each call there at its next turn boundary.
      break;
    case "closed":
      // The context's last event. Its id is free for a new start.
      calls.delete(message.context_id);
      if (calls.size === 0) socket.close();
  }
};
{"type":"started","context_id":"call-1042","connection":"7d9c2b7e-4a41-4c8e-9a35-1f26d8c04b7a","model":"paxa-stt-lite-realtime-v1-preview"}
{"type":"speech_started","context_id":"call-1042","turn":1,"time":0.3}
{"type":"speech_ended","context_id":"call-1042","turn":1,"time":12.252}
{"type":"charged","context_id":"call-1042","seq":1,"seconds":12.252,"credits":2.56,"total_credits":2.56}
{"type":"transcript","context_id":"call-1042","turn":1,"is_final":true,"text":"สวัสดีค่ะคุณสมชายยอดชำระของท่านจำนวน พันสองร้อยห้าสิบ บาทครบกำหนดวันที่ สิบห้า สิงหาคมนี้ชำระผ่านแอปได้ตลอด ยี่สิบสี่ ชั่วโมงค่ะ","start":0.3,"end":12.252}
{"type":"done","context_id":"call-1042","total_seconds":12.252,"turns":1,"total_credits":2.56}
{"type":"closed","context_id":"call-1042","code":1000,"reason":"session complete"}

เชื่อมต่อครั้งเดียว แล้วเปิด context ให้แต่ละสายด้วยเฟรม start ที่ระบุ context_id ซึ่งคุณกำหนดเอง จากนั้นสตรีมเสียงของสายนั้นเป็นเฟรมไบนารี โดยทุกเฟรมขึ้นต้นด้วยเฮดเดอร์ระบุ context แต่ละ context รับเฟรมชุดเดียวกับ WS /v1/stt/live และส่งอีเวนต์ชุดเดียวกัน โดยทุกอีเวนต์ระบุ context ของตัวเอง แต่ละ context จบด้วยอีเวนต์ closed เว้นแต่ WebSocket จะปิดก่อน โดย context อื่นยังทำงานต่อ context ที่เปิดอยู่ถือหนึ่งช่องคำขอพร้อมกันฝั่งถอดเสียงของแพ็กเกจ ตั้งแต่เฟรม start จนถึง closed คู่มือ Multiplexing อธิบายเฮดเดอร์ของเฟรม การรีสตาร์ตของเซิร์ฟเวอร์ และลิมิต

เฮดเดอร์#

authorizationBearer pxa_...จำเป็น

Authorization: Bearer pxa_... (recommended).

x-api-keypxa_...

Alternative to the Authorization header. Ignored when Authorization is present.

การยืนยันตัวตน

การเชื่อมต่อ#

wss://api.paxalabs.com/v1/stt/multi Binary frames carry audio in. JSON text frames carry control in, and turn, transcript, and billing events out, each naming its context. Where a message below says connection, read context.

ข้อความฝั่งไคลเอนต์#

startConfigures the session. Must be the first frame; exactly one per connection.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"start"จำเป็น
modelstringจำเป็น

Realtime transcription model id, for example paxa-stt-lite-realtime-v1-preview. GET /v1/models lists the served catalog; a recording model answers unknown_model here.

ตัวอย่าง "paxa-stt-lite-realtime-v1-preview"
audioobjectจำเป็น

What the binary frames carry, fixed for the connection. The bytes are read exactly as declared. A wrong declaration transcribes noise.

ดูฟิลด์ย่อย
audio.encoding"pcm_s16le" or "mulaw"จำเป็น

"pcm_s16le" is signed 16-bit little-endian PCM, "mulaw" is G.711 mu-law as telephony delivers it. Mono.

audio.sample_rate16000 or 48000 or 8000จำเป็น

Samples per second of the binary frames: 16000 or 48000 for pcm_s16le, 8000 for mulaw. Any other pairing answers validation.

languagestring

The language the speech is expected to be in, as a BCP 47 tag such as "th", "en", or "zh". The hint filters nothing. Speech in any other language is still transcribed as spoken, and Thai that switches into English mid-sentence needs no setting. Omitted, the model detects the language itself. Set the tag when detection has picked the wrong language.

up to 16 characterspattern ^[A-Za-z]{2,3}(-[A-Za-z0-9]{2,8})*$ตัวอย่าง "th"
timestamps"word"

Adds a words array to every final transcript, one span per word with its start and end in seconds on the connection's clock. Thai words are segmented for you. Costs nothing extra.

style"verbatim" or "clean"

How faithfully the transcript follows the speech. "verbatim", the default, writes exactly what was said: fillers, false starts, and self-corrections included. "clean" drops fillers and merges broken phrases into one, for a transcript that is easier to read. It is a readability choice. A clean transcript departs further from what was said, and our own measurement found it less exact. Any other value answers 400 validation.

ค่าเริ่มต้น "verbatim"
convention"spoken" or "written"

How the transcript writes what was said. "spoken", the default, writes numbers as they were said (สามสิบเปอร์เซ็นต์) and a word said twice as two words, which is what a verbatim transcript, a subtitle, or a language model reading the text needs. "written" keeps display conventions: digits and units (30%), the repetition mark ๆ, for dictated text that will be read on a page. The word timestamps are the same under both. Only Thai is respelled: a transcript in another language, English and Mandarin among them, reads the same under both.

ค่าเริ่มต้น "spoken"
vocabularyarray of strings

Terms the speech is likely to contain that the model should recognize as written: product names, people, places, jargon, in Thai or any other language. Up to 50 terms of 50 characters each, and a longer list or term answers 400 validation. The terms bias recognition and are never inserted; a term that was not said does not appear. Costs nothing extra and combines with every other option.

ตัวอย่าง ["Paxa Labs","กรุงเทพฯ","quarterly review"]

playbackTells the turn detector when your agent is speaking. Its own voice reaching the microphone then cannot open a turn. While speaking is true, opening a turn takes more sustained speech from the user; a real interruption still opens one and arrives as speech_started. Send it on every change. A session that never sends it treats the room as quiet.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"playback"จำเป็น
speakingbooleanจำเป็น

true while your agent's audio is playing into the same room as the microphone, false once it stops.

endCloses the turn in progress, if any, and delivers its transcript, then the server sends done and closes with code 1000. Audio after end is ignored.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"end"จำเป็น

อีเวนต์ฝั่งเซิร์ฟเวอร์#

เฟรมเสียง: Each frame opens with one byte counting the context id, 1 to 64, then the id in ASCII. The rest is audio in the encoding and sample rate that context's start frame declared, in any chunking and at any cadence, up to 64 KiB a frame with its header. Audio may follow the start frame at once. Audio that names a context which is not open is dropped.

startedThe session is configured and listening.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"started"จำเป็น
connectionstringจำเป็น

Connection id; quote it in support requests (the x-request-id analog).

modelstringจำเป็น

The model serving this session.

speech_startedThe user started speaking. This is the interruption signal. Stop your agent's audio and cancel the reply it was generating. It arrives once enough speech has been heard to be sure. The time it reports is therefore earlier than the moment it arrives.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"speech_started"จำเป็น
turnintegerจำเป็น

The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.

min 1
timenumberจำเป็น

Where the user started speaking. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

transcriptA transcript of one turn. The final transcript is the turn: send its text to your language model. An interim gives an earlier look at the same turn and never needs handling on its own.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"transcript"จำเป็น
turnintegerจำเป็น

The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.

min 1
is_finalbooleanจำเป็น

true for the turn's finished transcript, sent exactly once per turn. false for an interim transcript of the turn so far, sent at a natural pause and replaced by the final.

textstringจำเป็น

The verbatim transcript of the turn, or of the turn so far. Empty on a final when the turn held no speech the model could read.

startnumberจำเป็น

Where the transcribed audio starts. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

endnumberจำเป็น

Where the transcribed audio ends. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

wordsarray of objects

Word spans on the connection's clock, present on a final when the session set timestamps.

ดูฟิลด์ย่อย
words[].textstringจำเป็น

The spoken input text this span covers. An emotion tag takes no span.

words[].startnumberจำเป็น

Where the span begins in the audio, in seconds.

words[].endnumberจำเป็น

Where the span ends in the audio, in seconds.

speech_endedThe user stopped speaking and the turn is closed. Its final transcript follows. Use it to show that the agent is thinking.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"speech_ended"จำเป็น
turnintegerจำเป็น

The turn this event belongs to, counting from 1 per connection. A higher number is a later turn: an agent that already replied to turn 3 discards turn 3 and answers turn 4.

min 1
timenumberจำเป็น

Where the turn ended. In seconds of audio received since the connection's first binary frame, exact to a thousandth.

chargedBilling receipt. The connection is charged for the audio it receives, silence included. A period closes at the end of each turn, after a minute of audio with no turn, and at the end of the session. Receipts replace the x-credits-charged header.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"charged"จำเป็น
seqintegerจำเป็น

The billing period this receipt covers, starting at 1.

secondsnumberจำเป็น

Audio received in this period, in seconds, exact to a thousandth.

creditsnumberจำเป็น

Credits charged for this period, exact to a hundredth. The 0.1-credit request floor applies once per connection, on the first receipt.

total_creditsnumberจำเป็น

Credits charged on this connection so far, the audio of failed turns excluded, exact to a hundredth.

errorSomething failed. A recoverable error leaves the context open. After a fatal one the context sends closed with the code listed below, and the socket itself closes on 4401 and 4403.

context_idstring

The context the error belongs to. Absent when the frame named no context. That is a frame that was not a JSON object, one whose context_id was missing or malformed, or a binary frame the socket does not take or whose header was malformed. Nothing closes on such an error.

type"error"จำเป็น
codestringจำเป็น

Stable machine-readable code from the error catalog. Branch on this field; the errors reference documents every code.

turninteger

The turn the error belongs to, when it belongs to one. That turn delivers no transcript; the session keeps listening.

not_billednumber

Credits this turn's audio is held back from the bill, exact to a hundredth. It comes off what the periods after this turn owe. A turn that fails with little audio behind it returns less than this. Sent with a turn.

retry_afterinteger

Seconds to wait before sending start again. Sent with rate_limited.

doneEvery turn is delivered; the server closes with code 1000 after this event. A connection that closes without done did not finish delivering.

context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
type"done"จำเป็น
total_secondsnumberจำเป็น

Audio received on this connection, in seconds, exact to a thousandth.

turnsintegerจำเป็น

Turns the connection delivered or failed.

total_creditsnumberจำเป็น

Credits charged on this connection, the audio of failed turns excluded, exact to a hundredth. Zero when no turn was delivered, which returns the connection's charge in full.

closedThe context is closed, and no audio follows. A context ends with one closed event unless the socket closes first. The id is free for a new start once it arrives.

type"closed"จำเป็น
context_idstringจำเป็น

The context this message belongs to. You choose it: 1 to 64 characters of letters, digits, and . _ : -, such as your own call id.

pattern ^[A-Za-z0-9._:-]{1,64}$
codeintegerจำเป็น

The context's close code, one of those listed below.

reasonstringจำเป็น

The same in words, for logs.

drainingThe server is restarting. Open another socket now and start every new context there. Contexts already open keep running. Move each call at its next turn boundary. A start sent here from now on answers provider_unavailable and closed. The socket closes with 1001 once its last context has closed.

type"draining"จำเป็น
graceintegerจำเป็น

Seconds until the server closes the contexts still open on this socket.

โค้ดปิดการเชื่อมต่อ#

โค้ดคำอธิบาย
1000A context reports it after done. The socket closes with it only when you close it.
1001The server is restarting. A context reports it when it was still open after the grace the draining event named. The socket closes with it once its last context has closed. It can also close the socket while contexts are open, and those end with no closed event. Open another socket and start again there.
1009The socket closes: a frame exceeded 64 KiB.
4400A context reports it: a frame violated the session protocol, or the start frame failed validation. It also reports it when the start frame named an unknown model or a sample rate its encoding does not take. Any other frame that fails validation answers an error event, and the context stays open.
4401The API key stopped being valid. Every open context reports it, then the socket closes with it.
4402A context reports it: the account cannot cover the request floor at start, or the next billing period.
4403The key's spending cap was reached. Every open context reports it, then the socket closes with it.
4408A context reports it when no frame arrived for it and no event was sent for 300 seconds. The socket closes with it after 300 seconds with no open context and no frame.
4413A context reports it when its frames arrived faster than they could be handled, and a speech context also when its client read a leg too slowly for the leg's time budget, which keeps that leg's charge. The socket closes with it when the client read too slowly and the outbound buffer overflowed.
4429A context reports it: the account holds the plan's full count of concurrent transcription sessions, or the start passed the plan's requests per minute. End another context or connection, or wait the error's retry_after seconds, then start again.
4500A context reports it: internal server error.
4502A context reports it: transcription failed for the whole context. What its delivered periods charged stays charged; a context that delivered no turn is refunded in full. Start it again.
4503A context reports it: transcription was unavailable at start. Nothing was charged; retry.

ข้อผิดพลาด#

400validationThe request body, query, or headers failed validation against the endpoint schema, or broke a rule the endpoint's reference states. A field the schema refused adds two fields to the problem body. path names the first failing field by location and name in dot form, such as body.voice or headers.idempotency-key. A body that could not be read gives the path body alone. reason is one stable code from the validation reasons list.
400live_protocolA frame on a live WebSocket violated the session protocol. The frame arrived before start or repeated start. On a multiplexed socket, it also answers a frame for a context that is not open.
400unknown_modelThe model field does not name a served model.
401unauthorizedThe request carried no API key, or the key is invalid or disabled.
402insufficient_creditsThe account does not have enough credits for this request. Nothing was charged.
403key_limitThis API key reached its spending cap. Nothing was charged.
429concurrency_limitedThe account holds the plan's full count of concurrent requests for this product. Nothing was charged. Each product is limited separately, and an open live connection, or an open context of a multiplexed socket, holds one slot of its product for its lifetime.
500internalRequest state was inconsistent on the server.
502provider_errorModel inference failed after the request was charged.
503provider_unavailableThe model behind this endpoint is not available right now. Nothing was charged.
429rate_limitedRequests per minute for the plan are exhausted. One window covers the whole account, across every product and every key, and voice design calls also have one of their own at 20 per minute.