Optimize latency
What the API does between a speech request and its first audio, which of those steps a client controls, and what each choice costs.
Between a speech request and its first audio, the API verifies the key, charges the request, prepares the text, and synthesizes the opening of it.
curl -N -s -o /dev/null -X POST https://api.paxalabs.com/v1/tts \
-H "Authorization: Bearer $PAXA_API_KEY" \
-H "Content-Type: application/json" \
-d '{"text": "สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs", "voice": "khanomkrok", "model": "paxa-tts-flash-v2", "stream": true}' \
-w "connect %{time_connect}s, tls %{time_appconnect}s, first byte %{time_starttransfer}s\n"async function firstByteMs(text: string) {
const started = performance.now();
const response = await fetch("https://api.paxalabs.com/v1/tts", {
method: "POST",
headers: {
Authorization: `Bearer ${process.env.PAXA_API_KEY}`,
"Content-Type": "application/json",
},
body: JSON.stringify({
text,
voice: "khanomkrok",
model: "paxa-tts-flash-v2",
stream: true,
}),
});
if (!response.ok || !response.body) {
throw new Error(`TTS failed: ${response.status}`);
}
const reader = response.body.getReader();
await reader.read();
const elapsed = performance.now() - started;
// Read to the end. The client closes a connection with an unread body.
while (!(await reader.read()).done);
return elapsed;
}
// The first call includes the TCP and TLS handshakes.
// The second call reuses the connection.
console.log(await firstByteMs("สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"));
console.log(await firstByteMs("สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"));import os
import time
import requests
# One session keeps the connection open between requests.
session = requests.Session()
session.headers["Authorization"] = f"Bearer {os.environ['PAXA_API_KEY']}"
def first_byte_ms(text):
started = time.perf_counter()
with session.post(
"https://api.paxalabs.com/v1/tts",
json={"text": text, "voice": "khanomkrok", "model": "paxa-tts-flash-v2", "stream": True},
stream=True,
) as response:
response.raise_for_status()
chunks = response.iter_content(chunk_size=None)
next(chunks)
elapsed = (time.perf_counter() - started) * 1000
# Read to the end. The client closes a connection with an unread body.
for _ in chunks:
pass
return elapsed
# The first call includes the TCP and TLS handshakes.
# The second call reuses the connection.
print(first_byte_ms("สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"))
print(first_byte_ms("สวัสดีครับ ยินดีต้อนรับสู่ Paxa Labs"))What runs before synthesis#
The API caches a verified key for 5 minutes, and a request that arrives after the entry expires adds a database lookup. It also caches the account balance for 60 minutes after each charged request. When the cached balance is at least 4 times the cost of the request, synthesis starts at the same time as the charge.
The charge still completes before the API sends the first byte, and a request that the balance cannot cover answers 402 insufficient_credits with no audio. In four cases the API completes the charge first and then starts synthesis, which adds the time of the charge to the wait:
- The cached balance is under 4 times the cost of the request.
- The account has had no charged request in the last 60 minutes, and no balance is cached.
- The key has a credit limit. Use a key without one for requests where first audio matters, and keep the limits on batch keys.
- The request carries an Idempotency-Key. Send the key where a failed request is retried, and leave it off a conversation turn that is spoken again without a retry.
Warm the caches before the first turn#
GET /v1/me caches the key and opens the connection, but it charges nothing and leaves the balance uncached. Only a charged request caches the balance. To have both caches ready before a call's first reply, synthesize one word when the call starts. A first request that skips this step waits for a key lookup and for a charge that finishes before synthesis starts.
Split long text#
A streamed response starts from the opening of the text while the rest is still being synthesized, which keeps first audio nearly independent of length. Text preparation is the exception. It covers the whole text before synthesis begins, and its time grows in proportion to the length. When first audio matters on a long text, send the opening paragraph as its own request and queue the remainder behind it.
Read the stream at network speed#
A streamed request holds its concurrency slot until the response finishes, and synthesis runs many times faster than playback. A client that reads at playback speed holds the slot for the length of the audio. Read the body as fast as the network delivers it and buffer on your side, which frees the slot for the next request.
Choose between HTTP and a live connection#
A live connection authenticates once and then sends only text frames, but it completes the charge for each synthesis before that synthesis starts. For one complete sentence, a streamed HTTP request on an open connection does less work on the server. The live connection is faster when text arrives in pieces from a language model, and when the alternative is a new connection for every turn. Open it when the conversation starts. When the user interrupts, send cancel and keep it open, because a closed connection costs a full upgrade to replace.
Formats and timestamps#
wav and pcm skip the encoder, mp3 adds a short encode, and opus adds the longest one. Each is small beside synthesis. A streamed wav response is 16-bit little-endian mono PCM at 24 kHz behind a 44-byte header whose two size fields are 0xFFFFFFFF. pcm delivers the same samples with no header. A pipeline that takes raw PCM passes them on with no decoder and nothing to strip.
timestamps do not delay first audio. They change a streamed response to NDJSON events that carry the audio as base64, about one third larger and sent uncompressed. A buffered response with timestamps is gzipped when the client accepts gzip, which removes most of that overhead.
Transport#
The API host serves HTTP/2 and HTTP/3. One HTTP/2 connection carries concurrent requests up to your plan's concurrency limit, which removes the need for a connection pool. A response abandoned before its end closes an HTTP/1.1 connection and resets only its own stream on HTTP/2. Preflight responses carry an access-control-max-age of 24 hours.
Retries#
The API does not queue requests. A request over the per-minute window or the concurrency limit answers 429 at once. A refused request still counts against the window, which means a client that retries through a 429 extends its own lockout. Wait for retry-after on rate_limited. On concurrency_limited, wait until one of your own requests finishes.