# CPU speech synthesis under a serverless bill

Kokoro-82M served on serverless CPUs, and the two changes to the serving path that take a bursty day's bill from $2.41 to $0.38.

Technical report arXiv:2610.00063v1 (https://arxiv.org/abs/2610.00063v1), 2026-09-04, 6 pages, cs.DC, cs.AI, cs.CL.
Authors: Pakorn Nathong, Kunat Pipatanakul. Affiliations: Paxa Labs.

Figures below are the report's own, measured on its own benchmarks. They are not specifications of any Paxa Labs API.

Wayu-Paxa-TTS-Edge, the open Thai voice from the lab's [first speech report](https://paxalabs.com/research/thai-tts-from-synthetic-speech), is an 82M model on the Kokoro backbone. A model that small runs on a CPU, and a serverless container that scales with traffic is a natural place to host it. This report measures what such a container bills for, with Kokoro-82M itself as the workload.

Two changes to the serving path carry the result. Bounding the CPU each request may use doubles sustained efficiency on OpenVINO, from 1.35 to 2.71 audio-seconds per CPU-second. Releasing what an idle instance holds cuts its billed memory from 12.1 GB to 1.33 GB, and a request arriving afterwards reaches first audio in 2.2 seconds. Over one day of bursty traffic the bill falls from $2.41 on PyTorch defaults to $0.38.

## What a warm instance bills for

An instance-billed platform meters a container for its whole lifetime. It charges CPU-seconds while the container computes and GB-seconds for as long as it holds memory, idle or busy. The meter includes the page cache, and model files the kernel keeps there stay billable until they are reclaimed.

The report prices every configuration at $30 per vCPU-month and $8 per GB-month, round figures of the same order as public list prices. Its conclusions compare configurations and do not depend on the exact rates. At those rates a PyTorch server that holds 8.7 GB while idle costs about $69 a month before it serves a request. Scaling to zero removes the charge, and the next request then waits 7.7 seconds for first audio.

## Sizing each request

The exported model synthesizes one utterance per call. Padding requests of different lengths without masking changes the audio, and the model has no token-by-token decoding loop for a server to batch across. Parallelism comes from running independent requests side by side.

ONNX Runtime and OpenVINO size each request's thread pool to the machine's physical cores by default. Concurrent requests then compete for the same CPUs. The report gives each request a bounded share. Its server runs eight inference streams with two threads each on a 32-vCPU container.

**Audio-seconds per CPU-second at four concurrent clients** (higher is better)

| Configuration | Value |
| --- | --- |
| OpenVINO defaults | 1.35 |
| + sized threads, four streams | 2.31 |
| + compile cache | 2.33 |
| + eight streams | 2.71 |
| + idle release | 2.71 |

Table 2 of the report, each row adding one change to the one above. The thread budget and the stream count carry the gain. The compile cache and the release leave efficiency unchanged.

Sizing trades the latency of a lone request for throughput under load. A short utterance with the machine to itself reaches first audio in 134 ms on OpenVINO defaults and in 493 ms once sized. At four clients the comparison reverses, and median first audio falls from 1.26 to 0.69 seconds. Throughput plateaus near 38 times realtime at eight clients, and requests past that point queue.

## A third state between warm and cold

Eight streams buy that efficiency with memory. The sized server holds 12.1 GB while warm, more than any baseline, and an instance-billed platform charges for it between bursts. Scaling to zero is the usual answer, and it costs a process cold start of 5.7 to 7.7 seconds on the next request.

The report adds a released state between the two. After 45 quiet seconds the server frees the compiled model and its request pool, returns unused allocator memory, and evicts the compile-cache files from the page cache. The process and the compile cache on disk stay. The next request rebuilds inference state from that cache, which skips the restart and the recompile.

**Memory an idle instance holds** (lower is better)

| Configuration | Value |
| --- | --- |
| OpenVINO defaults | 5250 MB |
| + sized threads, four streams | 8684 MB |
| + compile cache | 7490 MB |
| + eight streams | 12097 MB |
| + idle release | 1331 MB |

Table 2 of the report, read 600 seconds after the last response. The eight-stream row is read at 30 seconds, before any release.

A request that arrives after release reaches first audio in 2.2 seconds, within 2.1 to 2.3 across ten cycles. A process cold start of the same configuration takes 6.2 seconds.

## One day of bursty traffic

The report tests both changes on a synthetic day of two audio-hours, delivered in eight equal bursts across 24 hours. The container is never scaled to zero between bursts.

**One day of bursty traffic, billed in US dollars** (lower is better)

| Configuration | Value |
| --- | --- |
| PyTorch defaults | 2.41 |
| Request sizing alone | 3.24 |
| Request sizing with release | 0.38 |

Figure 1 of the report, at $30 per vCPU-month and $8 per GB-month. Each row is one server left running through the same day.

Request sizing alone makes the day more expensive. The sized server spends less CPU on each burst and then holds 12.1 GB until the next one, and the day costs $3.24 against $2.41 on PyTorch defaults. Idle memory was already 95% of the PyTorch bill. Release is what turns the efficiency gain into a lower bill, at $0.38.

## What the server scores

**Kokoro-82M served six ways on one 32-vCPU host**

| System | Efficiency ↑ | First audio ↓ | Idle memory ↓ | Cold start ↓ | Per audio-hour ↓ |
| --- | --- | --- | --- | --- | --- |
| PyTorch eager | 0.66 | 2.46 s | 8659 MB | 7.7 s | $0.0631 |
| ONNX Runtime default | 0.89 | 0.94 s | 3969 MB | 5.8 s | $0.0468 |
| ONNX Runtime sized | 1.36 | 2.36 s | 3958 MB | 5.7 s | $0.0306 |
| OpenVINO default | 1.35 | 1.26 s | 5250 MB | 6.9 s | $0.0309 |
| OpenVINO sized | 2.71 | 0.69 s | 12097 MB | 7.4 s | $0.0154 |
| OpenVINO sized, with release | 2.71 | 0.94 s | 1331 MB | 6.2 s | $0.0153 |

Table 1 of the report. Efficiency is audio-seconds per CPU-second and first audio is the median, both at four concurrent clients. Idle memory is read 600 seconds after the last response, and at 30 seconds for the sized row without release. Cold start runs from process start to first audio. Cost is the busy cost per audio-hour at the report's example rates.

Busy cost per audio-hour falls to $0.0153, 4.1 times below PyTorch defaults and 3.1 times below ONNX Runtime defaults. The complete system holds the least memory once idle and ties for the highest efficiency. It gives ground elsewhere. Median first audio at four clients is 0.94 seconds against 0.69 for the sized server without release, and ONNX Runtime starts a fresh process faster.

## Where the method stops

With both changes in place, the report looks for further gains and keeps none. Static-shape compilation shortens per-call time by 3.3%, inside the variation between repeated runs. Padding inputs to a fixed length fails the output check. A 36-token utterance padded to 128 tokens yields 3.1 times as much audio, because the model synthesizes the padding.

The study covers one model family, one runtime for the approach, two CPU architectures, and a synthetic trace. Quantization, SMT, and core pinning were left untested. The Thai student built on the same backbone was outside the study. On an AMD Zen 5 host the same design reaches 3.26 audio-seconds per CPU-second and releases to 1.13 GB. A second model, the 100M pocket-tts on its own PyTorch server, holds 1234 MB through a ten-minute idle window and has no released state.

## If you serve speech on CPUs

On the report's bursty day a small speech model costs far more idle than busy, and a faster server raises the bill when it holds more memory. The implementation is about 1,900 lines of Python on OpenVINO with no custom kernels, and it is open with the raw logs behind every figure.

This is research. It is not a product specification. Paxa Labs serves speech through its own [text to speech API](https://paxalabs.com/text-to-speech), which is a different model with its own published limits. The dollar figures price cloud compute at example rates and say nothing about what the API costs.

## Released artifacts

- paxalabs/cpu-tts-to-the-wall: https://github.com/paxalabs/cpu-tts-to-the-wall