Thai has no word spaces, and it changes how a translation API has to work

What is worth writing down is how far that one absence reaches into an API. It is not confined to a tokenizer. It decides how markup can be preserved, how a glossary can be matched, what a chunker is allowed to do, and where a request's cost is actually determined. Four decisions in /v1/translate follow from it, and each of them was made the wrong way first.

Word segmentation in Thai is a modelling problem. A lexicon does not settle it. The canonical example is ตากลม, which reads as ตา กลม (round eye) or ตาก ลม (air-dry) depending on the sentence around it. There is no delimiter to consult and no dictionary lookup that resolves it, because both readings are real words.

Any code that needs to know where a word ends is therefore asking a model, and any code that thinks it can find out cheaply is wrong. The useful move is to stop needing to know.

The obvious way to translate marked-up text is to take the markup out, translate the plain text, and put the markup back by character offset. In English this survives contact with reality more often than it deserves to, because tag boundaries usually fall on spaces and a drifting offset lands harmlessly between words.

In Thai it fails on the first inline tag. An offset that moves by one character moves inside a word, and the word it splits becomes two strings that are not words. The failure is silent: the output is well-formed HTML containing nonsense, and nobody notices until a Thai reader does.

So /v1/translate takes a format field of text, markdown, or html, and the model receives the markup itself. Preserving structure requires knowing where the words are, and the model is the only party in the pipeline that does.

The endpoint takes a glossary, which is a set of source and target pairs the caller wants rendered a particular way every time. Sending an entire glossary on every request is wasteful. Terms whose source cannot occur in the text are pruned before anything is billed or sent upstream.

Pruning requires deciding whether a term occurs in a text. The standard test is a word-boundary match, \bterm\b, and it is unavailable here. \b is defined against a word-character class that Thai script does not participate in usefully. Depending on the engine, the assertion then either never fires or fires everywhere. The languages where the test breaks are precisely the languages a Thai translation API serves.

The matcher therefore uses plain containment, and every other decision in it exists to pay for that one.

The suffix rule runs one direction only, and the asymmetry is the interesting part. Stripping a suffix from the term widens what it matches. Appending one would be dead code. term + suffix always contains term, and containment already covers that. Writing the second direction is a common mistake and produces a function that looks thorough and does nothing.

The documented limit is stem-changing inflection. A glossary entry for run does not match ran, and no amount of suffix arithmetic will make it. Two escape hatches exist for the cases that matter: always: true on a term sends it regardless, and glossary_mode: "all" sends the whole set.

Reference material bills at its own rate, and the glossary is reference material. Only the terms actually sent are billed, which means pruning is not an optimization sitting beside the charge. It determines the charge.

That forces an ordering. Pruning a full glossary is the request's one block of real synchronous CPU, measured at 20 to 45 milliseconds at the contract ceilings. A request about to be refused must not spend it. So the text ceiling, the availability check, and admission all run first; the glossary is matched after. A request_too_large refusal that arrives after pruning settles the concurrency slot it briefly held.

The response reports the split back: usage.credits alongside meta.text_chars, meta.glossary_terms (sent) and meta.glossary_available (attached). Without those a caller cannot tell a large glossary from an expensive one, and the number they would optimize is not the number they are paying.

Anything that splits long text falls back on a character count when it has nothing better: a retrieval chunker, a UI truncation, a batching loop. In a spaced language a fixed window usually lands between words by luck, and the failure mode is an awkward cut. In Thai the window lands wherever it lands, including inside a word, inside a number, and between a consonant and the vowel that belongs to it.

There is no cheap fix, because the boundary the splitter wants is the thing the language does not mark. What exists instead is structure the document already carries. A heading is a boundary. A list item is a boundary. A table row is a boundary. None of them require knowing where a word ends.

That is why the OCR endpoint returns Markdown with its headings intact, and why the translation endpoint takes segments. Both give the caller the boundaries the document already has. Neither asks anyone to invent one.

• Send format whenever the text carries markup, and do not strip it yourself.
• Put brand names, product names, and place names in the glossary in the exact form you publish.
• Send segments in one request. The per-request minimum applies once, and the model sees the batch as one context.
• Split long input on the headings and list items the document already has.
• Read meta.glossary_terms against meta.glossary_available before deciding a glossary is too expensive.

None of this is specific to our API. It is what building text infrastructure for a language without word spaces costs, and the cost is paid once in the design, after which the output stops paying it.