# Thai PDFs into a search index: OCR, chunking, and the missing spaces

How to turn scanned Thai documents into chunks a search index or a language model can use, when the text has no spaces to cut on.

Published 2026-09-07. Paxa Labs.

Most Thai source documents a company wants to search arrive as scans. Government circulars, contracts, board minutes, and older annual reports are PDFs of paper, and the ones with a text layer often carry a broken one. Once the pages are read, the text has no spaces between words and no full stops between sentences, and the chunker written for English cuts it in the wrong places.

A Thai document goes from a scan to a chunk in an index in five steps. Read the pages as structure, see why fixed windows fail on Thai, choose where to cut, decide what to carry with each chunk, and normalize before anything is indexed. The examples use [Paxa OCR Lite](https://paxalabs.com/docs/ocr) for the reading. The chunking applies to any reader that returns structure.

## Scans and broken text layers

A Thai PDF is one of two things. A scan has no text at all, and every character on the page has to be read from pixels. A born-digital PDF has a text layer, and Thai text layers fail in the ways the [extraction guide](https://paxalabs.com/guides/thai-pdf-text-extraction) catalogs. Tone marks are stored out of order, legacy fonts map bytes to the wrong characters, and columns come back interleaved. Both cases end in the same place. The page has to be read as a page.

A pipeline that tries the text layer first and falls back to OCR on failure needs a way to detect the failure. A corrupted Thai text layer looks like text to every check short of a Thai speaker reading it. Reading every page the same way costs a few credits per page and removes the branch.

## Read the document as blocks

Ask for structure at the reading step. With `output` set to `"structured"`, [POST /v1/ocr](https://paxalabs.com/docs/api/ocr) returns each page as typed blocks in reading order. The blocks are headings with a level, paragraphs, lists with their items, tables as rows of cells, and figures with their captions. The lines below are the kind of blocks one page of a regulation returns.

one page as structured blocks:

```
{ "type": "heading", "level": 1, "text": "ระเบียบการเบิกค่าเดินทาง" }
{ "type": "heading", "level": 2, "text": "หมวด ๒ อัตราค่าพาหนะ" }
{ "type": "paragraph", "text": "ข้อ ๗ การเดินทางโดยรถไฟให้เบิกได้ตามที่จ่ายจริง ..." }
{ "type": "table", "rows": [["ประเภท", "อัตรา (บาท/กม.)"], ["รถยนต์ส่วนตัว", "๔"], ["รถจักรยานยนต์", "๒"]] }
```

One request reads at most 50 pages and 10 MiB, and a longer document is sent as parts. Keep the page numbers continuous across the parts, because the page is what a citation points at. The Markdown output carries the same reading with the structure as heading marks and pipe tables, and a pipeline that already parses Markdown can start there. The blocks save the parse.

## Why fixed windows fail on Thai

The usual chunker takes a window of a few hundred characters and slides it forward with some overlap. It trusts that a boundary near a space or a period lands near a meaning boundary. English gives it a space every five characters and a period every twenty words. Thai gives it a space every clause and nothing between words. A window that ends on a character count ends inside a word most of the time, and inside a sentence almost every time.

The overlap hides the damage without repairing it. A word cut in half appears whole in the next chunk, and the half in this one is a string no query will match. A sentence split across two chunks has its subject in one and its verdict in the other, and a retrieval that returns one of them returns a fragment that answers nothing.

Chunkers that count tokens have the same problem one step removed. The token boundary is a property of the tokenizer, and it has no more idea where a Thai word ends than the character count does.

## Cut on the structure the reader returned

The document already carries its boundaries. Headings open sections, paragraphs are units of meaning the author chose, and a table is one object. A chunker that walks the blocks needs three rules and no Thai dictionary.

- A paragraph is the default chunk. Merge short consecutive paragraphs under one heading up to a size cap. A merge stops at a heading.
- A table is one chunk, whole, with its header row kept. A row alone loses the column names, and a table split by size loses the rows that answer the question.
- A list is one chunk when it fits. When it does not, split between items and keep each item whole.

A paragraph that exceeds the cap on its own is the one place a cut inside prose is needed. Cut on the Thai space, which is a clause boundary at least and a sentence boundary often, and prefer the space nearest the cap from below. That keeps a clause together, and it never lands inside a word. The [segmentation post](https://paxalabs.com/guides/thai-word-segmentation) explains why a cut inside a word is worse in Thai than a cut inside a sentence.

The size cap is set by the model that reads the chunks and the number of chunks returned per query. Set it in characters of Thai, measured on the corpus. A Thai character carries more of a word than a Latin one, and a cap tuned on English prose is loose on Thai.

## Carry the heading trail and the page

A paragraph under หมวด ๒ อัตราค่าพาหนะ means something different from the same paragraph under หมวด ๓. Prefix each chunk's text with its heading trail when it goes to the embedding, and store the trail as a field for display. Store the page too. A user who asks where a rule came from wants the page, and a reviewer who checks a reading wants to open the scan at it.

one chunk in the index:

```
{
  "doc": "travel-regulation-2569.pdf",
  "page": 3,
  "trail": ["ระเบียบการเบิกค่าเดินทาง", "หมวด ๒ อัตราค่าพาหนะ"],
  "text": "ข้อ ๗ การเดินทางโดยรถไฟให้เบิกได้ตามที่จ่ายจริง ...",
  "match": "ข้อ 7 การเดินทางโดยรถไฟให้เบิกได้ตามที่จ่ายจริง ..."
}
```

Numbered clauses such as ข้อ ๗ are worth a field of their own in a regulation or a contract, because a query often names the clause. The reader returns the number as printed. The pipeline reads it off the start of the paragraph.

## Normalize a matching copy, keep the original

A Thai document carries several spellings of the same value, and a query uses one of them. Build a second copy of each chunk for matching, and leave the original for display and citation.

- Thai digits to Arabic. ๒๕๖๙ and 2569 are the same year, and a document mixes the two on one page.
- Buddhist Era years carried as printed, with the Common Era year in a separate field. Subtracting 543 in the text itself breaks the citation.
- The vowel อำ as one code point. Some documents store it as นิคหิต followed by สระอา, which Unicode normalization leaves alone, and the two forms never match.
- Unicode NFC, which fixes the order of stacked marks a broken text layer produced.
- Whitespace collapsed to one space, and zero-width characters removed. A reader returns none, and a text layer returns many.

Run the same normalization on the query. A mismatch between the two sides is the most common reason a term that is visibly on the page is not found.

## Lexical, vector, or both

A vector index embeds the chunk and needs no word boundaries. The embedding model reads the characters, and the chunk boundary above is the only Thai-specific decision. A lexical index stores terms, and a term is a word. A Thai lexical index therefore needs a segmenter on both the chunk and the query, the same one on both, and a rebuild when the segmenter changes.

Exact strings are where the lexical side earns its place. Clause numbers, document ids, tax ids, amounts, and names are what a vector index blurs and a term index nails. A hybrid that scores both and merges the lists handles a regulation well, and the normalized copy is what the lexical side indexes.

## What a fifty-page report costs to read

A page costs 6.5 credits whatever it holds, and the page count is known before the charge. A 50-page report is 325 credits, which is $0.33. Read a document once, and rebuild the chunks from the stored blocks whenever the chunking rules change. Re-read a page only when the page changes.

Store the blocks as well as the chunks. The chunk cap, the trail format, and the normalization all change as the index is tuned, and each change is a rebuild from stored blocks at no cost.

## Next steps

- Read the [OCR guide](https://paxalabs.com/docs/ocr) for the outputs and limits, and the [reference](https://paxalabs.com/docs/api/ocr) for the block fields.
- Read the [extraction post](https://paxalabs.com/guides/thai-pdf-text-extraction) for what a Thai text layer gets wrong.
- Open the [documents tool](https://paxalabs.com/app/documents) to read a page and see the blocks it returns.