API reference

Engine — savitr.mlx_ocr

Generic MLX runtime for Surya OCR — load the model once, OCR page images to text.

This is the engine layer (roll-agnostic): it turns a page image into the model’s raw text — HTML for the base surya-ocr-2, terse rows for the distilled roll model — given whatever prompt you pass. The electoral-roll parsing of that text lives in savitr.rolls.

Runs ~175-180 tok/s on Apple Silicon (~3.6x the llama.cpp f16 pipeline). Reusable for any Surya OCR task, not just rolls.

class savitr.mlx_ocr.MLXSuryaOCR(mlx_path='models/surya-mlx-4bit', max_tokens=8192, prompt='OCR this image to HTML.')

Load an MLX-converted Surya model once; OCR page images to text.

Parameters:

  • mlx_path (str)

  • max_tokens (int)

  • prompt (str)

ocr_image(png_path)

OCR one page image; return (text, generation_token_count).

Return type:

tuple[str, int]

Parameters:

png_path (str)

Electoral-roll parsing — savitr.rolls.parse

Electoral-roll parsing: Surya output (HTML or terse) -> voter records.

This is the roll-specific layer. The generic MLX engine (savitr.mlx_ocr.MLXSuryaOCR) turns a page image into text; these functions turn that text into voter dicts with the canonical fields number, id, elector_name, relationship, father_or_husband_name, house_no, age, sex.

savitr.rolls.parse.TERSE_REPO = 'gojiberries/savitr'

Hugging Face repo for the distilled terse roll model (used when no local copy is present).

savitr.rolls.parse.resolve_terse_model(local='models/surya-terse-8bit')

Return a local terse-model dir if present, else download TERSE_REPO from the Hub.

Return type:

str

Parameters:

local (str)

savitr.rolls.parse.parse_voters(html)

Split page HTML into per-voter records by the Name : anchor (layout-agnostic).

Return type:

list[dict]

Parameters:

html (str)

savitr.rolls.parse.dedupe_voters(voters)

Collapse duplicated rows, keeping the fullest record per voter.

Keys by EPIC id when present (the unique voter key — a repeated EPIC means the model mis-associated or looped, so those rows must collapse), else by identity (name+relation+age). Robust to the messy duplication the VLM emits on looped pages.

Return type:

list[dict]

Parameters:

voters (list[dict])

savitr.rolls.parse.TERSE_COLS = ['number', 'id', 'elector_name', 'relationship', 'father_or_husband_name', 'house_no', 'age', 'sex']

Canonical column order for one terse voter line.

savitr.rolls.parse.TERSE_PROMPT = 'Extract every voter from this electoral-roll page as pipe-delimited rows, one per line, columns: serial|epic|name|relation(F/H/M)|relation_name|house|age|sex'

Instruction given to the distilled model.

savitr.rolls.parse.to_terse(voters)

Render voter dicts to terse pipe-delimited text (the training target).

Return type:

str

Parameters:

voters (list[dict])

savitr.rolls.parse.parse_terse(text)

Parse the terse model’s output into voter dicts (value-anchored, not positional).

The distilled model occasionally drops the relation-code column on hard pages, which would shift every field if split blindly. So we anchor on recognizable values — EPIC at the front, sex (M/F/T) and age at the end, relation only when it is literally F/H/M — and the free-form name/relname/house fall out of what remains. This keeps the reliable fields (EPIC, name, age, sex) aligned even when a middle column is missing.

Return type:

list[dict]

Parameters:

text (str)

Pipeline — savitr.rolls.pipeline

Parse Manipur 2025 rolls into the canonical voter CSV — using the fast MLX OCR engine.

Drop-in alternative to the repo’s RapidOCR-based parse_manipur_2025.py, but ~5× faster and higher fidelity: it OCRs each page with MLX Surya (~34 s/page), parses voters layout-robustly, and reuses the existing fields.py (cover-page metadata + serial dedup) and column schema so the output is byte-compatible with the rest of the pipeline.

Run in the MLX env:

.venv-mlx/bin/python savitr/parse_manipur_mlx.py -f AC01_part001_final_ENG.pdf -o out.csv .venv-mlx/bin/python savitr/parse_manipur_mlx.py -d english/ -o out.csv [–limit N] [–resume]

savitr.rolls.pipeline.html_to_text(html)

HTML -> newline-separated text (so fields.py’s cover regexes see line structure).

Return type:

str

Parameters:

html (str)

savitr.rolls.pipeline.parse_pdf_mlx(eng, pdf_path, dpi, terse=True, cover_eng=None)

OCR + parse one PDF with MLX; return (rows, recon) in the canonical schema.

terse mode (default): the terse model eng reads every page for voters (covers yield none), and — only if cover_eng is provided — the HTML model reads the first two pages for cover metadata. Without a cover model, voter extraction still works; cover metadata is just skipped and the AC/part number is taken from the filename.

HTML mode (terse=False): the HTML model eng reads every page for voters + metadata.

Return type:

tuple[list[dict], dict]

Parameters:
savitr.rolls.pipeline.main()

Run the savitr parse-rolls command: roll PDFs -> canonical voter CSV.

Return type:

int

Schema — savitr.rolls.schema

Canonical in-rolls voter CSV schema (vendored from parse_unsearchable_rolls).

Kept byte-compatible with parse_unsearchable_rolls/scripts/manipur/parse_manipur_2025.py so savitr’s CSV output drops into the same downstream pipeline.

savitr.rolls.schema.ac_part_from_filename(name)

AC01_part001_final_ENG.pdf -> (‘1’, ‘1’).

Return type:

tuple[str, str]

Parameters:

name (str)

Generic MLX backend — savitr.mlx_backend

An MLX inference backend for Surya — Apple Silicon, ~3.6× faster than llama.cpp.

Surya ships vllm (NVIDIA) and llamacpp (CPU/Apple Silicon) backends. This adds a third, mlx, that serves the MLX-converted model through mlx-vlm’s OpenAI-compatible server and plugs into the exact same SuryaInferenceManager / chat_completions_batch path — so the full Surya pipeline (layout, full-page OCR, table rec) runs unchanged, just faster.

Design mirrors LlamaCppBackend: spawn an OpenAI server, attach an OpenAI client, delegate generation to Surya’s shared chat_completions_batch. The mlx-vlm server runs as its own subprocess (its own venv), so it never conflicts with Surya’s dependencies.

Usage (drop-in, no fork needed):

import mlx_backend; mlx_backend.register() os.environ[“SURYA_MLX_MODEL_PATH”] = “models/surya-mlx-4bit” os.environ[“SURYA_MLX_PYTHON”] = “.venv-mlx/bin/python” # python that has mlx-vlm manager = SuryaInferenceManager(method=”mlx”)

Convert the model once:

.venv-mlx/bin/python -m mlx_vlm convert –hf-path datalab-to/surya-ocr-2 –mlx-path models/surya-mlx-4bit -q –q-bits 4

Upstreaming: this file is the body of surya/inference/backends/mlx.py; register() is the two-line addition to surya.inference._build_backend (add an “mlx” branch).

class savitr.mlx_backend.MlxBackend(*args, **kwargs)

Surya inference backend that serves the model via mlx-vlm’s OpenAI server.

start()

Spawn (or reuse) the mlx-vlm server and return its handle.

Return type:

ServerHandle

stop()

Terminate the mlx-vlm server and clear cached client/handle.

Return type:

None

generate(batch)

Run a batch of chat completions against the running server.

Return type:

list[BatchOutputItem]

Parameters:

batch (list[surya.inference.schema.BatchInputItem])

savitr.mlx_backend.register()

Teach SuryaInferenceManager(method=’mlx’) to build this backend.

Upstream, this is just an if method == ‘mlx’: return MlxBackend() branch in surya.inference._build_backend. Here we patch it so no fork is required.

Return type:

None