9 Engine and Option Reference

Model selection, validated option profiles, and why 1 worker is correct

9.1 Model selection by document type

Document class	Default engine	Notes
Printed scholarly PDFs (Euler, Kepler, Copernicus, Bischoff, Waefler)	`kimi-k2.7-code:cloud` via `ai-scholar process`	Standard PDAI default (owner decision 2026-09-26)
Handwritten / archival correspondence at scale	`kimi-k2.7-code:cloud`	Validated on Karman Nachlass (1,066 docs, 38,172 pages)
Harvest and local-only runs	`qwen3.8:27b-mlx` (local MLX)	Harvest scripts must not import cloud APIs
Large-format printed with Fraktur + Latin + math	`qwen3.8:27b-mlx` (local MLX)	Validated on Kepler KGW 13–15
Fallback (any class)	`gemini-3.8-flash`	Only when Kimi fails on the page (HTTP error, timeout, empty answer); incurs API cost

Use kimi-k2.7-code:cloud (Ollama :cloud tag) as the default for new printed and handwritten collections. ai-scholar process --fallback gemini-3.8-flash (the default) re-runs a page with Gemini only when the Kimi call fails or returns no text, and stamps the engine that produced the page in <meta model>. qwen3.5:cloud is retired (HTTP 410 since 2026-09-25). Do not use fastimage for schema-first work (Apple Vision returns unstructured text; the PDAI pipeline requires typed XML).

9.2 Validated option profiles (ocr_runner)

The ocr_runner package (pdai/pipeline/ocr-runner/) provides named option presets:

9.2.1 `MLX_FAST` — validated default for local qwen3.8:27b-mlx

MLX_FAST = {
    "num_ctx":     8192,
    "num_predict": 2048,
    "num_batch":   512,
    "temperature": 0.1,
    "reasoning":   False,
    "think":       False,
}

Measured on KGW-13, page 57 (qwen3.8:27b-mlx, M1 Ultra):

Profile	Wall clock	Prefill	Decode
Legacy (`16384/4096`, no `num_batch`)	61.0 s	26 637 ms	30 859 ms
`MLX_FAST` (`8192/2048/512`)	31.7 s	126 ms	31 359 ms

Prefill collapses 211×. Decode is unchanged (memory-bandwidth bound). In production the validated fast profile yields ~34 s/page vs. ~97 s baseline.

9.2.2 `CLOUD` — for `:cloud` Ollama endpoints

CLOUD = {
    "temperature": 0.1,
    "think":       False,
}

num_batch and num_ctx are ignored by remote API endpoints; they are omitted from the CLOUD preset to avoid confusing log output.

9.2.3 Per-collection deltas

Collections may declare a small delta applied on top of the base preset. The diaries collection adds repeat_penalty: 1.15 to suppress repetition of Babylonian month names:

DIARIES_DELTA = {"repeat_penalty": 1.15}

9.3 Worker count: why 1 is correct

On an M1 Ultra running qwen3.8:27b-mlx, two concurrent Ollama requests share one model instance. The model is memory-bandwidth bound, not compute bound.

Measured concurrency impact (KGW-13):

Workers	Per-page latency	Throughput
1	~34 s	~1.76 pages/min
2	~62–68 s	~1.8–1.9 pages/min

Two workers roughly doubles per-page latency for a marginal throughput gain (~8%). The memory cost is disproportionate. Default workers is 1 in all production runners. Never label 2 workers as “2× throughput”.

The OLLAMA_NUM_PARALLEL=2 environment variable and --workers 2 flag remain available for experimentation but should not be used in production on M1 Ultra.

9.4 Image scale

Local qwen3.8:27b-mlx: use --scale 0.5 (50% of original resolution). Higher resolution does not improve OCR quality on this model but increases memory pressure and decode time.

Karman kimi-k2.7-code:cloud: images downscaled to 1200 px max width. The Kimi API bills by image dimensions; the 1200 px ceiling provides good quality at reasonable cost.

Gemini gemini-3.8-flash (cloud fallback): bills fixed 1120 tokens/image at high resolution regardless of pixel count.

9.5 Engine capabilities

Engine	Fraktur	Handwriting	Math/tables	Local	Cost
`qwen3.8:27b-mlx`	✓	△	✓	yes	$0
`qwen3.5:cloud`	retired	—	—	HTTP 410 since 2026-09-25	—
`kimi-k2.7-code:cloud`	✓	✓✓	✓	via Ollama `:cloud`	API
`gemini-3.8-flash`	✓	✓	✓	no (API only)	API
Apple `fastimage`	✗	✗	✗	yes	$0

fastimage (Apple Vision) returns unstructured plain text and must not be used for schema-first OCR work. It remains available as the backbone for non-schema classification tasks in ai-classifypdf.

9.6 MLX compile and Python version notes

qwen3.8:27b-mlx runs via the Ollama mlx-community backend; no recompile needed.
The Python 3.15rc1 environment on mac-studio does not have a pydantic_core wheel. Run new FastAPI/pydantic projects from the platform venv with PYTHONPATH rather than creating a fresh venv.
mlx-vlm breaks on transformers>=5.6 — pin transformers==5.5.0 if using mlx-vlm directly (e.g. for the GLM-OCR experiment).