Jina AI Launches jina-ocr-v1: A 3.4B MoE Document Parsing Model Featuring Native Speculative Decoding That Runs on Budget-Friendly GPUs

Jina AI, now part of Elastic, has introduced jina-ocr-v1, an end-to-end visual document parser that converts PDFs, scans, tables, charts, and invoices into clean Markdown in a single pass. The model carries 3.4B total parameters with roughly 570M decoder parameters active per token, and it ships with a speculative decoding head built into the checkpoint. The design goal is clear: make high-quality document parsing run on inexpensive GPUs like the NVIDIA L4. According to its technical report, the model scores 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench. Weights are open for research and non-commercial use at around 6.8 GB in BF16, runnable on Transformers or vLLM; the CC BY-NC 4.0 license means commercial deployments require contacting Jina AI.

What Is jina-ocr-v1?

The model post-trains DeepSeek-OCR and retains its two efficiency components. DeepEncoder, at roughly 380M parameters, chains SAM, a 16x convolutional compressor, and CLIP-L. It converts a 1024×1024 page view from 4,096 patches into 256 visual tokens, and a dynamic-resolution mode can add up to 9 local tiles of 100 tokens each, capping a page at 1,156 visual tokens.

The decoder is DeepSeek-3B-MoE with 12 layers, 64 routed experts, and 2 shared experts. Top-6 routing activates about 570M parameters per token, and the position limit sits at 32,768. Output is Markdown, with tables rendered in HTML and formulas in LaTeX.

How FastMTP Speculative Decoding Works

Because OCR output is nearly deterministic and locally structured, it suits speculative decoding well. Jina AI adds a FastMTP head: a single dense draft block applied recursively over K=3 steps, with draft parameters kept constant as depth increases.

The decoder then greedily verifies the drafts, accepting the longest matching prefix and committing one additional token of its own. If all three drafts match, that extra token comes free. Since committed text always matches plain greedy decoding, the speedup is lossless. At K=3, the model commits an average of 2.73 tokens per step.

Post-Training With Verifiable Rewards

Post-training combines instruction alignment, robustness fine-tuning on degraded pages, and GRPO. Every reward term is deterministic code scored against a reference transcription, covering content, formulas, tables, structural validity, unit tests, repetition, and format.

Terms are multiplied and graded, so partially correct pages earn partial credit. Structural, unit-test, and format terms are floored at 0.2, and the table term at 0.1, while the repetition term has no floor since loops could inflate the content score. On natural pages, formula and table rewards apply to few samples, so the team built JinaOCRSynth—synthetic pages packed with both, each carrying olmOCR-Bench-style unit tests. An agent also merges candidate checkpoints under a fixed evaluation budget, and the draft head is trained last against the frozen final verifier.

Benchmarks and Throughput

| Model | Params | OmniDocBench v1.6 | olmOCR-Bench |
|---|---|---|---|
| jina-ocr-v1 | 3B/570M | 91.14 | 83.4 |
| DeepSeek-OCR | 3B/570M | not listed | 76.0 |
| DeepSeek-OCR-2 | 3B/570M | 90.25 | not listed |
| PaddleOCR-VL-1.6 | 0.9B | 96.34 | not listed |
| chandra-ocr-2 | 4B | not listed | 85.8 |
| Qwen3-VL-235B | 235B/22B | 89.78 | not listed |

The model does not top accuracy charts—PaddleOCR-VL-1.6 and HunyuanOCR-1.5 (94.74) score higher on OmniDocBench, while chandra-ocr-2 and dots.mocr (83.9) lead on olmOCR-Bench. Still, post-training added 7.4 points over the DeepSeek-OCR backbone on olmOCR-Bench.

Throughput is the headline result. On one A100 40 GB at concurrency 32, jina-ocr-v1 parses 2.57 pages per second—the highest of 14 systems measured, versus 1.22 for olmOCR-2 and 0.38 for chandra-ocr-2. It emits 1,085 output tokens per page, reportedly the shortest output among systems scoring above 83. On an NVIDIA L4 at batch size 1, eager decoding rises from 42.7 to 83.1 tokens per second—a 1.95x speedup at a 57.6% acceptance rate. With CUDA graphs, the baseline is already 158.3 tokens per second, where K=1 performs best at 185.6 tokens per second, a 1.17x gain.

How to Run It

The quickest route is Jina Reader: send a URL to r.jina.ai with the header X-Respond-With: jina-ocr-v1, and Reader fetches the page or PDF, runs the model, and returns Markdown. An X-Page header transcribes a single page of longer documents. Jina AI also hosts an OpenAI-compatible endpoint at https://api.jina.ai/v1/chat/completions, plus a hosted demo.

For self-hosting, weights and custom code ship in one repository and load with trust_remote_code=True. FastMTP requires vLLM 0.21 or later and a one-time register() call; the Transformers path runs only the MoE decoder and ignores the draft weights.

Key Takeaways

• 3.4B total parameters, about 570M active per token, built on DeepSeek-OCR.
• FastMTP drafts 3 tokens per step; greedy verification keeps decoding lossless.
• Scores 91.14 on OmniDocBench v1.6 and 83.4 on olmOCR-Bench.
• Reaches 2.57 pages per second on one A100, the highest of 14 measured systems.
• Available on Hugging Face and through a Jina Reader header today.

Meta description: Jina AI's jina-ocr-v1 is a 3.4B MoE document parser with built-in speculative decoding, hitting 2.57 pages/sec on an A100 for low-cost GPUs.

Tags: Jina AI, OCR, DeepSeek-OCR, speculative decoding, vLLM

Featured image: Abstract illustration of a scanned document dissolving into structured data streams, rendered in cool blues and teals with no people or logos.