Documents#Extraction#OCR

MonkeyOCR: a small, multimodal document-parsing model

A lightweight multimodal model parses PDFs in three steps, rebuilding formulas and tables, and beats Gemini 2.5 Pro on OmniDocBench.

Project facts

GitHub Ecosystem
Repositorygithub.com/Yuliang-Liu/MonkeyOCR
License
Apache-2.0
Language
Python
Stars
6,650
Data checked
2026-10-06

Snapshot figures reflect the check date and may change over time.

Parsing PDFs with complex layouts is a mess: modular OCR pipelines are fiddly to assemble, and general-purpose multimodal LLMs aren’t efficient enough for the job. MonkeyOCR is a document-parsing model built on a lightweight multimodal LLM, with a paper already accepted by Science China Information Sciences. It parses PDFs and images in three stages—structure, recognition, relation—and outputs Markdown, layout-annotated PDFs, and block-level JSON. GitHubDaily recommended it in June 2025; on the OmniDocBench benchmark alone, the README claims it beats larger models such as Gemini 2.5 Pro and GPT-4o on both Chinese and English documents. This repository is the v1 line; in July 2026 the team also shipped MonkeyOCRv2 with a document-native vision backbone and a 0.7B parser that claims the new open-source SOTA on 17-language document parsing, while v1 keeps getting maintenance updates.

Core features

  • SRR three-stage paradigm: layout detection, content recognition, and order prediction replace a chain of separate tools, skipping the usual work of stitching OCR components together.
  • Two model sizes: pro-3B and the lighter pro-1.2B—the README reports the latter beats the older 3B version by about 7.4% on Chinese documents, runs roughly 36% faster, trades only about a 1.6% overall performance gap, and beats Nanonets-OCR-3B by 7.3% on olmOCR-Bench.
  • Chinese and English documents: evaluation covers nine page types, including papers, financial reports, exam papers, and magazines, scored separately for each language.
  • Local deployment: Gradio, FastAPI, Docker, and AWQ quantization support local GPU inference with no external API dependency.
  • Three output formats: one pass produces plain Markdown, a layout-annotated PDF, and block-level JSON with coordinates and content, so you can read, proofread, or feed it to downstream code without re-parsing.

Typical use cases

  • Batch-converting formula-heavy papers and textbook PDFs into Markdown.
  • Structured extraction and reading-order reconstruction for multi-column layouts like financial reports and newspapers.
  • Turning the parser into a service to mass-produce RAG corpora.

Benchmarks

The README’s own inference speed, in pages per second on an RTX 4090 across different batch sizes:

Model50 pages100 pages500 pages1000 pages
MonkeyOCR-pro-3B0.9720.9690.9861.006
MonkeyOCR-pro-1.2B1.1941.3141.4421.434

These numbers come from the README’s “Inference Speed” section, with no test date given; figures differ on the 3090, A6000, and H800, so check the full table before picking hardware.

Quick start

  1. See docs/install_cuda_pp.md for environment setup.
  2. Download the weights and parse a document:
python tools/download_model.py -n MonkeyOCR-pro-3B
python parse.py input_path

There’s also a Gradio demo and a FastAPI endpoint for two other starting points: run the Gradio page to upload a file and see the result without writing code, or stand up the FastAPI service and get back Markdown, the layout PDF, and the JSON for your own pipeline; the README gives the exact commands for the Docker image and the AWQ-quantized build too.

Summary

This suits developers, researchers, and data teams who need to parse Chinese and English PDFs locally, especially for preparing RAG source material; if you only scan the occasional document, a cloud OCR API may be less hassle than standing up a GPU box. A few caveats: the source code is Apache 2.0, but the v1 weights are licensed for academic research and non-commercial evaluation only—commercial use requires written authorization; inference requires an NVIDIA GPU, and some Docker images need patching for certain cards; support for scanned photos, handwriting, Traditional Chinese, and other languages is still incomplete, which is exactly what the July 2026 v2 release targets; and this repository, the v1 mainline, sits at about 6.6k stars as of 2026-10-06 and is still maintained. Overall, it’s a controllable, locally deployable document-parsing solution that fits Chinese official documents and financial reports particularly well; for translating a whole English paper into a bilingual side-by-side PDF rather than extracting structure, BabelDOC is the closer match.