vit.cpp, modernbert, lightonocr

Joined January 2024
🚀 LightOnOCR-3 is out! 🦉 This time we go beyond OCR: text transcription, document layout, image descriptions and chart extraction, all in one model. We release 3 sizes: 0.8B, 1B and 4B, all under Apache 2.0!
18
29
6
223
12,419
Said Taghadouini retweeted
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
134
775
143
6,367
754,782
« Scores are adjusted for olmOCR-bench based on standard 1,288px rendering for all models » this is like using the same lr for all models in an ablation and claim fairness 😹
Today we’re releasing OCRia-1, a frontier VLM for document parsing that sets a new SOTA in accuracy, cost, and latency. It not only sets a new high score on ParseBench (87.4), but also leads in three 3rd-party benchmarks: olmOCR-bench* (83.5 overall, 92.9 on tables!), PulseBench-Tab (84.1), and fr-bench-pdf2md (72.1). OCRia-1 is really, really good at parsing complex tables: nested headers, merged cells, irregular grids, scanned tables, and more. It is also fantastic at parsing complex unlabeled charts (line, bar, pie, technical diagrams), and preserving complex formatting. Most real-world enterprise documents contain at least one of the these, whether it’s a big tax withholding table or a public economic report. It is a 4B model and outperforms all other models within a similar parameter class: LightOnOCR-3, Chandra-OCR 2, Mistral OCR 4.1, and more. It is also extremely fast. All credit goes to our research team, who spent countless hours on curating a diverse dataset, tuning our training pipeline, and squeezing every last drop out of model optimization so that it can process billions of pages at scale. OCRia-1 as a base is powering all of our modes within LlamaParse (cost-effective, agentic, agentic plus), so you can access its capabilities today! If you’re processing millions/billions of documents at scale, we’re offering enterprises that book a demo up to $3k in credits for a guided evaluation.** *olmOCR-bench scores are adjusted based on unifying rendering at 1,288px for all models **enterprise POC credits apply to eligible organizations that book a demo
3
4
21
2,518
First finetunes for LightOn landed over the weekend!!! So far, the finetunes are for medieval Latin 0.8B and 4B and cultural heritage English 0.8B and 4B. As expected, these models took to finetuning nicely and outperform my models from a few weeks ago. I think I can push more.
World-leading document recognition. Beyond text extraction. Introducing LightOnOCR-3, our new family of state-of-the-art AI models for document intelligence. Building on LightOnOCR-2, it recognizes text and handwriting, describes images, extracts data from charts and preserves document structure, all in a single processing step. 🏆 Leading benchmark results : ahead of Chandra-OCR-2, Mistral OCR 4.1 and dots.mocr on OlmOCR-Bench, and first among open-weight models on ParseBench. 📊 More information from every document : turn text, tables, images and charts into content your business applications can use. ⚡Compact and efficient: 0.8B and 4B versions, with document processing up to twice as fast as LightOnOCR-2. 🔓Apache 2.0 license: use, adapt and deploy the models on your own infrastructure. From handwritten administrative files to financial reports and technical documents, make more of your information searchable and usable. 👉 Explore the models and benchmark results: huggingface.co/blog/lightona… 👉🏻Explore the model page on our Website : lighton.ai/research/lightono… #LightOn #DocumentIntelligence #OCR
2
5
1
22
1,164
Said Taghadouini retweeted
🚀 Introducing ColGrep-Agent-2B! An open 2B model, now integrated into NextPlaid via ColGrep, that runs locally to make coding agents more efficient. With Claude Code: → 34% fewer API tokens → 40% lower API costs All by delegating code search to a tiny agent. Here's how 🧵
5
10
2
29
2,773
Said Taghadouini retweeted
Today we’re releasing LightOnOCR-3, the third generation of our OCR models—a joint effort with @staghado and @baptaubertin! Here’s how we built its grounding data, and how intermediate checkpoints helped improve the annotations used for subsequent training. 🧵
3
12
41
2,677
Said Taghadouini retweeted
🚀 We just updated LightOnOCR! 🦉 LightOnOCR-3 is faster, more accurate, and brings new capabilities: it now tells you where everything is on the page, describes images, and pulls the numbers out of charts. Three sizes (0.8B, 1B, 4B), still Apache 2.0.
3
4
10
421
Said Taghadouini retweeted
World-leading document recognition. Beyond text extraction. Introducing LightOnOCR-3, our new family of state-of-the-art AI models for document intelligence. Building on LightOnOCR-2, it recognizes text and handwriting, describes images, extracts data from charts and preserves document structure, all in a single processing step. 🏆 Leading benchmark results : ahead of Chandra-OCR-2, Mistral OCR 4.1 and dots.mocr on OlmOCR-Bench, and first among open-weight models on ParseBench. 📊 More information from every document : turn text, tables, images and charts into content your business applications can use. ⚡Compact and efficient: 0.8B and 4B versions, with document processing up to twice as fast as LightOnOCR-2. 🔓Apache 2.0 license: use, adapt and deploy the models on your own infrastructure. From handwritten administrative files to financial reports and technical documents, make more of your information searchable and usable. 👉 Explore the models and benchmark results: huggingface.co/blog/lightona… 👉🏻Explore the model page on our Website : lighton.ai/research/lightono… #LightOn #DocumentIntelligence #OCR
28
87
21
665
61,089
We also push quality further with multi-objective RLVR! In the same training runs, we jointly optimize grounding, OCR across different document types, and empty-page handling.
1
1
7
533
As always LightOnOCR-3 is a team effort with @CavaillesAdrien and @baptaubertin at @LightOnIO 🦉🚀
1
12
451
Why not just output HTML for grounding? We use compact inline markers for region labels and coordinates to save on output tokens. On the same 512 pages, our grounded outputs use 9–14% fewer tokens than Chandra 2 and Infinity Parser Pro 2.
1
1
8
596
How did we train grounding? Turns out getting reliable document bounding boxes is quite hard. We(@CavaillesAdrien) built an annotation pipeline combining multiple OCR and layout detectors, then aligned their outputs with our LightOnOCR-2 transcriptions.
1
1
11
577
What about document understanding? On ParseBench, LightOnOCR-3 takes the top 2 spots among open-weight models: 🥇 4B: 75.1 🥈 0.8B: 74.6 Infinity Parser Pro (35B): 74.3 Chandra 2 (4B): 70.1 The 4B also leads on chart extraction and semantic formatting.
1
1
9
739
Speed matters too 🏎️ Our 4B is 19% faster than Chandra 2(4B) on single-page latency and processes 21% more pages/s at their respective benchmark settings. Both use Qwen3.5-4B, but we generate fewer tokens thanks to a more compact output format.
1
1
8
669
What's new compared to LightOnOCR-2? We have added full page grounding with full backward compatibility for OCR only mode. The models accept a single prompt "grounding" to steer between grounding and OCR only modes. We also caption images and convert charts to tables
1
1
10
998
How good is the OCR? On olmOCR-Bench: - LightOnOCR-3-4B: 86.3 - Infinity Parser Pro (35B): 87.6 - Chandra 2 (4B): 85.8 And our smallest 0.8B model scores 85.5! Pretty close to a 35B model with less than 1B parameters.
1
1
1
15
1,391