Taqi Jaffri (@tjaffri) profile picture

Product @uipath | Co-Founder @docugami (ex) | MSFT (ex). Physicist and engineer by training, product by passion.

Seattle
Joined July 2009
Photo by Taqi Jaffri (@tjaffri)Photo by Taqi Jaffri (@tjaffri)Photo by Taqi Jaffri (@tjaffri)Photo by Taqi Jaffri (@tjaffri)
The industry needs benchmarks with real world examples of the most difficult documents (which happen to be the most valuable for the future of work). Great to see ParseBench leveling up! @llama_index @jerryjliu0 @imaanxsultan
Today we’re introducing ParseBench-2 - our most ambitious document parsing benchmark to date. It is the only benchmark that provides both extreme granularity into every single type of production parsing error, while also making sure that metric scores are highly aligned with human judgment. We do this by painstakingly assigning severity scores to 330+ error types, and tuning each metric to match the severity scores. Existing benchmarks either use brittle deterministic metrics (e.g. OmniDocBench) or LLM-as-a-judge (ParseArena), which can help recognize equivalent representations but are coarse/stochastic/subject to bias by the prompt+model. ParseBench-2 also contains: ✅ 4.3k+ document pages (double from ParseBench-1). ✅ An extra dimension (forms) ✅ Much better ground-truth across all dimensions compared to ParseBench-1. This is the next evolution of ParseBench, our hugely popular benchmark that has 50k+ monthly downloads, over 150+ OCR solutions benchmarked, and has been adopted by companies like NVIDIA, Cohere, Aleph Alpha, Perplexity, and others. Alongside ParseBench-2, we are also releasing OCRia-1.5 - a new family of document parsing models that powers all our modes in LlamaParse. They represent a massive step up in capabilities from our latest release, and are state of the art in parsing complex tables, forms, charts, formatting, and assigning bounding boxes to complex layouts. Come check out our blog post, technical report, and website. Blog: llamaindex.ai/blog/introduci… Technical Report: llamaindex.ai/files/parseben… Website: parsebench.ai/ If you’re interested in trying out our OCRia-1.5 capabilities, come sign up for LlamaParse: cloud.llamaindex.ai/
1
1
4
674
8 years ago, a few ex-Microsoft colleagues and I co-founded a document understanding company. Congrats team @Docugami on joining @InveniamIO! This photo from one of our earliest morale events still makes me laugh out loud (iykyk!!)
3
3
121
Thinking back on the heady early days of LLMs, so many great war stories! Thanks to @hwchase17 at @LangChainAI, @jerryjliu0 at @llama_index, and @lqiao and @the_bunny_chen at @FireworksAI_HQ for working with us super early on community connectors and sovereign AI with open weights models.
1
1
71
Forge is next week… mark your calendars!
Getting a model to work in a demo is the easy part. Making it reliable enough for enterprise automation is the harder job. @tjaffri & Kevin Koehncke of @UiPath take the stage at Fireworks Forge to talk about their learnings from building with Fireworks to scale enterprise-level automation with open models. Watch the keynote live Nov 3: bit.ly/forgelivex
25
This is 🔥 . Leading across benchmarks is quite an accomplishment, since it is important for doc parsing models to be well balanced across real-world document types.
Today we’re releasing OCRia-1, a frontier VLM for document parsing that sets a new SOTA in accuracy, cost, and latency. It not only sets a new high score on ParseBench (87.4), but also leads in three 3rd-party benchmarks: olmOCR-bench* (83.5 overall, 92.9 on tables!), PulseBench-Tab (84.1), and fr-bench-pdf2md (72.1). OCRia-1 is really, really good at parsing complex tables: nested headers, merged cells, irregular grids, scanned tables, and more. It is also fantastic at parsing complex unlabeled charts (line, bar, pie, technical diagrams), and preserving complex formatting. Most real-world enterprise documents contain at least one of the these, whether it’s a big tax withholding table or a public economic report. It is a 4B model and outperforms all other models within a similar parameter class: LightOnOCR-3, Chandra-OCR 2, Mistral OCR 4.1, and more. It is also extremely fast. All credit goes to our research team, who spent countless hours on curating a diverse dataset, tuning our training pipeline, and squeezing every last drop out of model optimization so that it can process billions of pages at scale. OCRia-1 as a base is powering all of our modes within LlamaParse (cost-effective, agentic, agentic plus), so you can access its capabilities today! If you’re processing millions/billions of documents at scale, we’re offering enterprises that book a demo up to $3k in credits for a guided evaluation.** *olmOCR-bench scores are adjusted based on unifying rendering at 1,288px for all models **enterprise POC credits apply to eligible organizations that book a demo
1
7
542
Taqi Jaffri retweeted
When you look back, you’ll wish you’d just done the long-term version of everything.
846
7,190
633
55,042
2,993,628
Taqi Jaffri retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
3,373
12,219
2,709
71,693
9,649,764
Taqi Jaffri retweeted
parsing pdfs isn't that different from building self-driving cars
40
9
6
208
16,174
Bull case for OpenAI
It is somewhat like recommending “dark side of the moon” to someone, but this album is probably the most user friendly introduction to nusrat fateh ali khan Good starting points include “ali maula ali dam dam” and “haq ali ali,” though no one can resist the charm of the first track. “Allah whooooo”
60
Taqi Jaffri retweeted
90% of Eng leadership roles hiring now are one two two shitty options: 1) Please fix our company that is dying in the AI era. All of our good people have left. Please come in and do the AI transformation. You’ll be unable to hire anyone good. You’ll have absolutely no chance. And we can pay you the same as a Staff Engineer at Google. But there is major upside if you literally fix everything. 2) Please make our wayyyy overvalued AI startup actually do real AI stuff. We never had a real Eng leader on the team. We raised at $1B valuation on $1M revenue with a ChatGPT wrapper. We never had a real CTO or Eng leader, just a 25 year old cofounder named Dave who was the only one of us who knew how to launch an EC2 instance. We can pay you the same as a Staff Engineer at Google, but there’s major upside if you literally fix everything.
Interesting trend: CTO/ Head of Eng / VPE folks at startups and mid-sized companies are... leaving / burning out. Hiring for these roles is HARD, but even after filling the role, they will often leave a few months later and take a career break And they have v good reasons
92
167
37
3,345
454,986
Taqi Jaffri retweeted
We've officially entered the era of the B+ employee. The one who shows up. Delivers. Easy to work with. But refuses to be the hero.
77
341
31
6,269
527,721
Taqi Jaffri retweeted
We have opened #DGML under the Apache 2.0 license, with @InveniamIO, creating an open standard for verifiable document data. Combined with @NVNM_Chain, DGML enables document data to be traced to its source, verified, and consumed by AI agents, investors, and auditors. We invite the community to help shape the evolution of the format! dgml.io
6
9
5
29
3,457
Taqi Jaffri retweeted
i used to do this a lot. but then i felt like i was getting progressively dumber at writing. i think there's some value in forcing your own brain to sharpen your thinking, instead of offloading it to AI what i found helpful is to manually type out my own understanding in distilled bullet points after reading the LLM output
One pattern I find useful for working with LLMs is a nice long ramble session. Sometimes the LLM needs more bits to understand what you're trying to achieve, but you're too lazy to type them. In these cases I like to lean back, switch to /voice and just ramble for like 10 minutes, total mess, anything goes, full stream of consciousness. Sometimes I declare it up top, something like "switching to speech recognition sorry for any typos...". Sometimes I turn it into a small interview of a few turns. But I find that the LLMs are somehow very good at reconstructing long incoherent rambles and often their echo of your own tangle of thoughts comes out quite a bit cleaner than what you started with. The result is that you improve the mind meld and have to correct things less from that point on.
26
21
8
335
57,800
Taqi Jaffri retweeted
Hands-on labs, live demos, customer stories on stage. @UiPath VP of Product Management @tjaffri joins GeekWire's John Cook to preview FUSION 2026 and talk about how coding agents are moving teams from task automation to true business process transformation. Watch the full conversation: youtu.be/oMmzMIlJClU #UiPathFUSION
1
741
Taqi Jaffri retweeted
You are far more dangerous to your startup than competitors are. A hundred times more startups die from poor execution by their founders than are killed by competitors.
224
650
89
8,417
430,974
Taqi Jaffri retweeted
If you've adopted AI at your company but haven't seen any tangible results, read this 1990 article: "The Dynamo and the Computer" by Paul David. When electricity first arrived, factories that "adopted" it barely got faster. They just swapped the steam engine for an electric one and ran everything else exactly as before: same machine layout, same workflow, same management. Electricity in, no real gains out. The most common mistake with any new technology is to drop it into the old organization and then declare the transformation done. The real leap came decades later, when each machine got its own small motor. Suddenly machines no longer had to be lined up around one central drive shaft. They could be rearranged around the actual flow of work. The productivity gains didn't come from electricity. They came from REDESIGNING THE ENTIRE FACTORY around it. AI is the same. Bolting it onto your existing process gets you a faster steam engine. The payoff comes when you redesign the work itself. (link to paper in comments)
145
751
119
4,206
299,936