@log10io

AI-powered medical writing for CROs, consultancies & life sciences companies. Generate compliant docs faster across clinical, regulatory & strategic workflows.

log10.io
Joined May 2023
  • Tweets396
  • Following44
  • Followers157
230Media
1/ We tested Jev, @typesafeai's low-cost model that returns probabilities, on clinical research tasks from ClinReg, our benchmark for clinical and regulatory work. Three roles: answering, judging, and selecting answers for review. everest.log10.io/news/evalua…
2
1
2
68
more replies
4/ Selecting for review: Jev's picks led to more corrections than random on screening, but not on evidence answering.
1
8
5/ The takeaway: Jev’s probabilities can be useful in the right role, but that has to be tested task by task.
6
🎉 Our applied AI research team had a paper accepted at the NeurIPS 2026 Workshop on Evaluation of Interactive Agents. Our work focuses on evaluating agents in clinical and regulatory work. 🧵👇
1
2
2
85
more replies
8/9 Where the evaluator sits matters too: a grader that's safe after a task finishes can leak information or shift behavior when its feedback returns mid-task. Our fix: report four layers — outcome, trajectory, reliability, and system configuration.
1
11
9/9 These are the hard problems our applied AI team takes on — helping pharma and medtech teams build and ship custom AI into production. See you in Atlanta on Dec 13. #NeurIPS2026 #AIEvaluation #AIAgents
18
Which AI models are best at systematic literature review? We added TrialPanorama's SLR tasks — a peer-reviewed dataset from a Nature journal — to ClinReg. Two of the top six are open-weight. GLM-5.3 ranks 4th overall, ahead of Opus 5. 🔗 everest.log10.io/news/expand…
1
1
187
ClinReg, our real-world benchmark for this work, now has a live website: the best model for TLF programming is open-weight. clinreg.log10.io Updated results with GPT-6 Astra, GLM-5.3, GLM-5.3 Flash, Gemini 3.8 Flash, Gemini 3.7 Flash, and Muse Spark 1.3
3
1
4
346
Models from @OpenAI, @Zai_org, @GoogleDeepMind, and @AIatMeta
21
Regulatory documents that build themselves from your data — and stay current as it changes. 📄 Source data in, submission-ready doc out 🔍 Checked by multiple models ✍️ Your reviewer signs off 🔄 Affected docs update when new data lands How Everest works, in 90 seconds 👇
1
1
209
Open-weight models match the frontier on benchmarks. But can they be trusted with regulatory and clinical-trial work? We built ClinReg to find out — and ran 19 leading models through it. 🧵
1
1
1
92
more replies
One more: the top open-weight models weren't American. The best came from GLM-5.2 and Kimi K3. The strongest US open-weight models were our lowest performers — Gemma 4 31B (73.5) and Nemotron 3 Ultra (70.3).
1
31
The takeaway: model choice is now the easy part. What's left is engineering — model + harness fit, validation, inference routing. That's the complexity we absorb so life-sciences teams can focus on the deliverable. Full write-up 👇 everest.log10.io/news/open-w…

Open-Weight LLMs Have Caught Up on Accuracy

Introducing the ClinReg Benchmark for Life Sciences. This post walks through what we measured, what surprised us, and what it means for anyone deploying AI in clinical and regulatory settings.

everest.log10.io
1
14
Load more