Head of Multimodal AI @AstraZeneca • PhD @harvardmed
Cambridge, MA, USA
Joined January 2009
- Tweets290
- Following559
- Followers1.1K
- Likes1.7K
Pinned Tweet
I’m incredibly excited to share some big news to ring in the new year: @modella_ai is joining @AstraZeneca!
From the very beginning, Modella was built with a clear conviction: for #AI and #machinelearning to solve grand challenges in #oncology, it has to meaningfully improve how clinicians and scientists work - reducing friction in discovery and enabling better decisions where complexity is highest. That belief shaped our focus in building not only better foundation models, but also breakthrough systems that can reason across the biological spectrum, from single cells to #pathology slides to patient cohorts.
When we first partnered with AstraZeneca, it was immediately clear that we shared this mission. Transforming oncology and biomarker discovery requires more than powerful models. It demands deep clinical context, rigorous validation, and scale. This next step brings those elements together by combining Modella’s #GenAI and #AgenticAI capabilities with AstraZeneca’s oncology portfolio and translational science expertise, unlocking impact that wouldn’t be possible otherwise.
Building Modella alongside this team has been one of the most meaningful experiences of my career thus far. From our earliest angels to our most recent hires, every person has contributed to our mission in immeasurable ways: @AI4Pathology, Jill Stefanelli, Long Phi Le, @MYLu97, Gabi Raia, David Dalton, @kuanchen22, Paul Doucet, @chenbowen118, Drew Williamson! I’m excited to bring this team into AstraZeneca and continue our work with Jorge Reis-Filho and the broader AZ organization.
Please see our press release at: businesswire.com/news/home/2…
Richard J. Chen retweeted
We removed an LM's ability to speak German by fine-tuning on only 4 German tokens.
As part of a 1-day hackathon with our product Silico, we removed a 67M-parameter language model's ability to predict German text, by tuning only a scalar factor on one subcomponent of the weights. (1/6)
Really awesome work by @Junhao_Liang @jnkath !
Prototypes are the way to go for representation learning and biomarker discovery in pathology. CRC-100K is still slept on!😎
very happy to share our lab's latest work, published today in @Cancer_Cell : sciencedirect.com/science/ar…. We show that we can decompose pathology images of cancer tissue into their constituent parts, measure tissue patterns with AI, and that these tissue patterns (1/n)
Richard J. Chen retweeted
very happy to share our lab's latest work, published today in @Cancer_Cell : sciencedirect.com/science/ar…. We show that we can decompose pathology images of cancer tissue into their constituent parts, measure tissue patterns with AI, and that these tissue patterns (1/n)
Richard J. Chen retweeted
there is no better time in tech than now to be a jack of all trades, master of a few.
just make sure to keep adding to the few year over year, such that the cumulative breadth of expertise you collect becomes an increasingly rare combo. remember, if you're top 10% in 3 different areas, that already makes you top 0.1%. keep switching it up until you get to "your best", and then switch it up again (great for a particular flavor of people who don't enjoy resting on laurels, maybe not so great for others).
question all institutional value and pedigrees, all traditional career paths or corporate ladders: the college industrial complex is getting shaken up, alongside a disappearing managerial class, so if you're pursuing either make sure you are fully internally aligned with why. social/political capital in a particular institution can feel incredible, but if you're spending all your energy on complex political people games, you're not a technologist anymore, you're an unelected politician. if you're ok with that, then all's well.
critical thinking is more important than ever: take nothing at face-value, question everything and everyone. the equivalent of ai slop can be found in humans operating under misaligned incentives and interests. the sooner you're clued into disambiguating the talkers/larpers from the doers, the better off you'll be figuring out where and who to invest your time in.
the anxiety of job displacement is very real, since a surprising amount of white collar work/prestige is built on a performative house of cards, significantly lacking in correlation with technical breadth, depth, and skill. as long as you keep learning, keep building, keep producing receipts, you will be fine.
if all that sounds ok to you, welcome to the world of technology! it's truly one of the few places you can experience child-like wonder every few years, and be constantly humbled & excited by new adventures, as scary as they may seem at first.
don't give up, drink your water, get your sunlight, and take breaks as needed. tech careers are notoriously nonlinear, so you might as well embrace it and enjoy the ride!
Richard J. Chen retweeted
Excited to share our latest work, "Mixture of Mini Experts: Overcoming the Linear Layer Bottleneck in Multiple Instance Learning," to appear at ICLR 2026 @iclr_conf This study tackles an overlooked but critical bottleneck in computational pathology: the linear layer that connects foundation models and multiple instance learning.
The Problem: Standard MIL pipelines rely on a familiar three-step process: patch feature extraction, feature transformation with a linear layer, and aggregation for slide-level analysis. While we have made substantial progress on the first and last steps, we identify the intermediate linear layer as a major, hidden bottleneck. Tissue morphology is highly complex, and processing this heterogeneity through a "one-size-fits-all" linear transformation severely limits model performance.
Our Solution: Mammoth 🦣. We replace the single linear layer with a parameter-efficient, low-rank Mixture of Experts (MoE). Mammoth uses a soft routing mechanism to assign each patch to a combination of mini-experts tailored specifically to its phenotype.
Key Highlights:
🔹 Plug-and-Play: Mammoth is a drop-in replacement for the first linear layer in any MIL method (e.g., CLAM, ABMIL, etc).
🔹 Consistent Gains: Evaluated across 8 MIL methods and 19 classification tasks, Mammoth improved performance in 130/152 configurations with an average boost of +3.8%.
🔹 Parameter efficiency: By utilizing factorized (LoRA-style) linear layers, the parameter count stays comparable to a standard linear layer.
🔹 Structured features: The MoE routing yields a highly structured feature space where experts naturally specialize in diverse morphological phenotypes.
Incredible work by Daniel Shao, co-advised with Andrew H. Song and myself, and the rest of the team: Joel Runevic, Richard J. Chen, Drew F. K. Williamson, Ahrong Kim.
Read the full paper and try it out in your own MIL pipelines below 👇
📄 Paper: openreview.net/pdf?id=S5Io33…
🔬 OpenReview: openreview.net/forum?id=S5Io…
💻 Code: github.com/mahmoodlab/mammot…
Richard J. Chen retweeted
📣 Excited to share our new ICML 2025 Spotlight article, “Do Multiple Instance Learning Models Transfer?” – addressing a foundational question for building robust and generalizable MIL models.
Read the article: arxiv.org/pdf/2506.09022
👉Enhanced Performance & Robustness: Pretrained MIL models consistently lead to improved performance even when the pre-training data comes from a different organ, site, disease model than the target task.
👉Aggregation Transfers: Transfer gains come from the MIL aggregation module, not just patch encoders. Resetting attention layers drops performance by 5–8%, showing they encode generalizable pooling logic.
👉Pancancer Generalization: Models pretrained on a more diverse and challenging data (e.g. 108-class pancancer classification task) achieve the stronger overall transfer performance.
👉Robust benefits across patch encoders: Benefits from MIL transfer are consistent across a wide range of patch encoders, from out-of-domain encoders such as ResNet50 pretrained on natural images, to in-domain encoders including Gigapath and UNIv2.
This research highlights supervised pretraining as a highly accessible path to generalizable MIL models, offering a data and compute-efficient route for developing slide level encoders with flexible combination of MIL method and patch encoder.
Congratulations to @Daniel__Shao @GreatAndrew90 @richardjchen and everyone else who contributed.
Stay tuned for an array of pre-trained MIL models ready to transfer to any task! Visit us at @icmlconf.
Richard J. Chen retweeted
👉📣Introducing KRONOS, a panel-agnostic foundation model purpose-built for multiplex spatial proteomics! Access the pre-print here: arxiv.org/pdf/2506.03373
Trained self-supervised on 47 million single-marker patches spanning 175 protein markers, 16 tissue types, 8 imaging platforms and 5 institutions. Its architecture couples a shared channel-wise stem with sinusoidal marker-identity embeddings, making it natively compatible with high-dimensional multiplex data.
🔬 Biology at scale: label-efficient cell phenotyping, unsupervised tissue phenotyping, tissue/region classification, artefact detection, patient stratification, and spatial biomarker discovery.
🔎 Spatial search engine: drop in any cell, patch, or region of interest and KRONOS retrieves morphologically or immunologically similar areas across multi-cohort databases—supporting both exploratory and hypothesis-driven queries.
💻 Code, pretrained models & tutorials: github.com/mahmoodlab/KRONOS
Exciting collaboration with my buddy @SizunJ's lab. Congratulations to @mshaban_ai, Yuzhou Chang and all the other co-authors.
Read the blog post from Sizun: lnkd.in/dwrZpYkP
Read the blog post from Shaban:
lnkd.in/dwBsnX7u
Richard J. Chen retweeted
One year since releasing UNI and CONCH, our computational pathology models have 1.4M+ downloads, have been used by hundreds of studies, and unlocked unique capabilities. Update here: linkedin.com/pulse/one-year-…
🚀 I'm finally excited to share this milestone from @modella_ai ! This FDA breakthrough device designation for PathChat DX is a significant step forward in our mission building #generative and #agentic #AI that empower pathologists and improve patient care. This would not have been possible without the hard work and dedication of our incredible team 🎉 .
⚡ We are just getting started with updates, and also #hiring! If you are a Software Engineer, ML Engineer or Researcher, or Pathologist interested in joining our mission, please reach out to us with your CV at [email protected]!
#gupath #breastpath #neuropath #bstpath #gipath #gynpath #renalpath #clinpath #moldx #renalpath #pulmpath #surgpath #pathology #machinelearning #computervision #softwareengineerjobs #techjobs
This Post is from an account that no longer exists. Learn more
Richard J. Chen retweeted
🚀 Proud to introduce #FastGlioma: the first foundation model enabling rapid, accurate detection of brain tumor infiltration during surgery, in under 10 seconds. With FastGlioma, we’re minimizing the risk of residual tumor and enhancing outcomes for glioma patients. This work sets a new standard in real-time, microscopic-level detection, powered by AI in healthcare. Kudos to the incredible team @mlins, @HerveyJumper, @DanOrringerMD, @InvenioImaging, @CameloPiragua!
Read the full paper in @Nature: nature.com/articles/s41586-0…
Richard J. Chen retweeted
The in-print version of the PathChat @Nature article is now available online with open access nature.com/articles/s41586-0… learn more about next steps and how PathChat is further being developed at @modella_ai
🚀 Thrilled to announce the launch of Judith, our #ai #agent for biomedical image analysis at @modella_ai
⚡Judith brings the power of #genai and foundation models to clinicians, researchers, and scientists in streamlining complex workflows and accelerating discoveries across biomedicine (#pathology , #radiology , #molecular data analysis, and more)!
Check out our demo below, and sign up for our waitlist at modella.ai.
This Post is from an account that no longer exists. Learn more
#Pathology and laboratory #medicine is experiencing a convergence from the advancements in #digitalpathology, #computationalpathology, and #AI.
Published in Nature Reviews Bioengineering 1 year ago, this scoping review provides a fundamental introduction to both historical and modern AI development in pathology, and a now timely perspective into how it will be steered in the years to come with foundation and multimodal models.
If you have heard me give a talk in the past year, my first slide is an animation of Figure 1 which touches on some of the big moments that have transformed the field :^). All credit goes to @GreatAndrew90, @GuillaumeJaume, @AI4Pathology and team for putting this review together!
Article: nature.com/articles/s44222-0…
ReadCube (free): nature.com/articles/s44222-0…
Richard J. Chen retweeted
New research from @katherlab on #computational #pathology: We benchmark pathology foundation models 👇 Several of these are publicly available under permissive licenses.
arxiv.org/abs/2408.15823
Some takeaways:
1. The vision-language model CONCH (@AI4Pathology) outperforms vision-only models in many tasks.
2. Data diversity matters more than volume for improving pathology model performance.
3. Combining multiple models boosts accuracy, outperforming single models in most tasks.
Richard J. Chen retweeted
⚡️🔬📣 We are excited to announce our new #ECCV 2024 @eccvconf paper "Multistain Pretraining for Slide Representation Learning in Pathology" Led by @GuillaumeJaume & @anurag_vaidya7 this work is the latest iteration of our efforts on whole slide representation learning for pathology our lab started with HIPT (CVPR’22) and recently Tangle (CVPR’24). We pretrain new models using cross-stain alignment between H&E slides and widely deployed immunohistochemistry and special stains slides.
📄 Paper: arxiv.org/abs/2408.02859
🔍 Code: github.com/mahmoodlab/MADELE… #ComputationalPathology #DigitalPathology
Richard J. Chen retweeted
⚡️🔬📣 Here are our two latest preprints on how AI for Pathology can advance pre-clinical drug safety and toxicity assessment. Work led by our superstar postdoc @GuillaumeJaume:
Deep Learning-based Modeling for Preclinical Drug Safety Assessment
📄 Preprint: biorxiv.org/content/10.1101/…
🔍 Demo: mahmoodlab.github.io/tox-fou…
🌟 Insights: We trained a Vision Transformer model (TRACE) on H&E-stained whole-slide images from 150+ preclinical toxicity studies. We showed that TRACE can assist and augment pathological assessment for lesion detection, quantification and automatic dose-response characterization. TRACE was also evaluated alongside ten expert pathologists and showed better agreement with the consensus.
AI-driven Discovery of Morphomolecular Signatures in Toxicology
📄 Preprint: biorxiv.org/content/10.1101/…
🔍 Demo: mahmoodlab.github.io/tox-dis…
🌟Insights: We developed GEESE, an AI model trained to predict gene expression of 1,500+ targets from histology. We showed that GEESE can reveal molecular signatures associated with distinct morphologies and toxicity mechanisms that are preserved across multiple compounds and species.
Congrats to Thomas Peeters, Simone de Brot @GreatAndrew90 and everyone involved! Stay tuned for more coming soon..
Richard J. Chen retweeted
Research Highlight: Vision–language AI assistance in human pathology nature.com/articles/s41587-0…
⚡️🔬📣 Excited to share our new @Nature article building and evaluating PathChat, a multimodal generative AI copilot and chatbot for human pathology. Article: nature.com/articles/s41586-0… Open Access Link:
rdcu.be/dKC0r
We leverage our previous success in building foundation models for computational pathology such as UNI / CONCH and combine it with the advancements of large vision language models and generative AI to enable PathChat to answer diverse pathology-related queries. We assessed PathChat using both multiple choice diagnostic questions and open-ended questions.
Congratulations to @MYLu97 @chenbowen118 @DFKW_MD @richardjchen and everyone else who contributed to this work.
Also see blog post from @MYLu97 about this work: linkedin.com/pulse/towards-m… , also teasing the development and preview of PathChat 2, a successor to PathChat 1 bringing new capabilities and substantially improved performance to the state-of-the-art.
Richard J. Chen retweeted
With 350K+ downloads of UNI (nature.com/articles/s41591-0…) and CONCH (nature.com/articles/s41591-0…), it has been really amazing to see how quickly these models are being adopted into CPath research.
Sign up to access models: huggingface.co/MahmoodLab/UN… and huggingface.co/MahmoodLab/CO….
With @richardjchen, here are some key examples for how our models are being used by the community:
