Founder @stanfordnlp & cs224n—Senior Fellow @StanfordHAI—Prof. CS & Linguistics @Stanford—GP @aixventureshq—MTS @moonlake—Australian🇦🇺—Do #NLProc & #AI 👋
Palo Alto
Joined September 2014
- Tweets3.1K
- Following399
- Followers172K
- Likes1
Christopher Manning retweeted
What happens if you put Tversky similarity inside an RNN cell?
I added a Tversky branch to mLSTM in Vision-LSTM and xLSTM. At equal params, half the depth matched or beat full depth, with ~20% fewer FLOPs and up to 2x faster training.
Read the Article: arian32.medium.com/tversky-s…
Christopher Manning retweeted
Synthetic data is used all the time now for LM pretraining...but can we carefully quantify the gains in compute efficiency and data efficiency?
Great that @cityofpaloalto is building some new housing – but extremely little of it near the University Ave downtown transit and shopping hub, where it would be so useful 😿🌇
paloalto.gov/City-Hall/Housi…
Christopher Manning retweeted
Congratulations to @aixventureshq on a great event. You are all so impressive… and it’s just the beginning!
@RichardSocher @chrmanning @ellie Krish, Nick, Sebastian, Sara
Christopher Manning retweeted
Meet Pika H3 Flex, debuting today as the #1 open weight model on DesignArena’s t2v board.
It’s built for professional iteration, with exportable audio stems, resolutions up to 4k, and faster-than-realtime generation. It can produce a 15-second clip with soundtrack in as little as 7 seconds.
So, yeah. It’s kind of a flex 💅
Christopher Manning retweeted
Introducing Fern, Mirendil’s first model: pretrained from scratch, with up to 10M tokens of context!
That’s ~0.5M lines of code (roughly Megatron-LM scale) or days of an agent’s work, all in context at once.
Its first version, Fern 0.1, had a short round of post-training 🚢
Christopher Manning retweeted
Shoutout to Christopher Manning (@chrmanning)! He founded the Stanford NLP Group, co-introduced GloVe word vectors, and taught CS224N nearly every year from 2000 to 2024. So many people learned NLP from his work. 🙌
aipeoplewiki.com/people/chri…
#NLP #Stanford #AI
Christopher Manning retweeted
Introducing Kimi K3.1 Pro: our most capable 3T-class open-weight model.
🔹 1M-token context, 2.8T total / 104B active params (MoE), Multimodal.
🔹 Full-project coding: games in Unreal, Unity & Godot.
🔹 Excels at parametric 3D modeling in Blender & CAD.
🔹 More reliable at finance work: Accurate modeling with hallucination down by 65%.
🔹 Matching the strongest proprietary models across coding, agentic and knowledge benchmarks.
Kimi K3.1 Pro and K3.1 Pro Highspeed (2–3× faster) are now live on Kimi.ai, Kimi Work, Kimi Code, and the Kimi API.
Open weights by Nov 4, 2026.
🔗 API: platform.kimi.ai
🔗 Tech blog: kimi.ai/blog/kimi-k3-1-pro
Christopher Manning retweeted
I’m still working through it, but this work is incredibly cool and (unless there’s some kind of serious mistake) the lineage of work including this and “Why Larger Models Learn More” is exactly the kind of work that I think is necessary to develop a science of DL. I’m a huge fan
The wonderful “Don’t just Fix it in Post” paper identifies predicting memorization as a key open problem.
A progress report: we can now accurately predict verbatim memorization using just three well-understood variables. Experiments by Jing Huang, using the Hubble model suite:
ALT Dot plot addressing whether memorized sequences can be predicted without training the full model. Headline: “Largely yes, with Frequency and Recency doing most of the work.” Four rows show AUC-ROC (0.5 = chance, 1.0 = perfect) for a linear predictor of verbatim extraction, one mark per Hubble model (1B or 8B params; 100B or 500B tokens). All three features: 0.82–0.94. Frequency alone: 0.81–0.93. Recency alone: 0.82–0.90. Hardness alone: 0.51–0.55, barely above chance. In every row the 1B/500B model scores lowest and the 1B/100B model highest. Footnote: held-out strings, mean of 5 seeds; Frequency = copies of the string, Recency = steps since last copy, Hardness = NLL on models that never saw it.
Impressive performance, just a little below @thinkymachines’s Inkling with an over 10x smaller model.
But I think I’m still going to wait a little before buying my Mac Studio….
I reacted with excitement but also skepticism to @bneyshabur’s plan for making frontier-performance LM pre-training possible by a single researcher.
Could 1 person compete with the huge teams at big co’s?
$10M is generous for an academic budget, but @mirendil has delivered!!!
Today, we're sharing a major milestone in our mission to democratize frontier AI development:
Pretraining at scale without a pretraining team.
We set out to answer an ambitious question: Can we significantly reduce the time, human expertise, and compute needed to pretrain a competitive foundation model from scratch?
The result is Mirendil-Base, our first pretrained coding model, built by one researcher in just 3.5 months with less than $10M in compute, including all research, experimentation, and the final training run.
Our approach is already showing promising results as we scale to even larger models.
This is an important step toward a future where a single researcher, empowered by AI, can accomplish what once required an entire research organization. As these capabilities improve, we believe frontier AI development can become accessible to far more people and teams.
More to come soon, including independent evaluations, post-training benchmarks, and API access.
Christopher Manning retweeted
Best decision models in the world, free, and open-source, with all the details of how we trained it at @baselabs. Enjoy!
Today we’re releasing two decision models, elephant-1 (9B) and mammoth-1 (27B), trained by the @baselabs team, available for free for one week on our Model APIs.
mammoth-1 is #1 on the Jev Decision Index and elephant-1 is tied for #1 on the Artificial Analysis Decision Index.
It’s a new model, but more importantly, this is an open-source model you can access, deploy, and post-train on Baseten (or wherever you do your best work).
The free week ends 10/27/26 at 11:59 PM PT. Try it and let us know what you think.
Try mammoth-1 here: baseten.co/library/mammoth-1…
Try elephant-1 here:
baseten.co/library/elephant-…
Thank you!!! But I certainly can’t take credit for the whole group. So much of it due to all the students and the other faculty:
@jurafsky @percyliang @ChrisGPotts @YejinChoinka @MonicaSLam @Diyi_Yang @tatsu_hashimoto
which AI labs turn into startups?
counted lab size vs startups founded by members, public sources only
stanford hazy: 7 startups from 13 people
stanford NLP: 12
berkeley sky lab: 7
@chrmanning your group is the only big lab up top
Christopher Manning retweeted
Beam is now available across platforms, with open weights under the Apache 2.0 license - in collaboration with an ecosystem of 40+ partners.
We're releasing:
- Weights, tokenizer, and reference implementation
- BF16, FP8 and NVFP4 weights and a DFlash2 drafter
- The open source harness we built and trained Beam on: Mirror CLI
Weights and model card on Hugging Face: huggingface.co/reflection
Christopher Manning retweeted
We developed a model reaching frontier-level multimodal performance, matching or exceeding Astra on 5 of 12 benchmarks, all with 17x lower token prices. The model is smaller and cheaper than Gemini 3.8 Flash while dominating it in real world vision workloads.
We achieved this by inventing a technique for visual reasoning, where the model is no longer restricted to text chain-of-thought. Conditioned on visual tokens, Glimpse can perform recurrent computation over latent internal representations and can even orchestrate subagents that send messages over internal visual states.
Really proud of what our small team and only hundreds of GPUs could accomplish.
elorian.ai/blog/introducing-…
Christopher Manning retweeted
Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.
Christopher Manning retweeted
Mike was my colleague at Berkeley, and he's been fairly skeptical of deep learning's trajectory for years. Most scientists are skeptics, and it can take a big shock to get someone to consider a possibility that goes against the anchoring effect coded in our DNA.
I too was somewhat of a skeptic until ~6 months ago, and people ask me all the time why I joined Anthropic.
It started in April, when a friend I've known for 20 yrs messaged me saying he'd be in the Bay Area in a week, and he suggested we meet up. I invited him to my house for dinner, and he told me he had just joined Anthropic. He said the models were becoming increasingly more capable, and he believed both that they'd soon be smarter than him and also that all intellectual work would be highly devalued in the near future, and lastly that society was not at all ready for what's coming.
I'd of course heard claims similar to this before, but not from a friend I'd known for this long, and who I trusted, at my dining room table. It was the initial shock needed to destabilize my own anchoring. Also, I've been fortunate to work with and/or mentor many smart people in my life (e.g. Fields medalists, a 4x Putnam Fellow, IOI first place winners, etc.), so I feel pretty calibrated in recognizing very smart people, and this was someone who is definitely very smart and with impressive such credentials. So, it was hard to ignore this friend saying the models would be smarter than even him in particular.
After that dinner, I decided to do an informal survey, calling and meeting up with a lot of my friends who now work on improving model capabilities at these frontier labs. They all agreed with that first friend. These were all people I'd known 10, 15, 20 years, who mentored me, or I mentored them, or we met professionally in academia. These are people I trust, and their beliefs on this topic were consistent.
I then spent a whole lot of time understanding the missions of different companies, and talking to current and former employees and people in leadership. I read Dario's (long) essays and watched videos on YouTube of company leaders being interviewed. Anthropic's stated mission "to ensure that the world safely makes the transition through transformative AI" felt like exactly the right guiding principle to handle such a society-disrupting technology, and everyone I spoke to at the company was very aligned with this mission. It's easily the most mission-focused place I've ever been at, and everyone takes the work extremely seriously and with a profound sense of responsibility.
And now here I am. And being on the inside, the pace of progress has surprised me more than I first imagined. The moment that pure mathematicians are in, to me, is just a microcosm of what I predict a much larger of society will be grappling with in the not too distant future, and I really hope our leaders and the general public internalize sooner than later just how significant a moment we're in.
Michael Jordan (the machine learning one): "Superintelligence is Science Fiction" 🔥🔥🔥
di.ens.fr/~fbach/MJordan_ITW…
It sure looks like LLMs’ models of the world and visual “world models” aren’t as different as some people think!
Christopher Manning retweeted
Sherpa: Teaching LLMs to Teach Adaptively
Weixian Xu, Yanzhe Zhang, Zora Zhiruo Wang, Changyu Chen, Diyi Yang
arxiv.org/abs/2610.08778 [𝚌𝚜.𝙰𝙸 𝚌𝚜.𝙲𝙻]
💬Code: github.com/SALT-NLP/Sherpa
ALT Large language models (LLMs) have become increasingly capable problem solvers, but being able to solve a problem is not the same as being able to teach it. Existing approaches to training LLMs as teachers rely on demonstrations, preference data, or predefined pedagogical criteria that specify what good teaching looks like. However, these signals are often not grounded in individual student learning outcomes, where effective teaching strategies can vary substantially across learners. To address this, we introduce Sherpa, a multi-turn reinforcement learning framework that instantiates multiple student archetypes with LLMs conditioned on distinct learning preferences and trains a teacher model to adapt its instruction by directly maximizing their learning outcomes. Teacher LLMs trained with Sherpa improve instructed students' performance across all archetypes by an average of 20.5 percentage points. Under MathTutorBench's evaluation, Sherpa raises the overall pedagogy score from 52.5% to
Christopher Manning retweeted
We’re sharing new results showing that frontier AI models are able to outperform human experts on financial predictions for the first time. We built an environment for the challenging task of Earnings Predictions, finding that the most recent frontier models (Opus 5.5, Fable 5.1, Astra) with Samaya’s harness, meaningfully outperform expert consensus predictions on commonly reported earnings metrics such as company revenue, gross margin and more.
We built this environment to have a “point-in-time” gate in the harness to ensure no information leakage, and created financial retrieval and data tools that would respect this gate and give the models access to the same information available to experts. We also adjusted for bias in expert consensus to create a hard, bias-corrected baseline, where outperformance has real signal. Only the most recent frontier AI models are able to beat this hard baseline, pointing to an important inflection point in AI capabilities in finance.
Like with FrontierFinance, we see substantial headroom for further RL training in this environment — the models show large variations in error across different rollouts. We’re continuing research on using earnings prediction and other predictive tasks to post-train models to learn from their mistakes. Get in touch if you’d like to collaborate or try out an alpha version. Link to the full research below.