
PhD-ing at NYU in NYC 🚕🍎 | Previously @nyuniversity ➡️ @UW ➡️ @allen_ai @yuling_gu@sigmoid.social
Joined September 2019
- Tweets142
- Following829
- Followers897
- Likes1.3K
Yuling Gu retweeted
i gave a keynote talk at NeurIPS'25 just last week. here's the slide deck (link below) i've used to share my thoughts on who we are and what we do.
Yuling Gu retweeted
🌍 Introducing WorldValuesBench!
A benchmark to evaluate how well LLMs reflect cultural differences in human values.
Built from 94k+ participants in the World Values Survey → 20M examples of (demographics, value question → answer).
🧵
Yuling Gu retweeted
Evaluating language models is tricky, how do we know if our results are real, or due to random chance?
We find an answer with two simple metrics: signal, a benchmark’s ability to separate models, and noise, a benchmark’s random variability between training steps 🧵
Yuling Gu retweeted
Imagine AI doing science: reading papers, generating ideas, designing and running experiments, analyzing results… How many more discoveries can we reveal? 🧐
Meet CodeScientist, a promising next step toward autonomous scientific discovery. 🧵
Yuling Gu retweeted
kicking off 2025 with our OLMo 2 tech report while payin homage to the sequelest of sequels 🫡
🚗 2 OLMo 2 Furious 🔥 is everythin we learned since OLMo 1, with deep dives into:
🚖 stable pretrain
🚔 lr anneal 🤝 data curricula 🤝 soups
🚘 tulu post-train
🚜 compute infra
👇🧵
Yuling Gu retweeted
Meet Tülu 3 -- a set of state-of-the-art instruct models with fully open data, eval code, and training algorithms.
We invented new methods for fine-tuning language models with RL and built upon best practices in the community to scale synthetic instruction and preference data.
Demo, GitHub, technical report, and models below 👇
Yuling Gu retweeted
Next week on Wednesday I'll be talking about Open-Instruct and Tulu on the PyTorch youtube channel: youtube.com/watch?v=e1qUJFAo…
I'll talk about how open-instruct has changed to fit the needs of the Ai2 post-training team over the past year. Maybe some tulu 3 preview results too :)
Yuling Gu retweeted
Releasing OLMoE - the first good Mixture-of-Experts LLM that's 100% open-source
- 1B active, 7B total params for 5T tokens
- Best small LLM & matches more costly ones like Gemma, Llama
- Open Model/Data/Code/Logs + lots of analysis & experiments
📜arxiv.org/abs/2409.02060
🧵1/9
Yuling Gu retweeted
Congrats to our team for winning two paper awards at #ACL2024!
OLMo won the Best Theme Paper award, and Dolma won a Best Resource Paper award!
All the credit goes to the whole team for the massive group effort 🎉🎉
Yuling Gu retweeted
Lots of our teammates are in Thailand this week for #ACL2024, including @gu_yuling who will be presenting work on Digital Socrates, an explanation-critiquing model, tomorrow in a poster session. Learn more about DS here: blog.allenai.org/digital-soc…
Yuling Gu retweeted
Replying to @soldni
@soldni and I are arriving to #ACL2024 🇹🇭today!
come find us at our talks & poster sessions for our OLMo & Dolma projects with @allen_ai & frens 🤩
also dont miss our poster on KIWI🥝for interactive science QA w/ our intern @brunchavecmoi & mentors @eunsolc @davidjwadden
Yuling Gu retweeted
Great talk from Peter Clark @allen_ai @ai2_aristo #AIED that covers two directions I've been excited about:
- Education for AI: improve LLM reasoning w/ human reasoning, focus on *real-world* tasks (science discovery)
- AI for education: eg, learning via teaching machines
Yuling Gu retweeted
🤔What does it mean for a model to have a value?
To answer, we first ask, are large language models 🤖 consistent over value-laden questions? 🧵
Yuling Gu retweeted
Happy to share some small updates to the Tulu suite!
We’re releasing ~Llama 3 Tulu 2~ update: llama 3 finetunes + DPO tunes that outperform the original Tulu V2 suite. Reward models included!
🐪find the models here: huggingface.co/collections/h…
⬇️some results below!
Yuling Gu retweeted
Overwhelmed by the number of model evaluation options?🫨 Use OLMES, a completely documented, practical, open standard for reproducible LLM evaluations, which can lead to more robust & measurable AI progress. Read how the recommendations in OLMES were made: allenai.org/data/olmes
Yuling Gu retweeted
Very inspiring keynote by Peter Clark from @allen_ai about #knowledgegraphs and #beliefgraphs for uncovering the mental models of #LLMs #eswc2024
Yuling Gu retweeted
⁉️ Let's check how GPT-4o, Gemini, Llama3, Mixtral, and Claude perform on theory of mind, shall we?🌟We report new results on Benchmark FANToM👻
- GPT-4o tops the chart by finally achieving score of 2.0/100 (vs. Human 87.5)
- Huge boost for Gemini-1.5-flash compared to Gemini-1.0-pro (0.2 -> 1.4)
- Llama3 70B is much better than Claude Opus
🔎Check out more at github.com/skywalker023/fant…
Yuling Gu retweeted
Announcing our latest addition to the OLMo family, OLMo 1.7!🎉Our team's efforts to improve data quality, training procedures and model architecture have led to a leap in performance. See how OLMo 1.7 stacks up against its peers and peek into the technical details on the blog: blog.allenai.org/olmo-1-7-7b…
Yuling Gu retweeted
OLMo is here! And it’s 100% open.
It’s a state-of-the-art LLM and we are releasing it with all pre-training data and code. Let’s get to work on understanding the science behind LLMs. Learn more about the framework and how to access it here:
blog.allenai.org/olmo-open-l…










































