Yuling Gu (@gu_yuling) header
Yuling Gu (@gu_yuling) profile picture

PhD-ing at NYU in NYC 🚕🍎 | Previously @nyuniversity ➡️ @UW ➡️ @allen_ai @yuling_gu@sigmoid.social

Joined September 2019
Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)Photo by Yuling Gu (@gu_yuling)
i gave a keynote talk at NeurIPS'25 just last week. here's the slide deck (link below) i've used to share my thoughts on who we are and what we do.
3
28
245
20,666
🌍 Introducing WorldValuesBench! A benchmark to evaluate how well LLMs reflect cultural differences in human values. Built from 94k+ participants in the World Values Survey → 20M examples of (demographics, value question → answer). 🧵
1
2
5
666
Evaluating language models is tricky, how do we know if our results are real, or due to random chance? We find an answer with two simple metrics: signal, a benchmark’s ability to separate models, and noise, a benchmark’s random variability between training steps 🧵
📢 New paper from Ai2: Signal & Noise asks a simple question—can language model benchmarks detect a true difference in model performance? 🧵
4
54
9
240
47,914
Yuling Gu retweeted
Imagine AI doing science: reading papers, generating ideas, designing and running experiments, analyzing results… How many more discoveries can we reveal? 🧐 Meet CodeScientist, a promising next step toward autonomous scientific discovery. 🧵
6
92
12
361
41,679
Yuling Gu retweeted
kicking off 2025 with our OLMo 2 tech report while payin homage to the sequelest of sequels 🫡 🚗 2 OLMo 2 Furious 🔥 is everythin we learned since OLMo 1, with deep dives into: 🚖 stable pretrain 🚔 lr anneal 🤝 data curricula 🤝 soups 🚘 tulu post-train 🚜 compute infra 👇🧵
3
72
9
358
47,452
Yuling Gu retweeted
Meet Tülu 3 -- a set of state-of-the-art instruct models with fully open data, eval code, and training algorithms. We invented new methods for fine-tuning language models with RL and built upon best practices in the community to scale synthetic instruction and preference data. Demo, GitHub, technical report, and models below 👇
14
133
37
523
219,314
Next week on Wednesday I'll be talking about Open-Instruct and Tulu on the PyTorch youtube channel: youtube.com/watch?v=e1qUJFAo… I'll talk about how open-instruct has changed to fit the needs of the Ai2 post-training team over the past year. Maybe some tulu 3 preview results too :)
13
3
43
14,534
Releasing OLMoE - the first good Mixture-of-Experts LLM that's 100% open-source - 1B active, 7B total params for 5T tokens - Best small LLM & matches more costly ones like Gemma, Llama - Open Model/Data/Code/Logs + lots of analysis & experiments 📜arxiv.org/abs/2409.02060 🧵1/9
23
226
48
921
203,721
Congrats to our team for winning two paper awards at #ACL2024! OLMo won the Best Theme Paper award, and Dolma won a Best Resource Paper award! All the credit goes to the whole team for the massive group effort 🎉🎉
11
42
16
242
53,003
Yuling Gu retweeted
Lots of our teammates are in Thailand this week for #ACL2024, including @gu_yuling who will be presenting work on Digital Socrates, an explanation-critiquing model, tomorrow in a poster session. Learn more about DS here: blog.allenai.org/digital-soc…
2
2
11
2,832
Yuling Gu retweeted
Replying to @soldni
@soldni and I are arriving to #ACL2024 🇹🇭today! come find us at our talks & poster sessions for our OLMo & Dolma projects with @allen_ai & frens 🤩 also dont miss our poster on KIWI🥝for interactive science QA w/ our intern @brunchavecmoi & mentors @eunsolc @davidjwadden
11
3
66
12,661
Yuling Gu retweeted
Great talk from Peter Clark @allen_ai @ai2_aristo #AIED that covers two directions I've been excited about: - Education for AI: improve LLM reasoning w/ human reasoning, focus on *real-world* tasks (science discovery) - AI for education: eg, learning via teaching machines
1
7
27
3,930
Yuling Gu retweeted
🤔What does it mean for a model to have a value? To answer, we first ask, are large language models 🤖 consistent over value-laden questions? 🧵
4
19
101
15,610
Happy to share some small updates to the Tulu suite! We’re releasing ~Llama 3 Tulu 2~ update: llama 3 finetunes + DPO tunes that outperform the original Tulu V2 suite. Reward models included! 🐪find the models here: huggingface.co/collections/h… ⬇️some results below!
2
9
29
2,853
Yuling Gu retweeted
Overwhelmed by the number of model evaluation options?🫨 Use OLMES, a completely documented, practical, open standard for reproducible LLM evaluations, which can lead to more robust & measurable AI progress. Read how the recommendations in OLMES were made: allenai.org/data/olmes
2
14
67
8,155
Very inspiring keynote by Peter Clark from @allen_ai about #knowledgegraphs and #beliefgraphs for uncovering the mental models of #LLMs #eswc2024
6
24
1,561
⁉️ Let's check how GPT-4o, Gemini, Llama3, Mixtral, and Claude perform on theory of mind, shall we?🌟We report new results on Benchmark FANToM👻 - GPT-4o tops the chart by finally achieving score of 2.0/100 (vs. Human 87.5) - Huge boost for Gemini-1.5-flash compared to Gemini-1.0-pro (0.2 -> 1.4) - Llama3 70B is much better than Claude Opus 🔎Check out more at github.com/skywalker023/fant…
LLMs achieve adult human performance on higher-order theory of mind tasks This paper examines the extent to which large language models (LLMs) have developed higher-order theory of mind (ToM); the human ability to reason about multiple mental and emotional states in a
2
15
10
77
35,969
Yuling Gu retweeted
Announcing our latest addition to the OLMo family, OLMo 1.7!🎉Our team's efforts to improve data quality, training procedures and model architecture have led to a leap in performance. See how OLMo 1.7 stacks up against its peers and peek into the technical details on the blog: blog.allenai.org/olmo-1-7-7b…
12
47
14
160
73,118
Yuling Gu retweeted
OLMo is here! And it’s 100% open. It’s a state-of-the-art LLM and we are releasing it with all pre-training data and code. Let’s get to work on understanding the science behind LLMs. Learn more about the framework and how to access it here: blog.allenai.org/olmo-open-l…
26
324
113
1,347
358,573
Yuling Gu retweeted
Such a great first conference! I'm so grateful for the opportunity to present a poster, meet amazing researchers, and learn about tons of interesting work at EMNLP 2023.
3
33
2,283