PhD @YaleNLP | undergrad @UWcse @nlpnoah | interned at @kotoba_tech
Joined November 2021
- Tweets66
- Following333
- Followers118
- Likes183
Pinned Tweet
Thank you @kotoba_tech and special thanks to @jungokasai and @noriyuki_kojima! Wonderful and rewarding experience in Tokyo for the summer, surrounded by such a passionate team of talented engineers. Always excited about Kotoba's next release and look forward to keeping in touch!
Kotoba's former intern, Alan Li (@alanli2020), is starting his CS PhD at @Yale. Best of luck on your PhD journey, and we'll stay in touch!
Alan Li retweeted
I wrote a blog post detailing the writing approach I took while working on my recent paper: sophielwang.com/blog/paper-w…
I describe the behind the scenes for how this paper came to be, why and how I storyboarded figures before running experiments, and some figure design tips! :)
Can “chicken” make a base model reason? 🐔
Yes!
Why? With the right first tokens, base models can match RL in reasoning. We trace this to learned training data associations and use a simple data edit to make “chicken” elicit reasoning too.
blog: sophielwang.com/cues
Alan Li retweeted
🔥 Excited to share my new preprint: Can LLMs Use Linguistic Uncertainty Markers to Reliably Reflect Intrinsic Confidence? 🔥
When an LLM says "I think" or "probably," does it actually mean something consistent internally? The answer: not really 😬
Check out details in 🧵(1/n):
Alan Li retweeted
Check out our #ICLR2026 paper on how strong references can especially help improve LLMs on non-verifiable tasks.
New ICLR 2026 paper: References Improve LLM Alignment In Non-verifiable Domains
RL with verifiable rewards works well when correctness is checkable, but alignment tasks usually are not verifiable.
We ask: can reference-guided LLM evaluators act as soft verifiers for alignment? Yes! This also helps frontier judges when high-quality human references are available.
Alan Li retweeted
I will be attending #EMNLP2025 this week to present LiteASR, a compression method for speech encoders (a collaborative work with @kotoba_tech).
Catch our poster at the first poster session on Wednesday morning. Happy to chat about efficiency, speech, or both!
🚀 Presenting LiteASR: a method that halves the compute cost of speech encoders by 2x, leveraging low-rank approximation of activations.
LiteASR is accepted to #EMNLP2025 (main) @emnlpmeeting
Alan Li retweeted
As PhD students, we believe research automation systems should belong to everyone, not just Google, so we built freephdlabor.
Customize your multi-agent system for end-to-end research that WORKS FOR YOUR DOMAIN within hours.
full source code: github.com/ltjed/freephdlabo…
Alan Li retweeted
If you are at #ICCV2025 - the Knowledge-Intensive Multimodal Reasoning Workshop is about to start, in Room 313 C !
If you are at #ICCV2025, join us today for the multimodal reasoning workshop!
We have an amazing lineup of speakers and an exciting panel on the future of multimodal reasoning!
Love the thread, thank you Rohan!
New Harvard+Yale paper says, strong reasoning helps, but accessing the right knowledge first is what really limits performance.
So knowledge recall is the main bottleneck in scientific problem solving with LLMs.
They build benchmark suites SCIREAS and SCIREAS‑PRO to measure scientific reasoning end to end
They also release a simple 8B baseline for science tasks that benefits from a math+STEM mix.
🎯 The problem
Scientific questions need 2 things at once, solid domain knowledge and multi‑step reasoning, but most tests only hit one side or lock into one format.
There was no unified way to score science reasoning across domains, and almost no clean way to tell whether a model failed because it lacked a fact or because it could not reason with the fact.
1/9 🚀 New paper: Demystifying Scientific Problem-Solving in LLMs — How does reasoning enhancement affect knowledge recall, and do LLMs benefit from external knowledge complimentary to reasoning?
Tldr;
📊 SciReas: holistic and efficient evaluation suite for scientific reasoning
🧠 KRUX: a novel framework to study knowledge vs reasoning in LLMs
🔑 Findings: knowledge is a bottleneck; reasoners + in-context knowledge help; long CoT helps knowledge recall/utilization
8/9 This work is a collaboration between YaleNLP @yalenlp and Ai2 @allen_ai .
Code/benchmark 📈 github.com/yale-nlp/SciReas-….
Paper: 📄 arxiv.org/abs/2508.19202
Models: 🤗 huggingface.co/lihaoxin2020/…
Update: It’s happening at 2pm!
Exciting journey, Come and join us!
Today at 4 PM, we’re presenting our tutorial:
“Evaluating LLM-based Agents: Foundations, Best Practices, & Open Challenges”
If you’re in Montreal for @IJCAIconf, come join us to dive into the future of #AgentEvaluation! 🇨🇦🤖
w. @RoyBarHaim @LilachEdel and @alanli2020
Alan Li retweeted
Excited for the release of SciArena with @allen_ai!
LLMs are now an integral part of research workflows, and SciArena helps measure progress on scientific literature tasks.
Also checkout the preprint for a lot more results/analyses. Led by: @YilunZhao_NLP, @kaiyan_z
📄 paper: sciarena.allen.ai/SciArena_A…
This was a massive team effort and we're thrilled to finally share it!
Alan Li retweeted
Excited to see more investigation into LLM creativity. We have some pioneering work on this topic as well:
Creativity or Brute Force?
Using Brainteasers as a Window into the
Problem-Solving Abilities of Large Language Models. arxiv.org/pdf/2505.10844.
