
Ph.D student at @Princeton | Formerly @JohnsHopkins ’25, @HopkinsDSAI @JHUCompSci @jhuclsp @CCVLatJHU | AI/ML/NLP/CV
Princeton, NJ
Joined June 2024
- Tweets28
- Following667
- Followers402
- Likes48
On 30 benchmarks, Strategist beats the strongest hand-written baseline on 24, with the same average accuracy at 39% lower cost.
On 10 new tasks, the library alone, with no search, is more accurate than the baseline on 8.
2
136
We will present the paper at NeurIPS this December!
Paper: arxiv.org/abs/2611.03618
Try it on your own tasks? We’d love to see what it finds.
5
141
Taiming Lu retweeted
You want a strong small LLM. Would you start small — or inherit from something bigger?
📄 New paper: Small LLMs: Pruning vs. Training from Scratch
We find that pruning is more than a better initialization: simply giving randomly initialized LLMs more training tokens is often not enough to catch up.
Taiming Lu retweeted
Can a small academic team build a strong text-to-image model using only public datasets?
Introducing i1: a simple, fully open recipe for strong text-to-image models
Knowledge doesn't always flow downhill.
We find that in LLM pretraining, a weaker teacher can improve a stronger student, and pushing the teacher further can actually hurt.
New paper: Strong Teacher Not Needed? On Distillation in LLM Pretraining.
Distillation improves generalization more readily than in-domain fit.
Out-of-distribution perplexity and downstream accuracy improve more consistently than in-domain perplexity, where some configurations help OOD/downstream while doing nothing for in-domain.
Teacher–student compatibility matters more than raw teacher strength.
This changes how you pick a teacher: both for frontier training (where the best available teacher is often a prior generation) and for efficient small models, where "bigger teacher is better" isn't the right rule.
Thanks @liuzhuang1234 for the support!
arxiv: arxiv.org/abs/2605.23857
code: github.com/zlab-princeton/st…
Taiming Lu retweeted
Stronger Normalization-Free Transformers – new paper.
We introduce Derf (Dynamic erf), a simple point-wise layer that lets norm-free Transformers not only work, but actually outperform their normalized counterparts.
Taiming Lu retweeted
🤯 Think better visuals mean better world models? Think again.
💥 Surprise: Agents don’t need eye candy— they need wins.
Meet World-in-World, the first open benchmark that ranks world models by closed-loop task success, not pixels.
We uncover 3 shocks:
1️⃣ Visuals ≠ utility
2️⃣ Action data > bigger models
3️⃣ Scaling test-time compute = more success
🤗 huggingface.co/papers/2510.1…
🌍 world-in-world.github.io
📄 arxiv.org/abs/2510.18135
github.com/World-In-World/wo…
Taiming Lu retweeted
Excited to share our lab’s first open-source release: LLM-Distillation-JAX
supports practical knowledge distillation configurations (distillation strength, temperature, top-k/top-p), built on MaxText
designed for reproducible JAX/Flax training on both TPUs and GPUs
Taiming Lu retweeted
Meet the AI system that can envision an entire world from a single picture. @genex_world—developed by @jieneng_chen, @YuilleAlan, @TaiMingLu, @DanielKhashabi, & @tianminshu—imagines in-depth scenarios to make informed decisions. Learn more here: hub.jhu.edu/2024/12/19/a-gen…
Taiming Lu retweeted
Thrilled to introduce GenEx: Generating an Explorable World. ✨
✨ GenEx takes a single image 🖼️ and create a 3D generative world 🌍 — you can dive in for interactive exploration, and so as embodied AI agent.
Follow our X for more demos: teapawt.org/genex_world
Paper on huggingface: huggingface.co/papers/2412.0…
Tech details: genex.world
(1/n)













