CS PhD student at Stanford
Joined December 2022
- Tweets12
- Following159
- Followers22
- Likes20
A great method that extracts more signal about model capabilities at a low cost!
Can we forecast benchmark accuracy from models that score zero?
We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
Elise L. Han retweeted
Can we forecast benchmark accuracy from models that score zero?
We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
So cool!!
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Elise L. Han retweeted
We introduce 3D World Simulator, an interactive world model that runs at 50+ FPS on an RTX 5090.
Our model predicts in a unified 3D representation space, capable of generating:
• Novel view appearance
• Multi-view consistent RGB-D
• 3D geometry
• Spatially aligned tactile signals
Play with the world model now at jwang771.github.io/3dworldsi…!
Elise L. Han retweeted
I improved NanoGPT Speedrun from 39.9s → 6.7s (-83.2%), more than every record since 2025 combined (-81.4%)!
The web is full of rare, repeated content that the LM can’t memorize in time. I solve this with retrieval-augmented n-gram hints🧵
Elise L. Han retweeted
Introducing the world's fastest tokenizer implementation, Gigatoken!
Gigatoken is ~500-1000x faster than HuggingFace, and ~100x faster than OpenAI's tiktoken for most tokenizer definitions on most machines.
These baselines are already multithreaded Rust implementations! 🧵
Elise L. Han retweeted
Often in research level math, you can't verify a proof with Lean because the formal libraries needed don't exist yet.
Our fix: Pseudo-Formalization (PF), a proof verification method with the modularity and precision of formal proofs, but the flexibility of natural language.
Elise L. Han retweeted
Today, agents execute isolated tasks. Tomorrow, agents will steer complex decisions across long horizons.
Introducing CEO-Bench, a first step to measure "Steering Intelligence." In CEO-Bench, agents are asked to run a simulated startup for 500 days.
ceobench.com
Elise L. Han retweeted
How far can we compress the discrete tokens in an LLM's context into compact latent vectors?
With the right training recipe at large scale, our Latent Context Language Models (LCLMs) compress context up to 16× and land on a new Pareto frontier for long-context inference. 🧵(1/n)
Elise L. Han retweeted
Some new results I found surprising that I’m tweeting for Chris (who isnt on here). With enough compute, the best data filter for LMs (on DCLM) might be no filter. Why? Large models can tolerate a surprising amount of nominally 'low quality' data, and can sometimes even benefit.
Elise L. Han retweeted
Self-play led to superhuman Go performance, why hasn’t it for LLMs?
In practice, long run self-play plateaus like RL. We study why this happens, and build a self-play algorithm that scales better. It solves as many problems with a 7B model as the pass@4 of a model 100x bigger.
Elise L. Han retweeted
New paper!
Want to precisely optimize synthetic training data to do practical or even wacky things?
Dataset Policy Gradients get you there, letting you target any differentiable training or post-training metric. We embedded a QR code in GPT-2’s weights using only training data!
