
Ph.D. student @mldcmu & visiting student @stanford. Previously undergrad @berkeley_ai
Joined February 2021
- Tweets131
- Following511
- Followers1.5K
- Likes388
When LLMs donโt do what we want, we often tell them exactly what/how to change. Ideally, models could learn from this feedback, which is much richer and denser than scalar rewards used for RL. In our new paper, we study how to expand the capabilities of RL via text feedback:
At #COLM2026 Tue-Sat.
I will present our harness learning paper at the LLA workshop on Friday ( @ZAlvin39105 will deliver the oral).
I'd love to chat about making RL more general and efficient (or anything about RL!), and meta-topics like how to proceed with research at school.
Lili retweeted
Images, video, audio, actions โ generative modeling has converged on one recipe: compress into continuous latents, generate in latent space, decode.
Language is the lone exception: still generated token by token, as a long discrete stream.
Latent Thought Flows: We compress 256 text tokens into 8 continuous latents, generate them with a one-step flow model, and read them out as text using an autoregressive decoder.
Result: a better inference-compute vs. generation-quality Pareto frontier than a tuned autoregressive baseline โ thanks to compression and one-step generation.
Co-led w/ @ZhengyangGeng ๐งต
Lili retweeted
Excited to announce our new work, Message Passing Enables Efficient Reasoning accepted at #COLM2026!
Modern sub-agents are designed with fork-join style parallelism, where sub-agents are spawned for individual tasks and then their results are collated by a main thread. ๐งต1/n
Lili retweeted
A lot of excitement in synthetic data for science is motivated by ambitious uses of LLMs (e.g., running a survey with GPT before real participants, forecasting promising experiments with LLMs). We ask, what beliefs should the experimenter have to make this a good idea? ๐งต
Lili retweeted
๐ญ ๐๐
๐ฝ๐น๐ผ๐ฟ๐ฎ๐๐ถ๐ผ๐ป in the Era of ๐๐ฒ๐ป๐ฒ๐ฟ๐ฎ๐๐ถ๐๐ฒ ๐๐ผ๐ป๐๐ฟ๐ผ๐น ๐ค
Interacting with the world can be expensive!
Our #ICML2026 work shows how diffusion policies can ๐๐ญ๐ฅ๐ก๐ค๐ง๐ during online experience collection to achieve sample-efficient self-improvement! ๐
In our paper, we also explain why this seemingly simple loss should work. Check out our work if you are interested: rl-textfeedback.github.io
Lili retweeted
Proteo-R1 (ICML 2026), the first reasoning protein foundation model for protein design, is out! ๐๐งฌ
Most protein design models generate structures without ever *reasoning* about which residues matter. We think that's backwards.
Human protein engineers๐ฉโ๐ง don't work this way. They identify critical interaction residues first โ charged anchors, hydrophobic hotspots, specificity-determining motifs โ and only then optimize geometry around those decisions.
โโโโโโโโโโโโโโโโ
๐ฌ THE CORE IDEA
โโโโโโโโโโโโโโโโ
A dual-expert architecture that explicitly decouples molecular understanding from geometric generation:
โ โกA multimodal LLM (understanding expert) analyzes protein sequences, structures, and text to identify key functional residues governing binding and specificity
โ โกA diffusion model (generation expert) then co-designs sequence + structure โ but with those residues locked in as hard constraints
โโโโโโโโโโโโโโโโ
๐ HOW IT'S TRAINED
โโโโโโโโโโโโโโโโ
Three-stage curriculum:
โ Multimodal Alignment โ freeze the LLM, train projections to bridge ESM-2 + AF3-style structural features into language space
โก Structural Reasoning Mid-Training โ unfreeze the LLM, teach it residue grounding โ pairwise geometry โ interface localization โ hotspot prediction
โข Joint Reasoning-Guided Design โ end-to-end on antibody-antigen complexes. Gradients from the diffusion objective flow back through the reasoning expert.
โโโโโโโโโโโโโโโโ
๐ RESULTS
โโโโโโโโโโโโโโโโ
Evaluated on simultaneous multi-CDR redesign and the RAbD CDR-H3 benchmark:
โ
Best RMSD & DockQ on RAbD โ redesigned H3 loops are geometrically accurate *and* docked well
โ
Lowest backbone dihedral divergence (JSDbb) among all baselines
โ
Reduced intra- and inter-chain steric clashes
โ
Generated sequences score lower perplexity than native antibodies under IgLM & AbLang
โ
Plug-and-play: swapping the diffusion backend to UniMoMo still improves RMSD and IMP
โโโโโโโโโโโโโโโโ
๐ก WHY IT MATTERS
โโโโโโโโโโโโโโโโ
Proteo-R1 isn't just a better antibody design model. It's a blueprint for coupling deliberative LLM reasoning with any physical generative process โ interpretable, modular, and backend-agnostic.
๐ Paper: arxiv.org/abs/2605.02937
๐ป Code: github.com/smiles724/Proteo-โฆ
๐ Demo: smiles724.github.io/r1/
Great thanks to my wonderful collaborators Weihao Xuan, Heli Qi, @Hanqun_CAO, Heng-Jui Chang, @KKuanPang @XiangruTang Zehong Wang, @hcwww_ , @KejunYing @lupantech Chiho Im, Seungju Han, @richardxp888 @tikgiau. Also appreciate the guidance from advisors @YejinChoinka @jure @erranlli Naoto Yokoya, Masashi Sugiyama.
Lili retweeted
Sharing a super simple, user-owned memory module we've been playing around: nanomem
The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind.
For example:
- `nanomem add <fact>` starts an agent loop to walk the tree, read relevant files, and edit.
- `nanomem retrieve <query>` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query.
Whatโs nice about this approach is that the memory system is, by construction:
1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance)
2. portable and user-owned (itโs just text files)
3. interpretable (you know exactly whatโs written and you can manually edit)
4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better)
5. modularized (you can optimize ingestion/retrieval/compaction prompts separately)
Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & โneed-to-knowโ) and utility (as too much context for a query can harm answer quality).
Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below).
We built nanomem as part of the Open Anonymity project (openanoymity.ai), but itโs meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges!
- GitHub (MIT): github.com/OpenAnonymity/nanโฆ
- Blog: openanonymity.ai/blog/nanomeโฆ
- Beta implementation in chat client soon: chat.openanonymity.ai
Work done with amazing project co-leads @amelia_kuang @cocozxu @erikchi !!
Lili retweeted
What if AI learned physics the way Newton did โ by experiencing it?
We built Sim2Reason: train LLMs inside virtual worlds governed by real physics laws, zero human annotation.
Result: +5โ10% improvement on International Physics Olympiad, zero-shot. ๐งต
Lili retweeted
for data-constrained pre-training, synth data isnโt just benchmaxxing, it lowers loss on the real data distribution as we generate more tokens
for even better scaling, treat synth gens as forming one long ๐บ๐ฒ๐ด๐ฎ๐ฑ๐ผ๐ฐ: 1.8x data efficiency with larger gains under more compute
Lili retweeted
"Expanding the Capabilities of RL via Text Feedback (RLTF)"
RL for LLMs has a wall that it struggles to get past.
As a single scalar reward is too low-bandwidth to tell the model what was wrong, learning would stall on harder but useful tasks. LLM needs more specific guidance.
This paper proposes RLTF, where you can replace RLHF's scalar reward with rich text critiques during training, distills those critiques into better first-try answers.
So by letting the model see a critique and retry during training, then training it to bake those fixes into its first response, basically learn to spot the mistake it would be critiqued for, RLTF turns this feedback into a skill, especially on tasks where a single reward number can't explain what went wrong.
Lili retweeted
Standard RL is limited by sparse feedback, but distillation requires rollouts from a teacher. @yus167โs team found a happy medium by training on text feedback from judges, which the model then learns to internalize and predict.
teapawt.org/yus167/status/20187736โฆ
Lili retweeted
alright, so you can get more than 1 bit from RL:
1. instead of single turn rollout, do a 2-turn one. try - get critique - try again.
2. use the 2nd try for training conditioning on the input (same as first try)
3. compute CE on the critique as an auxiliary loss
methods:
- Self Distillation = applies (1) and (2)
- Feedback Modeling, no 2nd rollout conditioning trick, applies (3) as well
findings:
- FM > SD in math and reasoning tasks.
- SD > FM in creative writing. Will def try on HammingBench
results:
- Knights & Knaves: GRPO=0.352 - SD=0.802, FM=0.880.
- Shortest Path: GRPO 0.384 - SD=0.830, FM=0.905.
- MATH500: DAPO=0.523 - SD=0.548, FM=0.567.
- AIME24: DAPO=0.025 โ SD=0.088 (best), FM=0.083.
Very fun read :) @yus167 @lchen915 @FahimTajwar10 @MunosRemi @deepakpathak @Zanette_ai
Lili retweeted
Awesome to see LitBench showing improved performance with text feedback
Lili retweeted
Glad to see folks are finding LitBench useful
cc @DanielFein7
Lili retweeted
Are we done with new RL algorithms? Turns out we might have been optimizing the wrong objective.
Introducing MaxRL, a framework to bring maximum likelihood optimization to RL settings.
Paper + code + project website: zanette-labs.github.io/MaxRLโฆ
๐งต 1/n
Lili retweeted
RL from scalar rewards is inefficient.
Our work shows how to leverage the text feedback that's already abundant in human-AI interaction. Two simple methods, Self Distillation and Feedback Modeling, deliver strong gains.
I am very excited to see where this paradigm goes!


























