I'm not at @COLM_conf, but my awesome PhD student @noahpro99 is giving a talk on our EvoSkill work today at the COLM Social: Self-Improving Agents (Oct 6 1-2:30 PM PT, Franciscan D). Come chat with him about his first-hand experience with different skill evolution frameworks!
hosting the Self-Improving Agents social at @COLM_conf tomorrow
- 4 talks: @sethkarten @yoonholeee @qinyuan_ye and Noah Provenzano
- panel: @HuaxiuYaoML @ZhiruoW @jasonwu0731
my 4th one after ICLR, ICML and NeurIPS, and I still love hosting these.
1-2:30pm, Franciscan D. come by if you're at COLM
luma.com/m8v4fq76
Pin-Jie Lin retweeted
Accepted to NeurIPS as a Spotlight paper!🎉🎉
Thanks to all my collaborators for their support and feedback!
🌟New paper alert: The Master Key Hypothesis🌟
Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.
Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
Glad to share that our paper on The Master Key Hypothesis / cross-model capability transfer got accepted @NeurIPSConf 2026 as spotlight 🎉 !! Top 1.3% (292 spotlight + 112 oral) out of 30709 submissions. Congrats to my students @Sub_RBala @linusdd44804 @helix9550 and co-authors @mohitban47!!
🌟New paper alert: The Master Key Hypothesis🌟
Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.
Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
Pin-Jie Lin retweeted
🌟New paper alert: The Master Key Hypothesis🌟
Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.
Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
Nice work by @verma_apurv5 showing that post-training can be distilled into a portable logit delta that transfers across model sizes and families, which corroborates the findings in our recent work on the Master Key Hypothesis (arxiv.org/abs/2604.06377) and fine-tuning/alignment transfer (arxiv.org/abs/2503.20110).
Replying to @verma_apurv5
5/8 🔁
Second, the delta is portable. A delta drawn from a 1.5B Qwen lifts Llama-3-8B, a different family with a different tokenizer, by 15% on GSM8K.
Other recent works do weight-space transfer such as task arithmetic (arxiv.org/abs/2212.04089) and the recent Master Key Hypothesis (arxiv.org/abs/2604.06377 by @tuvllms, @Sub_RBala), and typically require same model family. A simple vocabulary bridge is sufficient for logit grafting.
Drop by if you're interested in search-augmented LLMs and reasoning for handling dynamic, real-world knowledge 👀
SEAL-0 is gaining popularity as a fact-seeking benchmark, used by models like DeepSeek-V3.1, ROMA (Recursive Open Meta Agent), and Kimi-Thinking.
#ICLR2026
Come checkout our posters at #ICLR2026 🇧🇷!
April 23, 10:30 AM - 1:00 PM
📍Pavilion 3 P3-#1424
SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models (arxiv.org/abs/2506.01062)
I'm also looking for opportunities this summer and would love to connect and chat!
3
253
🚨New preprint 🚨
Looking for a real challenge for multimodal agents? Found SealQA/RAMDocs useful and want a multimodal version?
Check out MERRIN (led by @HanWang98), our new benchmark w/ human-written questions for multimodal search + reasoning over conflicting information 👇
Paper: arxiv.org/abs/2604.13418
Webpage: merrin-benchmark.github.io
Benchmark: huggingface.co/datasets/HanN…
w/ @HanWang98 @hyunji_amy_lee @mohitban47 @thinhphp_vt @mikaelacankosy @WeiyuanChen01
@EliasEskin
🚨 Excited to announce MERRIN: a human-annotated, expert-vetted benchmark for Multimodal Evidence Retrieval and Reasoning in Noisy Web Environments! 🌐🔍
The web is noisy, conflicting, and multimodal. Even strong search agents fail when questions require reasoning across text, images, video & audio — MERRIN is built to diagnose exactly where they break down, via multi-hop reasoning on open-web search without explicit references to specific modality sources.
🚨✨ New preprint ✨🚨
Can we transfer post-training capabilities across model families, scales, and releases?
Our new work on UNLOCK, led by my PhD student @Sub_RBala, suggests that latent capabilities correspond to low-dimensional directions in latent space, which can be extracted as steering vectors and transferred across models via simple linear transformations. UNLOCK requires no training or labeled contrastive examples, only forward passes.
Most exciting result: Qwen3-14B-Base + transferred math reasoning from Qwen3-4B approaches the performance of Qwen3-14B (post-trained) without any training.
📜: arxiv.org/abs/2604.06377
💻: github.com/rishabbala/Steeri…
🤝: w/ wonderful students & collaborators @Sub_RBala @linusdd44804 @SharmaRituraj19 @anjiefang, Fardin Abdi, Viktor Rozgic, Zheng Du, and @mohitban47
🧵: Check out @Sub_RBala's thread below 👇
🌟New paper alert: The Master Key Hypothesis🌟
Post-training is expensive AND you have to repeat it for every new model. What if you could just transfer the capability instead?
We introduce UNLOCK — a training-free and label-free framework that transfers capabilities across models using low-rank linear projections.
Takeaways:
1⃣ Skills like CoT & math reasoning live as directions in the model's latent space. We capture these as a steering vector — we call this the Master Key.
2⃣ Simple low-rank linear transformations are sufficient to transfer the Master Key across models' latent spaces
3⃣ UNLOCK elicits behaviors that even prompting can't reliably trigger
4⃣ Gains scale with base model strength. Stronger models show larger gains post-transfer.
5⃣ UNLOCK works by sharpening the output distribution — steering the model toward the right answer.
🚨 New paper 🚨
Excited to share our new work on EvoSkill (led by @salahalzubi401), a self-evolving framework that automatically discovers and refines agent skills through iterative failure analysis 🔁🧬. It achieves state-of-the-art performance on @databricks's OfficeQA. Check it out!
📰: arxiv.org/abs/2603.02766
Introducing EvoSkill: a framework that analyzes agent failures and automatically builds the missing skills, leading to rapid improvement on difficult benchmarks and generalizable skills across use-cases.
+12.1% on SealQA
+7.3% on OfficeQA (SOTA)
+5.3% on BrowseComp via zero-shot transfer from SealQA
Read more below 🧵
🚨 New paper 🚨
Excited to share PRISM, a new “DeepThink” method that uses step-level correctness signals from a process reward model to guide inference over candidate solutions. PRISM matches or beats SOTA methods, enabling gpt-oss-20b to exceed gpt-oss-120b.👇
📰: arxiv.org/abs/2603.02479
#AI #LLMs
Pin-Jie Lin retweeted
we released Olmo 3! lot of exciting stuff but wanna focus on:
🐟Olmo 3 32B Base, the best fully-open base model to-date, near Qwen 2.5 & Gemma 3 on diverse evals
🐠Olmo 3 32B Think, first fully-open reasoning model approaching Qwen 3 levels
🐡12 training datasets corresp to different staged training recipes, all open & accessible
since I'm a pretraining person, I'll share some of my fav Base model ideas:
Drop by today if you’re around!
I am not at EMNLP this year, but my student @linusdd44804 will be presenting our paper on efficient model development through fine-tuning transfer. The presentation is tomorrow 2-3:30 pm, A109 (session 15). Please come talk to him!
79
Pin-Jie Lin retweeted
Replying to @SanghaniCtrVT @therealthapa
One more @SanghaniCtrVT paper at #EMNLP2025:
Efficient Model Development through Fine-tuning Transfer
Main proceedings
@linusdd44804 @Sub_RBala @tuvllms (all VT) w/@fyliufengyuan, @kandpal_nikhil
tinyurl.com/2kv9nr25
I’ll be presenting our fine-tuning transfer paper tomorrow!
TLDR: Alignment tuning effects can be captured as transferable model diff vectors — no need to fine-tune from scratch for every new base model version.
Come find me:
🕑 14:00–15:30
📍 A109 (Session 15)
#EMNLP2025
Excited to share that our paper on efficient model development has been accepted to #EMNLP2025 Main conference @emnlpmeeting. Congratulations to my students @linusdd44804 and @Sub_RBala on their first PhD paper! 🎉
Pin-Jie Lin retweeted
LoRA makes fine-tuning more accessible, but it's unclear how it compares to full fine-tuning. We find that the performance often matches closely---more often than you might expect. In our latest Connectionism post, we share our experimental results and recommendations for LoRA.
thinkingmachines.ai/blog/lor…
Pin-Jie Lin retweeted
DeepSeek achieved a strong result on SEAL0, a challenging benchmark for reasoning with conflicting search results. 🎊
Replying to @deepseek_ai
Tools & Agents Upgrades 🧰
📈 Better results on SWE / Terminal-Bench
🔍 Stronger multi-step reasoning for complex search tasks
⚡️ Big gains in thinking efficiency
3/5
🎉🎉
Excited to share that our paper on efficient model development has been accepted to #EMNLP2025 Main conference @emnlpmeeting. Congratulations to my students @linusdd44804 and @Sub_RBala on their first PhD paper! 🎉
1
97
Pin-Jie Lin retweeted
We just evaluated Grok 4 on our SEAL-0 dataset
👍Try it: huggingface.co/datasets/vtll…































