In LLM inference, temperature controls the trade-off between reliability and creativity. Low temperatures keep the model predictable, while high temperatures encourage exploration.
Naturally, different questions require different approaches. Some demand careful, step-by-step reasoning, while others require creative leaps between concepts. So, when attempting to solve the same unknown problem multiple times (as in pass@K), it seems natural to vary the temperature across attempts.
Yet in practice, almost everyone uses a single temperature. Why?
In our new paper with @orishemur and Prof. Yaron Oz, "One Temperature to Rule Them All?", which we will present at NeurIPS 2026, we investigate this gap, showing how multiple temperatures can be used, and why, most of the time, they shouldn't.
๐น The Hidden Cost of Diversity: While optimal multi-temperature schemes can improve performance, splitting the sampling budget across suboptimal temperatures incurs an exponential penalty, whereas sticking to a single well-tuned "compromise" temperature incurs only a linear cost.
๐น The Goldilocks Effect: At a given sampling budget K, performance is largely driven by questions whose difficulty matches the scale of K. Harder or easier questions have little effect on the choice of the ideal temperature. This means you don't need to know a question's exact difficulty to choose a good temperature; you mainly need to tune it to your compute budget K.
๐น A Practical Optimization Framework: In some cases, the diversity of questions is so great that using multiple temperatures does provide a meaningful benefit. For these cases, we show how to discretize the problem and reduce it to a fast, tractable convex optimization task.
All of our results extend beyond temperature to other sampling hyperparameters.
Paper: arxiv.org/abs/2611.13077
Remember Dangerous Dave from when we were kids?
Remember when we got stuck around level 7, and for every attempt to figure it out, we had to run through levels 1โ6, just to die on level 7 again, without ever figuring it out?
Well, we remade it, and now you can choose the level!
We also made the game run smoother and added a modern-resolution option that preserves the original look and feel, just with sharper graphics!
Come check it out at doshd.com
ืืฉ ืื ืกืืคืืจ ืืืฉ ื ืืื ืขื ืืฉืืจืช ืืฉืจืื.
ืืืื ืฉื ืืืจื ืฉืื, ืฉืืจ ืืจืืืืืช, ืืฉ ื'ืืค ืืืฉ ืืื ืื (ื ืกืขืชื ืื ืคืขื, ืืืืช ืืื ืื) ืฉืืืฉ ืืื ืืฉืื. ืืืคื ื ืืื ืืืื ืื'ืืค ืฉืื ื ืื ื. ืืื ืืืืื ืืืืื ืืฉืจ ืืืฉืืจื, ืืื ืืืจื ืื ืฉืื ืืืืืื ืืขื ืืื.
1
2
165
ืื ืื ืื ืืืืจ ืืฉืืืจ, ืขืืฉืื ื ืื ืกืช ืฉืืช, ืืื ืืื ืืฃ ืืื ืื ืืขืฉื ืขื ืื ืืืื (ืื ืื ืื ืืื ืืืฉื ืฉืืืจ ืืฉืจืื).
1
2
107
Ori Shem-Ur retweeted
Replying to @aphysicist
This has been my internal model for teamwork for years.
Itโs also why I named my company Vectโwe strive to align internally and help other companies do the same, with @Vect_io
Ori Shem-Ur retweeted
At #NeurIPS?โฃ Come to the ๐๐๐ข๐๐๐ ๐ฐ๐จ๐ซ๐ค๐ฌ๐ก๐จ๐ฉ this Sunday at ๐ 11:05 AM to catch our ๐จ๐ซ๐๐ฅ presentation on ๐๐ง๐ฉ๐ฎ๐ญ ๐๐ฉ๐๐๐ ๐๐จ๐๐ ๐๐จ๐ง๐ง๐๐๐ญ๐ข๐ฏ๐ข๐ญ๐ฒ.
I canโt make it this time, but Iโm pretty sure @DavidSKrueger will deliver an exciting talk!
ืฉืืืื ืคืจืกื ืื ืืื ืืคืืืืงื ืืฉื ื ืื ืืืืืง ืื ืืฉื ืฉืื ื ืืืงืจ (ืืืืื ืืฉืื ืืจื ืื ื ืืขืืจืช ืคืืืืงื)
1
2
128
Ori Shem-Ur retweeted
Mode connectivity is one of the wilder quirks of deep learning.
Turns out it applies not only to parameter space but also input space!
And, based on percolation theory, it seems likely to be a fairly general property of high dimensional spaces.
Very cool work from my visitor!
Just published a new paper with @VrabelJakub, @DavidSKrueger, and my advisor Yaron Oz about connectivity in the input space of deep neural networks. It was a great pleasure working with all of you, and I am excited for the next steps!
Ori Shem-Ur retweeted
1/12 Excited to share our new paper Input Space Mode Connectivity in Deep Neural Networks. ๐๐งต
Did you know that high-dimensional input spaces in classification vision models contain simple paths between distinct (low-loss) images of the same class?







