In LLM inference, temperature controls the trade-off between reliability and creativity. Low temperatures keep the model predictable, while high temperatures encourage exploration.
Naturally, different questions require different approaches. Some demand careful, step-by-step reasoning, while others require creative leaps between concepts. So, when attempting to solve the same unknown problem multiple times (as in pass@K), it seems natural to vary the temperature across attempts.
Yet in practice, almost everyone uses a single temperature. Why?
In our new paper with @orishemur and Prof. Yaron Oz, "One Temperature to Rule Them All?", which we will present at NeurIPS 2026, we investigate this gap, showing how multiple temperatures can be used, and why, most of the time, they shouldn't.
🔹 The Hidden Cost of Diversity: While optimal multi-temperature schemes can improve performance, splitting the sampling budget across suboptimal temperatures incurs an exponential penalty, whereas sticking to a single well-tuned "compromise" temperature incurs only a linear cost.
🔹 The Goldilocks Effect: At a given sampling budget K, performance is largely driven by questions whose difficulty matches the scale of K. Harder or easier questions have little effect on the choice of the ideal temperature. This means you don't need to know a question's exact difficulty to choose a good temperature; you mainly need to tune it to your compute budget K.
🔹 A Practical Optimization Framework: In some cases, the diversity of questions is so great that using multiple temperatures does provide a meaningful benefit. For these cases, we show how to discretize the problem and reduce it to a fast, tractable convex optimization task.
All of our results extend beyond temperature to other sampling hyperparameters.
Paper: arxiv.org/abs/2611.13077
Remember Dangerous Dave from when we were kids?
Remember when we got stuck around level 7, and for every attempt to figure it out, we had to run through levels 1–6, just to die on level 7 again, without ever figuring it out?
Well, we remade it, and now you can choose the level!
We also made the game run smoother and added a modern-resolution option that preserves the original look and feel, just with sharper graphics!
Come check it out at doshd.com

