Ori Shem-Ur (@orishemur) header
Ori Shem-Ur (@orishemur) profile picture

What happens in the light cone stays in the light cone

Joined October 2020
Ori Shem-Ur retweeted
In LLM inference, temperature controls the trade-off between reliability and creativity. Low temperatures keep the model predictable, while high temperatures encourage exploration. Naturally, different questions require different approaches. Some demand careful, step-by-step reasoning, while others require creative leaps between concepts. So, when attempting to solve the same unknown problem multiple times (as in pass@K), it seems natural to vary the temperature across attempts. Yet in practice, almost everyone uses a single temperature. Why? In our new paper with @orishemur and Prof. Yaron Oz, "One Temperature to Rule Them All?", which we will present at NeurIPS 2026, we investigate this gap, showing how multiple temperatures can be used, and why, most of the time, they shouldn't. ๐Ÿ”น The Hidden Cost of Diversity: While optimal multi-temperature schemes can improve performance, splitting the sampling budget across suboptimal temperatures incurs an exponential penalty, whereas sticking to a single well-tuned "compromise" temperature incurs only a linear cost. ๐Ÿ”น The Goldilocks Effect: At a given sampling budget K, performance is largely driven by questions whose difficulty matches the scale of K. Harder or easier questions have little effect on the choice of the ideal temperature. This means you don't need to know a question's exact difficulty to choose a good temperature; you mainly need to tune it to your compute budget K. ๐Ÿ”น A Practical Optimization Framework: In some cases, the diversity of questions is so great that using multiple temperatures does provide a meaningful benefit. For these cases, we show how to discretize the problem and reduce it to a fast, tractable convex optimization task. All of our results extend beyond temperature to other sampling hyperparameters. Paper: arxiv.org/abs/2611.13077
7
47
4
731
348,084
Ori Shem-Ur retweeted
Remember Dangerous Dave from when we were kids? Remember when we got stuck around level 7, and for every attempt to figure it out, we had to run through levels 1โ€“6, just to die on level 7 again, without ever figuring it out? Well, we remade it, and now you can choose the level! We also made the game run smoother and added a modern-resolution option that preserves the original look and feel, just with sharper graphics! Come check it out at doshd.com
112
340
41
3,013
1,564,551
ื™ืฉ ืœื™ ืกื™ืคื•ืจ ืžืžืฉ ื ื—ืžื“ ืขืœ ืžืฉื˜ืจืช ื™ืฉืจืืœ. ืœืื‘ื ืฉืœ ื—ื‘ืจื” ืฉืœื™, ืฉื’ืจ ื‘ืจื—ื•ื‘ื•ืช, ื™ืฉ ื’'ื™ืค ืžืžืฉ ืžื’ื ื™ื‘ (ื ืกืขืชื™ ื‘ื• ืคืขื, ื‘ืืžืช ืžื’ื ื™ื‘) ืฉืžืžืฉ ื˜ื•ื‘ ืœืฉื˜ื—. ื•ืœืคื ื™ ื›ืžื” ื™ืžื™ื ื”ื’'ื™ืค ืฉืœื• ื ื’ื ื‘. ื”ื•ื ื›ืžื•ื‘ืŸ ื“ื™ื•ื•ื— ื™ืฉืจ ืœืžืฉื˜ืจื”, ื•ื”ื ืืžืจื• ืœื• ืฉื”ื ื“ื•ืื’ื™ื ืœืขื ื™ื™ืŸ.
1
2
165
ื›ืš ืื• ื›ืš ืื•ืžืจ ื”ืฉื•ื˜ืจ, ืขื›ืฉื™ื• ื ื›ื ืกืช ืฉื‘ืช, ื•ื’ื ื›ื›ื” ืืฃ ืื—ื“ ืœื ื™ืขืฉื” ืขื ื–ื” ื›ืœื•ื (ื›ืŸ ื™ื ื•ื ื•ื›ืŸ ื™ื™ืฉืŸ ืฉื•ื˜ืจ ื™ืฉืจืืœ).
1
2
107
ื‘ื™ื•ื ืฉื‘ืช ืฉื•ื‘ ื”ืชืงืฉืจ ื”ืื“ื•ืŸ, ืฉื•ื‘ ืžืื•ืชื• ื”ืžืกืคืจ, ื•ืฉื•ืืœ ืืช ืื‘ื ืฉืœ ื—ื‘ืจื” ืฉืœื™ ืื ื”ื•ื ืจืฆื™ื ื™ ื•ืจื•ืฆื” ืœืกื’ื•ืจ ืืช ื”ืขืกืงื”. ื•ืžืฆื™ืข ืœื• ืœื‘ื•ื ืืœื™ื•, ื‘ืื•ื˜ื• ื”ืจื’ื™ืœ ืฉืœื•, ืขื 15,000 ืฉืงืœ, ื‘ืžื–ื•ืžืŸ...
1
3
107
Replying to @aphysicist
This has been my internal model for teamwork for years. Itโ€™s also why I named my company Vectโ€”we strive to align internally and help other companies do the same, with @Vect_io
1
3
2
592
At #NeurIPS?โฃ Come to the ๐’๐œ๐ข๐Ÿ’๐ƒ๐‹ ๐ฐ๐จ๐ซ๐ค๐ฌ๐ก๐จ๐ฉ this Sunday at ๐Ÿ•š 11:05 AM to catch our ๐จ๐ซ๐š๐ฅ presentation on ๐ˆ๐ง๐ฉ๐ฎ๐ญ ๐’๐ฉ๐š๐œ๐ž ๐Œ๐จ๐๐ž ๐‚๐จ๐ง๐ง๐ž๐œ๐ญ๐ข๐ฏ๐ข๐ญ๐ฒ. I canโ€™t make it this time, but Iโ€™m pretty sure @DavidSKrueger will deliver an exciting talk!
1/12 Excited to share our new paper Input Space Mode Connectivity in Deep Neural Networks. ๐Ÿ“œ๐Ÿงต Did you know that high-dimensional input spaces in classification vision models contain simple paths between distinct (low-loss) images of the same class?
2
3
1,531
ืฉื™ื•ืื• ืคืจืกื™ ื”ื ื•ื‘ืœ ื‘ืคื™ื–ื™ืงื” ื”ืฉื ื” ื”ื ื‘ื“ื™ื•ืง ื‘ื ื•ืฉื ืฉืื ื™ ื—ื•ืงืจ (ืœื”ื‘ื™ืŸ ืžืฉื™ืŸ ืœืจื ื™ื ื’ ื‘ืขื–ืจืช ืคื™ื–ื™ืงื”)
1
2
128
Mode connectivity is one of the wilder quirks of deep learning. Turns out it applies not only to parameter space but also input space! And, based on percolation theory, it seems likely to be a fairly general property of high dimensional spaces. Very cool work from my visitor!
1/12 Excited to share our new paper Input Space Mode Connectivity in Deep Neural Networks. ๐Ÿ“œ๐Ÿงต Did you know that high-dimensional input spaces in classification vision models contain simple paths between distinct (low-loss) images of the same class?
1
2
30
3,017
Just published a new paper with @VrabelJakub, @DavidSKrueger, and my advisor Yaron Oz about connectivity in the input space of deep neural networks. It was a great pleasure working with all of you, and I am excited for the next steps!
1/12 Excited to share our new paper Input Space Mode Connectivity in Deep Neural Networks. ๐Ÿ“œ๐Ÿงต Did you know that high-dimensional input spaces in classification vision models contain simple paths between distinct (low-loss) images of the same class?
1
4
443
1/12 Excited to share our new paper Input Space Mode Connectivity in Deep Neural Networks. ๐Ÿ“œ๐Ÿงต Did you know that high-dimensional input spaces in classification vision models contain simple paths between distinct (low-loss) images of the same class?
6
46
6
276
31,784
ื˜ื•ื‘, ืื ื™ ืฉืžื— ืœื‘ืฉืจ ืฉืœืคื—ื•ืช ื‘ืฉื—ืžื˜ o1 ืขื“ื™ื™ืŸ ื’ืจื•ืข
1
46