Gemini and Gemma, mostly pre-training (又菜又爱玩). past (?) life: graph machine learning

Joined August 2016
Anton Tsitsulin retweeted
We're also hiring Student Researchers in the Frontier Computation team @GoogleDeepMind, to work with @re_rayne & @IAmTimNguyen! Areas of interest: agentic optimisation, learning algorithms/simulations, OOD generalisation, AI4Physics, CFD, numerical math. Applying details below!
1
22
264
15,924
Anton Tsitsulin retweeted
The Gemini Large Scale Pretrain team is hiring a Research Engineer in London or MTV! The team works across the pretrain stack: research codebases, distributed training infra, data pipelines, tooling for training dynamics analysis, and most importantly using Gemini to accelerate our work! 🚀 If you're an engineer who has a strong "whatever it takes" attitude and is not afraid of spending tokens, apply here with some showcase work: google.com/about/careers/app… No particular qualifications needed, and we welcome early career applicants. Join if you want me to bug you with refactors to enable [new AGI compute multiplier]! 👀
14
60
4
564
58,075
Anton Tsitsulin retweeted
My group at the University of Edinburgh is recruiting #PhD students for 2027! 🎓 We are GLUEd: Geometric Learning & Uncertainty in Edinburgh. Group: vab.im/group/ How to apply: vab.im/vacancies/ 1/2
1
10
46
1,967
Anton Tsitsulin retweeted
I’m happy to share that our team's work on PiPNN—an ultra-scalable algorithm for high-dimensional Nearest Neighbor Search—has won three awards at KDD’26, VecDB’26, and SISAP’26 with up to 78x speedup on index build. Nearest-neighbor search is fundamental to deduping in data curation and embedding-driven retrieval for RAG for LLMs. It is great to see our work recognized across the community: Best Paper at KDD’26: PiPNN: A Framework for Ultra-Scalable Graph-Based Nearest Neighbor Indexing. This paper introduces the PiPNN framework, which delivers an order-of-magnitude speedup for building navigable proximity graphs. (dl.acm.org/doi/10.1145/37708…) Best Paper at VecDB’26: Turbocharging PiPNN for Proximity and 𝑘-NN Graph Building. The paper improved the original PIPNN kernels, resulting in an additional 8x speedup (up to a 78x total speedup!), and introduced a highly optimized GPU version. (openreview.net/forum?id=8Gzb…) 1st Place at the SISAP’26 Challenge: Our PiPNN-based solution ranked 1st for Task 1, which challenged teams to develop the fastest k-NN graph building algorithm. (sisap-challenges.github.io/2…) Nearest-neighbor search is an important part of Data Curation for LLMs, and retrieval for RAG pipelines, and it only becomes more important as we move to more long-horizon agents and handling long-context inference-time scaling. Congratulations to the team behind this work: Tobias Rubel, Richard Wen (UMD), Guy Blelloch, Laxman Dhulipala, Lars Gottesbüren(github.com/larsgottesbueren), Jakub Łącki, and Vahab Mirrokni.
5
20
3
167
9,450
Anton Tsitsulin retweeted
i am being paid way too much money to stop . but someone should really stop me
7
2
134
12,991
Anton Tsitsulin retweeted
Happy to share we just published Gemma 4 technical report! Take a look
37
125
11
1,062
91,479
Anton Tsitsulin retweeted
Six offline RL distillation losses. One base model. The exact same math rollouts. Do they actually learn the same thing? Turns out most "new" losses — RFT, DFT, offline GRPO — write nearly the same direction in weight space as plain SFT. Only DPO learns something genuinely different: near-orthogonal, its own loss basin, rewires what the network computes. Reward-weighting changes the step size. DPO changes the direction. Accepted at @icmlconf (MechInterp workshop) paper + interactive companion 👇 huggingface.co/spaces/AlexWo…
4
29
1
187
17,593
Anton Tsitsulin retweeted
AI has now solved a major open problem -- one of the best known Erdos problems called the unit distance problem, one of Erdos's favourite questions and one that many mathematicians had tried. openai.com/index/model-dispr…
71
601
239
3,520
1,512,656
Anton Tsitsulin retweeted
How to land a job at a frontier lab vladfeinberg.com/2026/05/10/…
The vibes in SF feel pretty frenetic right now. The divide in outcomes is the worst I've ever seen. Over the last 5yrs, a group of ~10k people - employees at Anthropic, OpenAI, xAI, Nvidia, Meta TBD, founders - have hit retirement wealth of well above $20M (back of the envelope AI estimation). Everyone outside that group feels like they can work their well-paying (but <$500k) job for their whole life and never get there. Worse yet, layoffs are in full swing. Many software engineers feel like their life's skill is no longer useful. The day to day role of most jobs has changed overnight with AI. As a result, 1. The corporate ladder looks like the wrong building to climb. Everyone's trying to align with a new set of career "paths": should I be a founder? Is it too late to join Anthropic / OpenAI? should I get into AI? what company stock will 10x next? People are demanding higher salaries and switching jobs more and more. 2. There’s a deep malaise about work (and its future). Why even work at all for “peanuts”? Will my job even exist in a few years? Many feel helpless. You hear the “permanent underclass” conversation a lot, esp from young people. It's hard to focus on doing good work when you think "man, if I joined Anthropic 2yrs ago, I could retire" 3. The mid to late middle managers feel paralyzed. Many have families and don't feel like they have the energy or network to just "start a company". They don't particularly have any AI skills. They see the writing on the wall: middle management is being hollowed out in many companies. 4. The rich aren’t particularly happy either. No one is shedding tears for them (and rightfully so). But those who have "made it" experience a profound lack of purpose too. Some have gone from <$150k to >$50M in a few years with no ramp. It flips your life plans upside down. For some, comparison is the thief of joy. For some, they escape to NYC to "live life". For others still, they start companies "just cuz", often to win status points. They never imagined that by age 30, they'd be set. I once asked a post-economic founder friend why they didn't just sell the co and they said "and do what? right now, everyone wants to talk to me. if i sell, I will only have money." I understand that many reading this scoff at the champagne problems of the valley. Society is warped in this tech bubble. What is often well-off anywhere else in the world is bang average here. Unlike many other places, tenure, intelligence and hard work can be loosely correlated with outcomes in the Bay. Living through a societally transformative gold rush in that environment can be paralyzing. "Am I in the right place? Should I move? Is there time still left? Am I gonna make it?" It psychologically torments many who have moved here in search of "success". Ironically, a frequent side effect of this torment is to spin up the very products making everyone rich in hopes that you too can vibecode your path to economic enlightenment.
59
178
52
3,262
1,548,497
Anton Tsitsulin retweeted
Happy to share new proofs of simple constructions of general Knuth's recent cycle conjecture for even case based on a new Gemini system for long proofs; one with 40+ pages & one with 70+ pages here: github.com/dpwoodru/knuthCyc… Excited about the system for long proofs. See Comments
2
4
2
30
3,805
Anton Tsitsulin retweeted
Tired of getting rejects and reading reviewer comments that clearly missed the point of your paper? I built an peer review skill for Claude Code that actually gives you useful feedback before you submit. 👇
5
9
41
1,523
Anton Tsitsulin retweeted
🔎 If you are interested in machine unlearning, check out our poster tomorrow at #ICLR2026. (Pavilion 4, P4-#3916 | Apr 24, 10:30 AM – 1:00 PM) We explore a new direction for machine unlearning, leveraging model checkpoints! 📜 Paper: arxiv.org/abs/2506.20941
1
5
2
31
7,499
Anton Tsitsulin retweeted
The growing KV-cache of attention is the key component for the long-context understanding of LLMs, but what holds back long-term memory modules (e.g., Titans)? What if we could have the compression power of Titans but with a growing memory similar to Transformers? Memory Caching: A class of architectures that compress the context into a slow growing memory (not as fast as Transformers, but not as static as RNNs), resulting in recurrent neural networks with non-fixed-sized memory (hidden states). Building on this formulation, we present Sparse Selective Caching, an architecture with growing effective memory (similar to attention) but with almost constant inference cost per token (similar to RNNs).
23
163
11
996
111,524
Anton Tsitsulin retweeted
Our goal in the Gemma team is to ship models that are useful by generalizing to unseen tasks. Hence, we are extremely strict about not doing anything that would target specific benchmarks instead of teaching the model broad capabilities. It's great to see this reflected here.
Qwen3.5 27B vs Gemma 4 31B IT  Here is why I think Gemma 4 31B wins: Benchmark scores of both models are very close. Generalization is not. I reused NVIDIA’s CoDeC metric to probe "benchmark affinity". Officially, CoDeC > 80 suggests contamination. I've never seen this. But even below that, it can still reveal how benchmark-affine a model is. Higher CoDeC scores mean the model is more "comfortable" with the benchmark. So the ideal profile is: high benchmark scores + low CoDeC. By that lens, Gemma 4 looks clearly stronger than Qwen3.5. Gemma 4 likely generalizes better to unseen tasks. I observed the same pattern earlier with Gemma 3 vs Qwen3. I wrote about this a few months ago: kaitchup.substack.com/p/did-…
12
38
5
409
46,974
Anton Tsitsulin retweeted
🚀 We present the first large-scale Lean 4 formulation of Statistical learning theory from scratch! Led by my student @yuanhezhang6 and collaborated with @jasondeanlee 📄 Paper: arxiv.org/abs/2602.02285 💻 GitHub: github.com/YuanheZ/lean-stat… 🤗 Dataset: huggingface.co/collections/l…
5
66
7
338
55,426
Anton Tsitsulin retweeted
There has been so much noise on AI for Math research. We have been working on research-level math for over a year (in parallel with our IMO Olympiad math effort) and obtained many results including solving Erdős problems (and beyond!). We haven't shared much in the past yet as we want to do things responsibly with respect to the math community. We're almost done! Yesterday, we released the first paper in our series: Solving a generalized version of Erdős-1051 problem! More to come! #DeepThink
14
77
12
527
87,374
Anton Tsitsulin retweeted
Get feedback on your #ICML2026 submission from Google's Gemini-based paper assistant tool, an advanced LLM tool specialized to assist with ICML submissions. Live now! More details in the blog post below 👇
6
40
13
318
73,537
Anton Tsitsulin retweeted
Pass/fail benchmarks are saturated. It’s time for FrontierCS. 🚀 150+ unsolved, verifiable problems ranging from competitive programming to real-world research. Designed by PhDs & ICPC experts to evolve model intelligence. 🎓🧠 🧵👇Check it out! Paper: arxiv.org/abs/2512.15699
7
42
7
256
43,511
Anton Tsitsulin retweeted
GraphBench: Next-generation graph learning benchmarking is now available! 🔍📊 This work introduces GraphBench: a comprehensive benchmarking framework for graph learning that provides principled baselines and reference performance across modern models. graphbench.io
2
12
1
50
7,619
Our new experiment uses an advanced version of Gemini 2.5 Deep Think to rigorously verify theoretical computer science papers. 97% of trial participants, authors for #STOC2026, found the feedback helpful for catching errors & improving clarity. More at: goo.gle/3MXslYP
15
95
7
777
132,629