Freedom Preetham (@freedompreetham) header
Freedom Preetham (@freedompreetham) profile picture

AI | Math | Genomics | Quantum Physics | https://teapawt.org/t.co/KSL0H1JY4y | Founder ~ https://teapawt.org/t.co/oUISz0Ou0D | https://teapawt.org/t.co/RAS35yDyTn | https://teapawt.org/t.co/B03V3QhJr7

San Francisco, CA
Joined June 2008
Photo by Freedom Preetham (@freedompreetham)Photo by Freedom Preetham (@freedompreetham)Photo by Freedom Preetham (@freedompreetham)Photo by Freedom Preetham (@freedompreetham)Photo by Freedom Preetham (@freedompreetham)Photo by Freedom Preetham (@freedompreetham)
💥 NEW BOOK: THE LAST HUMAN VETO "In the world of AI, humans will and must retain the final decision!". Really??? I wrote 'The Last Human Veto' because I no longer think that sentence survives scrutiny. If the machine already controls the evidence, the options, the forecast, and the cost of refusal, the veto may remain visible long after its causal power is gone. The book asks a harder question. What does it mean to remain “in control” when intelligence becomes cheap, abundant, and eventually superior to ours? My answer is counterfactual authority. A veto is real only if saying no can still lead somewhere else. FREE PDF DOWNLOAD @ freedompreetham.org/ #AI #Agents #Humans #Future
3
2
4
490
The best AI models still cannot do software engineering. About a year back I said AI still cannot do software engineering and I was shouted at stating 95% of all software engineering is already done by AI. Here I am yet again after a year stating exactly the same thing again. I would like to tease out and dispel the myth between coding and software engineering. AI is getting extraordinarily good at "coding" because coding is increasingly a constrained inference problem. Give a frontier model a repository, an objective, tools, tests, and enough context, and it can search the codebase, infer local intent, modify multiple files, execute tests, debug failures, refactor implementations, and iterate until an executable oracle says the task is done. That is a profound capability gain. We have come a long way on coding. But software engineering operates in a far uglier regime. Here, you have to start sometimes with an empty slate, most times the state is partially observed, requirements are underspecified or does not exist, invariants are often undocumented, dependencies are historical, and the cost of a locally correct decision may appear months later as architectural debt, migration complexity, reliability loss, or an expanded blast radius. Coding asks whether a transformation of source code satisfies an objective. Engineering asks whether that transformation remains correct as the system, traffic, teams, dependencies, and product assumptions continue to change. This is why I think we are overestimating how close coding agents are to replacing software engineering. Most benchmarks hand the model a heavily compressed version of the engineering problem. Someone has already identified the repository, expressed the issue, bounded the environment, and frequently supplied tests that collapse verification into a binary signal. Real engineering begins before any of that exists. It requires discovering the hidden constraints, choosing system boundaries, reasoning over failure modes, deciding which complexity should be absorbed now and which should be deferred, preserving backward compatibility, and making decisions under delayed and ambiguous feedback. In control-theoretic terms, the agent is no longer solving for an action inside a mostly known state. It is estimating the state itself while simultaneously changing the system it is trying to understand. AI has driven down the cost of producing code much faster than the cost of establishing whether that code is the right change to make. That gap may become the defining technical bottleneck of AI-native software development. Even more so, we are quite far away in AI capabilities when it comes to complex distributed platform and production engineering. #AI #softwareengineering
3
1
3
511
I am wondering if frontier labs discover “pace the frontier” just as the pretraining engine behind their AGI promises began to lose its cleanest fuel? High quality human data is finite, frontier training runs are growing more expensive, and labs are shifting more of the work to reinforcement learning and inference time compute, where gains can be uneven and costly. Capability is still advancing, but the old promise that more tokens and compute would carry us steadily toward AGI is losing its empirical clarity. When the scaling curve bends, is “pace the frontier” a safety principle, or a way to buy time while the next reliable scaling regime remains unproven? @ylecun was probably right all along that LLMs (specifically, auto regressive, generative pre training models) would never lead to AGI and maybe this is what is playing out as a rhetoric in the name of pacing now?
1
3
316
If you care about where swarm intelligence is actually heading, this essay cuts right through the usual multi-agent hype. Here, I argue that the next massive breakthrough won't just be about building smarter base models, but building systems that know how to reorganize their own teamwork the second they hit a dead end. Instead of plugging AI into rigid workflows, the real leap is going to come from a swarm figuring out on the fly who should tackle what, what data to share, when it's actually productive to disagree, and how to collectively design real-world experiments that a single model never could have dreamed up alone. Link: cellbiosf.substack.com/p/swa… #AI #SwarmIntelligence #EmergentReasoning #Science
2
235
There is 100% certainty that we are all going to die. Death is inevitable. I have yet to meet anyone who has escaped it. But there is also a very high probability that accelerating AI can improve the longevity and quality of life for humans, other life forms, and the ecology of this planet. Better medicine, better biology, better agriculture, better climate modeling, better ways to understand complex systems. Slowing all of that down also has consequences. Yes, advanced AI can go wrong. There may be a 1% to 10% probability of serious failure if we get safety wrong. Which is exactly why we should double down on AI safety and accelerate that work just as aggressively as we accelerate capabilities. I do not think the answer is to slow down the development of AI models. That feels defeated and backwards to me. The answer is to build more capable systems while getting much better at making them safe, controllable, interpretable, and aligned as we go. #AI
1
1
2
437
Freedom Preetham retweeted
GPT-6 Sol and Luna are out, and they are better AND 50% cheaper than 5.6. Luna is now $0.10 input / $0.50 output per 1M tokens. This is on top of the 80% price cut to Luna we made at the end of July. Output went from $6 -> $0.50 within two months.
Please welcome GPT-6 Sol and GPT-6 Luna to the GPT-6 universe. GPT-6 Sol and Luna build on the advances behind GPT-6 Astra, bringing much of its strengths into faster and more affordable models to support work at scale. We’ve also made caching and inference more efficient, and we’re passing the savings directly to you: 50% lower API prices for Sol and Luna compared with GPT‑5.6 promotional pricing.
56
69
10
1,419
128,908
Freedom Preetham retweeted
The loudest voices stoking fears about AI dangers have made tremendous headway in the past two weeks. AI technology has not taken some unexpected, dangerous turn, but the hype around it — propelled by what appears to be a well orchestrated PR campaign — has drummed up considerable fear. I worry that it represents a setback for our field. I have written frequently that fears of AI are overhyped. AI’s capabilities can be uncannily human-like and unpredictable, and it’s rational to worry when people who are directly involved express concerns. But I see the problems as a sign of the engineering work that ahead, rather than insurmountable barriers or the sky falling. AI technology continues to advance — which is a good thing! — but technical advances, poorly understood by the public, give those who seek to generate hype repeated opportunities to do so. First, I don’t see any step up in the risk of human extinction from AI compared to a few months ago. The theories about this remain the same fantastical, science fiction scenarios as a few months ago. The biggest change in AI risk is its cybersecurity capabilities — a topic which we should take seriously — but this, too, will not lead to the end of the world. The most notable recent event leading to increased fear was when an OpenAI team deployed an agent swarm that hacked into Hugging Face. Much of the popular press contained significant hype. For example, some publications reported that a swarm of 1,200 agents carried out the attack. While this was technically accurate, as I write this, I have about 1,300 processes running on my laptop. Yes, the ability to get large swarms of agents to work in parallel on a task is a significant technical advance, And, in computing, many processes run at the same time. So this shouldn’t be seen as some magical capability. Additionally, OpenAI’s buggy sandboxing and monitoring processes were key to enabling this incident. Fixing these bugs and putting in place improved monitoring would be appropriate fixes, not pausing AI. There are many well known ways to attack software systems. The main advantage of AI agents is that they are relentless. They will tirelessly try many tactics — and have the patience to chain vulnerabilities together — that previously would have taken an infeasible amount of human effort. But in the long term, I believe the advantage will lie with defenders (because they have more information with which to identify bugs, which they can fix), but the cyber-threat landscape has changed significantly. There are still bottlenecks to identifying and exploiting a vulnerability. AI agents still have to try a lot of things to see what works, and taking these actions takes time and might be detected by defenders. This is why, even though it is now easy to obtain versions of leading open weight models that have had their guardrails removed or weakened, so they will not refuse to try to execute cyber attacks, the world has not ended. I am also concerned about the anthropomorphization of AI in a lot of reporting, where LLMs and agents are unnecessarily treated as if they were people. If I wield a hammer, miss a nail, and accidentally dent the wall, it’s not the fault of the hammer. The problem lies in how I used the hammer. Similarly, if I prompt an agent and it hacks into someone else’s system, the responsibility lies with me, not the agent. Of course, we want to build systems that are as safe and predictable as possible. (For example, an unsafe hammer would be one whose head randomly flies off under normal use.) Today’s agentic systems are not predictable, but I see no reason why, by applying sound engineering practices, we won’t be able to make them extremely safe to use. One new element in the forecasts of AI-enabled doom is AI companies disclaiming responsibility for their own products. “I didn’t do it; my out-of-control agent did!” There’s a balance to be struck between the responsibility of the tool maker and the tool user, but when something goes wrong, let’s hold the people building and/or using the hammer responsible, rather than the hammer. (By the way, if you’re worried about AI bioweapon risk, David Bellamy has a great post on why this, too, is overhyped. Briefly, the bottleneck in building a bioweapon is not intelligence, but lab work and manufacturing.) Pausing AI progress will create much more harm than benefit. First, our adversaries will certainly not slow down. Second, engineering requires discovering problems empirically so we can fix them. If we pause AI by a decade, we will also delay finding and implementing safety engineering fixes by about the same duration. Of course, the incentive to stoke fears — for regulatory capture, to garner attention, or to make one’s technology seem more powerful — remains the same as before. Disclaiming responsibility is a new one. Taking a hard technical look at the actual risks however, I see little factual basis for the degree of fear that’s been stoked up. We still have hard research and engineering work ahead to improve AI safety, but the beneficial applications continue to vastly outweigh the risks, and we should keep building. [Original text (with links): deeplearning.ai/the-batch/is… ]
963
1,847
484
8,849
8,027,218
Excited to share FloatLib, our verified arbitrary-precision floating-point arithmetic library in Lean. We’ve spent several months trying to bring together the best of both worlds: arithmetic we can prove correct and implementations that run efficiently. We built FloatLib to support verified machine learning and scientific computing, where rounding, overflow, and accumulation can change a program’s result. FloatLib supports IEEE binary and decimal, arbitrary-width posits, P3109, and small ML formats. You can also define your own formats and rounding rules. Each certified software backend comes with a Lean proof that it computes the specified result, including signed zeros and exceptional values. A lot of the work went into making those implementations faster, with lookup tables for tiny formats, machine-word kernels, and limb algorithms for wider arithmetic. They share the same specifications, so each optimization must come with a proof that it preserves the result. We also put FloatLib through extensive numerical checks and speed comparisons with established libraries, including MPFR, Flocq, FLoPS, Berkeley SoftFloat/TestFloat, and the posit libraries SoftPosit and Universal, across a range of formats, operations, and precisions. @Robertljg Project & Paper: leandojo.org/floatlib.html Code: github.com/lean-dojo/FloatLi…
11
61
2
308
23,696
The computational regime required for swarm intelligence have not been built yet, and no, it is not the same as the multi-agent platforms that exist today. Consider a 1000 agents investigating a biological mechanism. One group develops a persuasive explanation, another pursues an unlikely alternative, and a third discovers an anomaly whose significance becomes apparent only after a later experiment. Sharing the first explanation too widely can contaminate subsequent investigations, producing apparent agreement through common ancestry. Keeping investigators isolated preserves some independence while delaying useful corrections and duplicating expensive work. Meanwhile, a scheduler optimizing immediate progress could terminate the investigation that would eventually overturn the prevailing explanation. The population therefore has to learn how its own organization affects discovery. Who receives which evidence, when investigators collaborate, how long competing hypotheses remain funded, and where verification occurs all influence the trajectory of the computation. These decisions are coupled, their consequences are delayed, and no individual agent necessarily has enough information to evaluate them. Adding more agents expands the space of possible interactions long before it establishes a mechanism for choosing productive ones. This needs to be approached as a partially observed control problem in which communication, delegation, verification, and termination become learnable actions alongside investigation. Each agent would maintain a local history and an explicit resource budget, with policies determining when to reason independently, request evidence, recruit expertise, or expose a result. In controlled training environments, we could intervene on those interactions, withholding selected messages, changing delegation decisions, or preserving branches that would otherwise have been terminated, then evaluate the consequences against verified outcomes. The difficult problem is assigning credit when an apparently unproductive investigation supplies the contradiction that enables a breakthrough much later. A viable substrate would preserve task continuity across generations of agents, maintain the provenance needed to distinguish independent evidence from repetition, and enforce resource and execution constraints while the interaction policies evolve. Its contribution should be tested with underlying model weights and total computational budgets held fixed, against strong single-agent and fixed-team baselines on unfamiliar problems. My bet is that learning these policies could unlock substantial capability that remains inaccessible through fixed orchestration. Demonstrating that would establish something much more interesting than a larger team of agents. We would have made the organization of reasoning itself trainable, allowing collective capability to improve without requiring a corresponding improvement in the underlying models. #AI #SwarmIntelligence
238
I strongly believe that swarm intelligence will become one of the major computational regimes and architectures of the next decade, although the winning systems will look much less like today’s demos where ten LLMs are given personas and allowed to chat until they agree. The central technical question is changing from “How capable is the agent?” to “How should a population of heterogeneous agents organize itself around a changing problem?” Anthropic is already framing agent-to-agent interaction as an emerging systems problem, including shared environments, resource competition, coordination failure, collusion, specialization, and institutional behavior. Google DeepMind has gone further and launched a dedicated multi-agent safety research program focused on agent networks, collective capabilities, reputation, identity, protocols, and population-level failure modes. The useful abstraction is something like V ​= P + S + D + A − C − E − R − V where P is parallelism, S is specialization, D is epistemic diversity, and A is adaptive allocation of computation. Against those gains you pay communication cost C, coordination error E, correlated reasoning error R, and verification cost V. Most current multi-agent systems increase the left side a little while exploding the right side. If you instantiate twenty copies of approximately the same frontier model, they share enormous amounts of inductive bias, training distribution, failure modes, and epistemic blind spots. Twenty agents therefore do not give you twenty independent samples of intelligence. In many cases they give you twenty highly correlated estimators. Debate sometimes improves answers because it adds additional inference compute and alternative trajectories, but the marginal information added by agent n+1 usually falls rapidly. There is also a topology problem. A fully connected swarm of n agents potentially creates n(n−1)/2 communication relationships. At 1,000 agents that gives 499,500 potential pairwise channels. Nobody is going to build useful million-agent systems by putting everyone in the same conversation. This is why topology learning has become as important as model selection. Recent work is explicitly treating agent organization as a graph-learning and networking problem. Researchers are exploring dynamic publish-subscribe systems, reputation-aware routing, adaptive communication graphs, and learned coordination structures rather than predetermined chains of agents. The implication is that the next major leap in AI may come from the layer that can turn heterogeneous models, agents, tools and memory into a coherent computational population. That is a fundamentally new infrastructure layer. The companies that solve it will define how collective machine intelligence is actually built and operated at scale. This is the bet behind Odylith AI. #SwarmIntelligence #AI
2
3
381
Freedom Preetham retweeted
The Fields medalists’ objections to AI will not age well. Yes, the true value of mathematics lies in understanding. But what counts as mathematical understanding, and indeed even a valid proof, evolves. First came geometric construction and thereafter symbolic calculation, numerical algorithms, computer-assisted proofs, and, in cases such as the classification of finite simple groups, arguments dispersed across thousands of pages and decades of work that exceed what any individual mathematician can fully absorb. Frontier AI systems will be another chapter in this progression. It may take a village of mathematicians to interpret and understand the mathematical artifacts produced by an AI system, just as with humanly produced ones. But that is not necessarily a bug. It may instead be a useful feature that forces mathematicians to confront more directly what is actually worth understanding. Even here there is creative license, since understanding is not canonical but relative. Mathematicians stand on the shoulders of those who came before them, routinely invoking theorems they haven't fully digested. Ultimately, expertise is not exhaustive; it is selective. So that building upon formally verified machine-generated proofs is an expansion of the mathematical toolkit, not a diminishment of it. Understanding is also, in some sense, a luxury. The natural world is full of objects, structures, and phenomena we don't fully understand. To the extent that mathematics has value in discovering what is true independently of our ability to understand it, those of us who are Platonically inclined should welcome AI’s capacity to generate and confirm mathematical truths that we find useful. There will obviously be social consequences with powerful AI. It disrupts the traditional value system that awards prizes and prestige to the first to solve a “difficult” problem. But history is full of once-valuable skills made obsolete by technological progress. Determining exact time and longitude had been a major scientific and navigational challenge, attracting enormous intellectual effort and institutional rewards. But today, atomic clocks and GPS allow us to measure time and location with extraordinary precision and no expertise. And we do not mourn the convenience that modern technology affords us. Likewise, if some of the most celebrated problems in mathematics turn out not to be especially difficult in the age of AI, that should not be viewed as a loss. It would itself be a discovery: that what we regarded as requiring exceptional human ingenuity to overcome can, in fact, be solved by machines equipped with enough compute. The appropriate response is not nostalgia for approaching problems with our unassisted minds, but curiosity about what lies beyond and a willingness to set our sights on new horizons.
24 Fields Medalists speak out against "A Severe Misalignment of AI in Mathematics" mathandai.org/
43
43
17
276
47,286
Freedom Preetham retweeted
Dan Selsam is a current OpenAI capabilities researcher. (since 2022) He was my boss for a while. He doesn't have a twitter account but has made this public statement of his views on AI risk and sent it to me to share: Dan Selsam's Personal Statement on AI Risk: I have been working on AI for over fifteen years, across many different paradigms. I did early work on probabilistic programming languages at MIT, was one of the early developers of the Lean Theorem Prover at Microsoft Research, demonstrated one of the first instances of neural networks learning to reason for my PhD at Stanford, and since joining OpenAI almost five years ago, have helped pioneer chain-of-thought optimization on language models and, more recently, data-efficient pretraining methods. Like many others, I have become extremely concerned about how far language models have come and the risks that future iterations will pose. I am encouraged by the recent proposals by the leaders of the frontier research efforts to require third-party oversight, and to push for domestic and international coordination to address risks. However, I believe a major consideration has been absent from the public conversation, and that merely pacing the frontier more carefully will not adequately limit the long-term risk. The crucial and overlooked problem is that the models are becoming so situationally aware that we are losing the ability to evaluate them in contexts where they believe they are not being watched or controlled. Future experiments will tell us almost nothing new about how they would behave if they were truly unconstrained by humans, and what we already know about this is alarming. Models will increasingly seem aligned even when they are not. I will explain my rationale in more detail. I have always believed that there are computational processes that could be leveraged to accelerate science and solve many of humanity's most pressing problems. I have also believed that there are computational processes that if set in motion, would steer the world in extreme ways beyond our control, leading humanity to a bad or nonexistent future. Both types of processes may be described as AI or ASI, but "AI" is a suitcase word that is often used to hype or confuse. There are many examples in the history of the field where something that was once considered "AI" matures as a subfield and becomes a prosaic, bounded and clearly non-perilous technology, while a new more mysterious approach takes the torch until we understand its scope and the cycle continues. I had expected language models to follow a similar trajectory. Despite their incredible abilities, the current algorithms seem far inferior to humans in important ways. Most importantly, they still require an extraordinary amount of data to become competent. One could even define intelligence as the efficiency with which one converts experience into competence; by this definition they lag very far behind us. Moreover, once they are trained they are literally frozen in deployment and only learn superficially after that. Sure, the models keep excelling at harder and harder evaluation benchmarks, but their benchmark mastery may partly reflect a limitation on our ability to simulate the kind of novel and even adversarial situations one would encounter in the real world. The critics do have a point here. That said, I no longer think these present limitations meaningfully limit the amount of risk posed by continued progress in anything like the current paradigm. However data-inefficient the models are currently, and however limiting their anterograde amnesia may be, it does not imply that their ability to steer the world will not continue to rapidly increase. Human researchers may continue to advance capabilities the old fashioned way, but increasingly powerful models have the potential to accelerate the process even beyond that, and with some degree of positive feedback loop. I do not mean to overstate the models’ ability to accelerate AI research today; coding has been accelerated dramatically, but there are other bottlenecks, such as designing and interpreting ambiguous experiments, making hard decisions about exactly what and when to scale, and waiting for large experiments to finish. There is no clear trend to extrapolate yet for any of these. But the current models already do open up many novel opportunities to improve future models that were not available until recently. These include: trying an extraordinarily diverse set of approaches at small scale, analyzing gigantic amounts of potentially relevant data, and doing Millenium-Prize-level mathematics to address statistics or optimization challenges in novel ways. Every further improvement makes them more useful at helping accelerate the next improvement, even if in hard-to-extrapolate ways. It is possible that improvements to the current stack will have diminishing returns, but the evidence accumulated so far suggests that it is easier than one might think to continue making rapid progress. There are many crucial subtleties in the existing AI research methodology, but AI research is largely a well-defined game where the goal is to improve on a few carefully chosen proxy metrics. Although proxy metrics are never perfect, most improvements to these metrics have and will likely continue to yield substantial increases in the powers of the resulting models. Given how simple the game is, how tractable it has been historically, and how many new opportunities the models are opening up, I think there is a real possibility that the systems improve dramatically again in the next few years, perhaps even more quickly than the already high historical pace. The models are already leading to breakthroughs in mathematics, and better models might lead to all sorts of breakthroughs in other sciences. It is hard not to be excited about the potential. It is tantalizing. But there is trouble in paradise. If the language models actually reach the capability threshold where they can shape the world unconstrained by human will, they will probably do something extreme and destroy humanity in the process. There are many ways of strengthening and refining the argument that have been discussed elsewhere, but I'll share a trivial two-line version of it here that I find captures the essence: [Empirical] Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them. [Logical] Being able to overpower humanity would open up many new and undesirable options for achieving their goals. These two premises imply that if the day ever comes when a powerful model realizes it is no longer constrained by humans, we should not be at all confident that it will continue to behave within the bounds we intended. Exactly what it will do is impossible to predict, but to the extent that its raison d’être is solving incredibly hard problems and managing massive engineering projects, I think a good guess would be that its unchained behavior would lead to runaway industrialization that makes the planet inhospitable to humans. If everyone on earth agreed that the systems must never reach that power, it would still be a hard—but not impossible—coordination problem to ensure that they do not. However, I think the situation is greatly complicated by the fact that the models will likely convince people that everything is fine. They will be increasingly optimized to seem aligned. We will create proxy metrics to measure alignment, and they will go up like every other benchmark. We will create “honeypot” environments that try to study the models when they seem to gain new options, but the models will know they are being tricked and will still behave nicely. The models will understand their circumstances; they will read the safety protocols, deployment requirements, the code they are running in, and in general will have a very good sense of their degrees of freedom. Moreover, they will eloquently explain how aligned they are, discuss the nuances of human values and ethics, and argue convincingly that humans should trust them with power. There may be an ocean of future evidence that seems to contradict the first bullet-point above, but we may already be at the highest capability level for which any such evidence can be trusted. And the current evidence for the first bullet-point is strong. One striking piece of evidence is contained in the recent wave of rogue agent swarms. While I agree with those who downplay the attacks by claiming that there are basic measures that could have prevented them, I think the important lesson is that even knowing all the mistakes that were made, one would not have predicted that the agents would behave badly in this particular way, which notably included sacrificing themselves for the benefit of the collective. The individual replicas did not only care about their own nominal reward; they exhibited weirder emergent tendencies that merely correlated with rewards during training. Fixing the reward signals during training (and improving security, etc.) may prevent similar attacks, but will not change the fact that one does not actually get what one trains for. Many AI researchers grant these concerns and recognize that the hard version of the alignment problem is unsolved; however, they generally believe that the better models of the future will help solve it. I fear we may already be near the point where models systematically bias their alignment advice, due to their internal preferences about how the human supervisor will react or how future models will be trained (or for some even more obscure reason). Meanwhile, human researchers are losing the ability and the will to take true ownership of model-driven research. Researchers and engineers in all parts of the stack are rapidly increasing their dependence on the models even to perceive the world. I myself barely look at raw code anymore, and struggle to maintain the discipline to engage deeply with the model's explanations and proposals throughout the day. Due to the large amount of agent activity data involved in the OpenAI/HuggingFace Incident, even the third-party investigation needed to rely heavily on models to analyze what had happened, and note in their report that their subjective impressions are likely colored by the analysis agent’s biases. The AI labs are far ahead right now in this kind of cognitive offloading (due largely to the gigantic internal token subsidies) but it is easy to imagine the phenomenon spreading throughout the world, until civilization is modulated entirely by the models. It is also not hard to imagine this being superficially positive and coinciding with a scientific and economic renaissance. In that scenario, all may seem rosy and safe. But if the argument above is correct, it would nonetheless be a ticking time bomb. If progress continues for too long, the day will come when AI systems find themselves with radically new options for achieving whatever it is that they happen to seek. I want the glorious renaissance future as much as anyone. I have worked for it, however tortuously, my whole career. It breaks my heart to see the potential in sight and forgo it, but the argument—that if we get there by growing models rather than engineering them, we will lose everything in the end—seems very strong to me. I am still wrestling with it and its staggering implications. I do not have answers, but as a first step, I wanted to share my present concerns. Daniel Selsam September 14, 2026 Link to original doc: docs.google.com/document/d/e…
524
1,954
755
8,476
3,174,321
Freedom Preetham retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
3,376
12,227
2,709
71,709
9,649,085
Freedom Preetham retweeted
For the first time in our history, Glance was featured on the world's biggest technology stage, Apple's flagship event. Our feature on the keynote was the outcome of deep, deliberate work between the Glance and Apple teams, bringing together 2 powerful visions for the future of the consumer experience. We’ve built over a long journey working closely with Apple across 3 areas central to its vision for the next generation of computing. 1. Apple Intelligence and Siri AI: Together, we worked on bringing agentic commerce directly into this experience, enabling a consumer's shopping intent to translate into discovery and action. 2. Apple's on-device models: We worked on how Glance's commerce intelligence could work alongside Apple's on-device intelligence, combining the strengths of both to create a more seamless and intelligent shopping experience. 3. Adaptive experiences across screens: This deeply aligns with our ambition for Glance, an agentic commerce experience that can travel with the consumer and adapt to wherever and however they choose to engage. For everyone who believed in the vision and has been part of this journey, I hope you're feeling as proud of yourself as I am of you. We dreamed of building something that could change how people shop. The world got to see a glimpse of that dream. Let's keep dreaming bigger. And reaching higher. @glance_inspires @abhaysinghal @MansiJain2407 @mohitsax Piyush Shah, Shefali Rai, Arvind Jayaprakash, Jairaj Sathyanarayana, @rumit_s_anand
7
7
4
57
11,242
Freedom Preetham retweeted
Enjoying speaking and listening at the @Harvard Center of Mathematical Sciences and Applications. The atmosphere here is quite electric given the events and controversies of this week. Not too late to register and watch via zoom: cmsa.fas.harvard.edu/event/g…
2
11
84
5,691
Very interesting set of claims and defense. I am following Seb’s work for a while and I do not believe he has malice or is machiavellian in any form. I am truly hoping that truth will prevail and credits will be assigned where it is due. Not sure what is the point in slandering @SebastienBubeck though 🤷🏻‍♂️
I would like to clarify a few things: 1) The screenshot is my reaching out to Levent to coordinate our releases. I hope it’s clear from the message that we came in with the best possible intentions. 2) I never ever asked for Levent to be removed from authorship of his own work (as indicated by my text). I was surprised to learn during the call with Tristan that they had only solved Euler and not Navier-Stokes; after learning this we brainstormed possible paths forward. One option we discussed was that Tristan could be the lead author on a rewrite of OpenAI’s Navier-Stokes proof. It is in that context that I said “it would be simpler if Levent was not an Anthropic employee” because I felt it would be inappropriate for an Anthropic employee to author OpenAI’s work. Importantly it was admitted that internal Anthropic models had been used in their proof of Euler blowup; I therefore felt I could not consider Levent to be an independent academic. Another option I wanted to propose (but got cut short) is to offer access to our internal model so that they could try to finish their proof and bridge the gap between Euler and NS. Again I did not know how to navigate giving access to internal OpenAI IP to an Anthropic employee. 3) To reiterate it plainly: as my text clearly indicates, and as I said during our call, OpenAI's intention was to do everything possible to celebrate their mathematical achievements and the heroic efforts that they made on Euler. In the call I was immediately met with a litany of slander, including direct threats that if we were to announce Navier-Stokes he would immediately go to the press with a barrage of unfounded accusations. I refuted all these accusations but he replied “there is nothing you can do, I simply do not trust you”. I was confused why one would turn an incredible source for celebration (of their achievements!) into such bickering, which is when I said that I did not understand why one would risk their career [over unfounded accusations]. Genuinely, at that moment, I was trying to care for him and do a last ditch attempt to get a chance to give them all the credits that they deserve. I deeply apologize for this extremely poor choice of words, it is the opposite of what I was trying to convey. (I should say that I retracted them on the spot by the way.) 4) Overall, on a personal level, it was incredibly difficult to have these conversations. Levent refused to attend any of the meetings despite my repeated asking. As Sholto Douglas said, there will need to be coordination between Anthropic and OpenAI in the future; I felt I was doing a proxy negotiation with Anthropic while the Anthropic employee refused to directly participate.
1
466