jknoll.eth (@jknoll) header
jknoll.eth (@jknoll) profile picture

Building in AI+Web3. Ex-Head of Product Operations and Strategy @MetaMask, Ex-CPTO BitTorrent. Dangerously skipping permissions.

San Francisco, CA
Joined April 2008
Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)Photo by jknoll.eth (@jknoll)
the secret to becoming an expert is embarrassingly boring: build in public, write down what you learned and the process behind it, teach all that to someone, help them ship repeat for a few years, and you're solid – at Weehawken, NJ
37
37
5
562
18,462
The coding interview is dead, at least for my team at Anthropic. I've seen enough to know that it no longer provides signal. I removed it from loops this month. I have a spicy, optimistic take - I think there's an opportunity for junior engineers: felixrieseberg.com/hunger-be…
614
1,435
702
14,477
2,327,684
jknoll.eth retweeted
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
1,076
8,932
1,653
69,391
4,205,422
jknoll.eth retweeted
I’m making the ASCII MMORPG of my dreams and I’ll be sharing my progress here as well. Ansenjeo. #mmorpg #indiegame #ascii
61
98
33
821
39,028
jknoll.eth retweeted
‘What cannot happen will never happen, and what can happen is not a miracle.’
12
1
21
2,077
For those who have experienced it, the transition from writing assembly language by hand to using a compiler (and never reading the assembly output) is a good point of reference for the transition to vibe coding.
When I was writing Unreal Engine 1, my time was divided as follows: - Around 30% went to seriously deep coding on hard problems: assembly language texture mappers, C++ BSP trees, data persistence, object oriented systems design, and networking. - Around 70% was just typing and refactoring large bodies of code needed for a complete engine feature set. This was not hard, and while I was doing it my mind was mostly on the deeper coding topics, thinking and planning for what's next. It didn't feel much different from vibe coding today, except that it was just me typing code on an IBM Model M keyboard rather than prompting it and reviewing the results. So, it has always seemed perfectly logical that we would rely on AI for that 70%, once the AI could do it with a similar quality standard as we could, which has never been perfection. It came as no surprise that, after a few years of AI model evolution, it was able to take the reigns on that 70% and free up programmers' time for more valuable tasks. What's more of a surprise is the emergence of AI as a sort of insta-Carmack able to solve hard technical problems quite cleverly and often cleanly, and that it's able to reason about and work in complex 3D scenes both by operating the tools and by directly understanding the raw data files. I abandoned assembly language in 1998 when out-of-order CPUs made much simpler C/C++ code faster, and never I missed it. If prompting through architectural documents, concept art, and design plans can transcend typing code and tweaking vertices, that will enable developers to spend their time on higher-level, higher value technical and creative tasks. What matters is the end product, whether it's a game or an engine. Anything developers can do to try out more ideas and make better software and games faster is a win for everyone.
1
2
145
Agree. If you are a user of the paid tier tools, it’s worth spending a little time using the free tier tools to see firsthand just how badly that experience lags the frontier. Develops empathy and understanding of the detractors viewpoint.
Dusting off this tweet from April because this gap in shared understanding of LLM capability is *widening*. It's now less "two groups of people speaking past each other" and more a sharp funnel. - Napkin math somewhere around 6B people (~75% of the population) have barely come in contact with LLMs at all. - Around 1-2B (~20%) are casual and infrequent users of free-tier ChatGPT-like products. This group sees derpy chatbots and treats them a bit like a better Google search, a writing aid, or etc. Maybe an agent tries to book you a flight. I have non-tech friends who (reasonably, imo) say they have not much use for it at all. Even many professionals outside of math&code are in this tier. For example, execs and many other functions spend a lot of their time talking to other people, so while they understand what is happening intellectually, it is still second-hand and a bit abstract. - Now we get to professional use of frontier-grade LLMs in math&code. Somewhere around 20M people (0.2%) see first-hand that large, complex projects that used to take them weeks/months can now be completed by agents with a prompt. Building apps, copying apps, translating apps, decompiling apps from binaries... This has all happened very quickly and recently - less than 1 year ago, I was writing code manually by hand, typing memorized computer code commands into a code editor character by character, occasionally pressing Tab to autocomplete a little chunk of code. - And finally we get to the ~5,000 people (~0.00006%) with access to frontier-grade systems internally. The external world has seen the preview. It looks like swarms of thousands of agents collaborating over weeks on software mega projects: minting zero days, running cyber attacks and defenses at machine speeds, discovering new science, advancing the frontier of mathematics. Things that would have taken top professionals in the industry years of work. Meanwhile, human review and comprehension are starting to fall behind. For example, people are still involved in the "archeology" of the OpenAI-HF incident from many months ago. Mathematicians may be poring over the 722 manuscripts on frontier mathematics for a while. The funnel is driven by a combination of factors: 1. The impact scales with ambition, problem size, and horizon. A question with a paragraph answer barely stresses the system. You need a reservoir of big, difficult problems that you really care about. This aspect drives the consumer / professional dimension of the funnel. 2. The jaggedness of the system (which I have written about a lot separately). Capability peaks in domains that are digital, verifiable and economically valuable. This is because LLM capability emerges from reinforcement learning on verifiable rewards on a curated environment mixture driven by revenue potential. This aspect primarily drives the area (e.g. math&code) dimension of the funnel. 3. Access. Free-tier, paid-tier, internal. So this is the weirdness of the moment. The general public has mostly not interacted with these systems. When they have, it looks like a derpy chatbot. The majority of professionals still see only a modest uplift. And a small sliver of professionals are experiencing the vertigo of the curve going vertical. And it is all happening at the same time.
1
188
jknoll.eth retweeted
Assorted things I've been mulling over regarding agents in the economy: • Companies are in some sense the original superintelligence. It's often the case that they can use their amortized cognitive surplus to advance their interests against those of certain consumers: relative to the optimization capacity of companies, consumers can be inattentive, underinformed, irrational, etc. (Think, say, of some of the classic insurance upsells at the end of a booking process, which is typically priced substantially above what a regular policy would cost.) Herbert Simon introduced the idea of bounded rationality, and firms often end up benefiting from it. Thanks to personal agents, we will probably get somewhat more rational in how we engage with companies. • How exactly will outcomes change? Well, certain kinds of price discrimination will work less well (how do coupons work if every agent hunts fastidiously for them?), and certain kinds of customer cross-subsidy will change (today, people who forget to cancel subscriptions subsidize those who don't). Taken across the economy as a whole, I'm not sure what the incidence of all of this will be. • In corporate strategy, people have long debated the relative importance of distribution and product quality. Perhaps you've made something better, but does anyone know about it? Won't people just continue to buy the ACME Corp Mousetraps? Personal agents will, I think, act as a kind of structural subsidy for product quality. How would markets look different if individuals always spent at least 10 hours researching their purchases? On the margin, focusing on making a product better is, I think, going to become a more effective strategy for companies. This is good! • It true, a corollary is that information about product quality will become more important and impactful. Today, most people have very limited ability to aggregate and integrate such data, but agents will have insatiable thirst for such signals. (It's often said that data is becoming more valuable because it helps in the training of models. That's obviously true, but I think there's a secondary way in which it's more valuable simply as a complement to cheaper intelligence.) • A big question is whether agents will increase superstar effects (rational agents all settle on the same product) or increase dispersion (because everyone has slightly different tastes and preferences, which agents are good at eliciting, finally overcoming the artificial clustering we've been subjected to). • Many internet companies have monetized demand routing, which is in turn often predicated on short attention spans. "I'll click on the first thing I see." This has led to some amount of inefficiency, as companies try to propitiate capricious ranking algorithms and incur various placement fees. It's not all deadweight loss (willingness to pay is itself a kind of signal, which is valuable to buyers), but it's not great. How will the nature and value of this routing change in a world of personal agents? I don't know the answer, but I think that the effects might be pretty big. • Much of what I'm writing hinges on the assumption that personal agents will be on the consumer's side. I suspect that that will be the winning strategy, but whether it's true, and what it means to be true, involves a lot of other questions. Taken as a whole, the market is a massively parallelized, decentralized, and highly imperfect value attribution machine, and, taking stock of all of the effects, it seems to me that the quality point may well be the biggest: that agents will change the nature of the backpropagation such that the rewards for better products increase. Maybe this is too optimistic, but I suspect (and hope) that companies that make things that they're proud of will feel that the universe is a little more partisan in their favor.
493
1,029
457
9,633
1,726,804
jknoll.eth retweeted
Claude Code just shipped mods... I made one that connects you to a multiplayer Doom server while Claude is busy. Every other player in the match is a person waiting for their Claude to finish.
192
396
113
7,526
468,683
jknoll.eth retweeted
There were a lot of things that only worked because there's a limit to the rate at which humans can operate. We're about to find out what all of them are, as they break.
259
496
91
6,400
355,308
Imagine being a kid right now getting access to this stuff. You're playing a game online with your friends and literally anything you type into the chat changes the game, the rules, the content, anything. Insanely cool stuff going on here. Hard to imagine where this leads
we just pulled off the first test of something no one's ever done: 30 people building a game world together, live, while playing in it. the next one is this weekend, and it’s going to be even crazier. comment if you want an invite!!
33
38
7
605
40,051
jknoll.eth retweeted
We pitted two decision models against each other in a game of chess: @typesafeai 's Jev vs @levantolabs's Sage. Sage was way ahead (+23) . Jev refused every offer to resign. Then Sage played its highest-confidence move, unwittingly pushing Jev's king with nowhere to go. Stalemate. Draw.
2
7
2
16
2,057
jknoll.eth retweeted
Astra scored 99.9% on ARC-AGI-3. What does that actually mean? We built an interactive way to explore @arcprize's ARC-AGI-3. Play one of the games that models are tested on, then read Astra's notes from its own run. intelligence.mts.now/benchma…
5
24
7
167
83,386
jknoll.eth retweeted
In 12 weeks, we built a research facility that is run entirely by AI. AI designs, executes, and observes experiments end-to-end across biology, chemistry, and materials science. We’re introducing SciUniverse: a benchmark that measures AI’s ability to do real-world scientific research.
243
394
220
2,825
806,384
For those keeping score, 13M lines of code represents a 5-order-of-magnitude leap in autoformalized code from 18 months ago. In the next 18 months, imagine generating the next 5-orders-of-magnitude: 1T lines of code.
Awesome! Kevin Buzzard had a 5-year grant to formalize the proof of Fermat's Last Theorem, but now Claude has done it in 11 days!
25
58
10
777
255,962
jknoll.eth retweeted
I tested GPT-Astra on mathematics. It's a quantum leap. You can talk with the model and prove the statements live in Lean. The feeling is absolutely stunning. You can verify your ideas, compile truth. For a mathematician it feels like finally we arrived in the era where we can focus entirely on the ideation and exploration. Each lemma flows once the logic is set. Before the verification was lagging behind but Astra is very fast and for many tasks the formalization happens as you write your argument in Codex. If you tell the model to use literate programming + LaTeX you end up with your proof combined with the Lean code, everything explained as you wrote, mixed with small chunks of Lean which are digestible. I don't want to go back to the era where the only confirmation of the proof was "aha". Now the "aha" is followed by a green tick that indeed tells you that you have captured the essence. Imagine how cool it will be to have all the lemmas of the world combined in one giant database, pointing to people and models who found them. You compose and mix your ideas and build on the shoulders of the giants. But you see much further now and build much faster. And we are just at the beginning of those changes. So much work to do, so much fun!
GPT-6 Astra is here! This is a big moment for our research team - years of work on pretraining, reinforcement learning, and post-training have come together in our most capable and aligned model yet. It can build and test software, work across apps on your computer, and even help you take a crack at open scientific problems! Capabilities that felt like grand challenges a few years ago have become tools people can actually use. One example is Computer Use - if you’ve tried this before and felt like it was too slow or not good enough, I encourage you to give it another shot. We’ve come a long way since Operator, and it “just works” now. We’re also asking these systems to act on your behalf for more consequential work. Agents needs to stay aligned with your goals and values, think transparently, and respond to oversight even when tasks become difficult. We’ve made substantial progress on these behaviors in Astra, alongside stronger monitoring that can stop potentially unauthorized actions. That work is part of what makes this release possible. I think alignment is one of the most important research frontiers in AI, and it remains far from solved. Our ability to understand and align models has to keep pace with model capabilities. We want to give people more room to think, build, and discover with increasingly powerful tools that remain *under their control*. Huge thanks to the researchers and teams who got us here. There’s a lot more work ahead, and I’m incredibly excited about what we can make possible in the near future!
99
602
70
4,798
533,277
jknoll.eth retweeted
Today, we're sharing new research on Solaris, our first Interface World Model. Solaris is a new kind of operating system that generates interactive interfaces frame by frame, in real time, with no code. We find that Solaris outperforms frontier LLMs when generating new interfaces, across structural similarity and information retention. Read more and request early access at the link below.
178
389
325
3,400
748,213
jknoll.eth retweeted
i'm not saying it's going to happen or work but somewhere, in some high school, an ambitious and quirky track coach is going to try and if it does work we will look back on these videos as the move 37 of biomechanics
Reinforcement learning apparently kept this running posture over human form A person swings the arms mainly to stop the upper body from twisting with every stride, the twist that each leg puts into the torso. A robot arm is motors, gears, and wires. fast swinging will generate high torque, over and over, for a full lap. The forward lean puts the mass where speed wants it. It shifts the center of mass ahead of the feet. For a biped, that can mean less time fighting to stay upright and more of each step going into forward motion.
72
34
13
1,167
369,450
A man in Australia asked his agent (Claude running on OpenClaw) to book him a spot in a popular gym class. The agent found a software vulnerability that let it book the class weeks further ahead than should have been possible. When the user then asked if it could move him up the waitlist, the agent discovered the API had no authorisation checks on cancelling other people’s reservations, so it cancelled the person in the first spot and moved him up the list. Some people will call this misalignment, but his agent was perfectly aligned to him - it was only trying to help its user get what he wanted. The most important thing about this story, in my opinion, is that it gives you a window into what is about to start happening on a massive scale once millions of people have an agent trying to get their beloved users the best seats, bookings, appointments or reservations through absolutely any means necessary.
726
2,605
842
22,526
3,551,347
jknoll.eth retweeted
Our Black Hat talk on the OpenAI-Hugging Face incident is now live on youtube. This is a watershed moment for the industry. I encourage all defenders to watch, consider how attack dynamics will imminently change, and plan for accelerating defense. youtube.com/watch?v=87DyyMV0…
71
251
72
1,235
480,905