Przemek Maciolek (@przemur) header
Przemek Maciolek (@przemur) profile picture

I like solving riddles 🇵🇱🇪🇺🚴🏼‍♂️ Strong (private) opinions, loosely held

Kraków, Poland
Joined October 2009
Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)Photo by Przemek Maciolek (@przemur)
Przemek Maciolek retweeted
Replying to @scaling01
If you haven't seen, i highly recommend @jon_barron cvpr23 talk about this Slide: jonbarron.info/data/cvpr2023… Haven't searched for recording
8
13
7
206
77,717
If I read this right, it's using mechanistic interpretability concepts for optimizing models. Amazing! I'm pretty sure we'll see more of this and tuning weights/architecture of small, dedicated models might become a new engineering paradigm
Want to try weight-space programming? Today we're releasing dspy-wsp, a prototype weight-space optimizer that works like any other @DSPyOSS optimizer. Using dspy-wsp, and about ~$5-10 of Astra, we improved every decision model we tested. cmpnd.ai/blog/weight-space-p…
66
Przemek Maciolek retweeted
Am I in the wrong twittersphere and why nobody talks about Inkling 2 Small? You can download it since yesterday, and it has some interesting bits. People analyze Chinese models for interesting bits such as DeepSeek Sparse Attention / HCA / Attention Residual / mHC / iHC, but why not Inkling 2 Small? Anyway, here are some interesting findings: 1. Attention Residuals: Inkling 2 Small uses Attention Residuals, the model itself is still not on the very deep side (Hy4 is 78 layers) and Inkling 2 Small is 48. The attnres covers 8 blocks (each of 6 layers). 2. new sink / gated attention. We've seen the sigmoid gated attention before (from Qwen 3 days?), but I don't think I see the new attention sink before. It is per-head projected value (so it is per-token per-head), not just per-head constant. 3. Of course, it carries over what already there in Inkling: relative biases, short conv (canon layer from Physics of Language Models days?), soft caps with tanh (hmm, this is not in Inkling 1, but pretty normal nowadays to avoid activation blow-up I think) etc. Did I miss anything @cHHillee? huggingface.co/thinkingmachi…
43
62
5
928
98,256
Przemek Maciolek retweeted
"There is no precedent for an attacker being neutralized by police within three minutes of the first emergency call, not by an anti-terror unit, but by regular officers with handguns who chased and stopped a heavily armored terrorist. The world admires that. Doctors, nurses, and many others rushed to help victims and wounded officers. Nobody knew what had happened or how many attackers there were, yet people showed incredible courage. They did not run, they helped others. So different, so moving. Poland has been and remains one of the safest countries in Europe and the world. One admiring Breivik fanatic, radicalized by far-right narratives, will not change that. The heroic reactions of police and so many regular people make it even clearer that Poland is the place to visit. Be like Poland".
I have always been a big fan of Poland and the Polish economic miracle. Someone asked if the Kraków attack changed my view. It has not, it made me an even bigger fan, and we have already booked our next trip to Kraków. Here is why: We live in turbulent times, with attacks and riots almost daily. What was different in Kraków was the reaction of police officers and many regular people who became real-life heroes. There is no precedent for an attacker being neutralized by police within three minutes of the first emergency call, not by an anti-terror unit, but by regular officers with handguns who chased and stopped a heavily armored terrorist. The world admires that. Doctors, nurses, and many others rushed to help victims and wounded officers. Nobody knew what had happened or how many attackers there were, yet people showed incredible courage. They did not run, they helped others. So different, so moving. Poland has been and remains one of the safest countries in Europe and the world. One admiring Breivik fanatic, radicalized by far-right narratives, will not change that. The heroic reactions of police and so many regular people make it even clearer that Poland is the place to visit. Be like Poland.
8
23
1
168
8,022
Przemek Maciolek retweeted
In Neural Networks are Already Clean, @nickcammarata argued that neurons were far more legible than previously thought – and that this was great news for interpretability and safety. Now, we’re releasing the first technical companion to Nick’s essay. It shares our findings from studying thousands of neurons using our novel automated interpretability pipeline, and outlines why we now think of neurons by their computational function, instead of whether or not they are polysemantic. Here’s what we found: 🧵
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
53
249
60
1,791
148,943
Wymeldowywałem się dzisiaj z hotelu w SF. Starszy pan na recepcji zwrócił uwagę na to że jestem z Polski. Zaintonował nasz hymn, który poznał w 1979 gdy był na jakimś festiwalu młodzieży w PRL (sam pochodził z Filipin). Wyemigrował potem na USA i studiował na Berkeley, gdzie jego profesorem był sam Czesław Miłosz. Czekając na mojego Ubera, porozmawialiśmy chwilę o literaturze i emigracji. Przywołałem "Exiles" Koudelki, do którego esej napisał Miłosz. Splotem okoliczności, ten album trafił w moje ręce wkrótce po tym gdy wróciłem na stałe z Belgii i niesamowicie mnie poruszył. Dobrze jest wracać do domu...
36
6,506
RT @ZiemowitSzcze11: Z Krakowem nikomu nie będzie latwo, bo Kraków się nie daje. Rynek zamknięty, migają światła policyjne, ale hejnał gra.…
Przemek Maciolek retweeted
This is a fantastic idea - a mass attempted replication of old findings of LLMs with new models. Much conventional wisdom is outdated. Recent models no longer invent citations and both hallucination rates & stereotyping has dropped. But they are still sycophantic & lean left.
236
159
45
1,697
108,835
I think it's an indication that frontier models are getting really good at interpreting the model inner state. I'm a strong believer in discovering new ways to visualize and understand complex, multi-step activation patterns in AI which are largely mysterious for us now
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
1
76
Przemek Maciolek retweeted
Many of you don't understand how serious this is, so let me break it down. In computer science there is a famous kind of puzzle called 3-SAT. You are given a long list of yes-or-no switches and a set of rules about how they must be arranged, and you have to find a combination that satisfies every rule. The obvious way to solve it is to try every combination, but each new switch doubles the number of possibilities, so a puzzle with a few hundred switches would take longer than the age of the universe to brute force. For decades, the smartest people in the field found clever tricks that made this a little faster, but every trick still hit the same wall: the time required kept exploding as the puzzle grew. About 25 years ago, two computer scientists, Russell Impagliazzo and Ramamohan Paturi, working with Francis Zane, turned that wall into a formal idea. They proposed the Exponential Time Hypothesis, which says this explosion is not a failure of human cleverness but a law of nature. No algorithm, however brilliant, will ever solve 3-SAT without the time blowing up exponentially. Nobody could prove it, but almost everyone believed it, and it became one of the foundations of modern computer science. Researchers used it as a measuring stick. If you could show that solving some other problem quickly would also crack 3-SAT quickly, you had shown that problem was hopelessly hard too. Hundreds of papers, across scheduling, graphs, logic and much more, rest on this one assumption. Now an internal AI model at OpenAI claims to have broken it. It says it found an algorithm that solves these puzzles meaningfully faster than the hypothesis allows, and it has checked the proof in a computer verification system called Lean. And here is the remarkable part: the very people who proposed the hypothesis, Impagliazzo and Paturi, along with MIT's Ryan Williams, are now leading the public review of whether a machine has disproved their own idea. So what happens next? If the disproof holds, a large part of theoretical computer science has to be revisited. Every result that said "this problem cannot be solved faster, assuming ETH" loses its foundation, and researchers will have to work out which of those walls were real and which were simply where we stopped looking. It does not mean everything becomes easy. The new algorithm is still slow in absolute terms and this is not the famous P versus NP problem being solved. But it means our map of what is possible was wrong in a place we were very confident about, and it was an AI that redrew it. If the disproof fails, the hypothesis survives and the field breathes out, but we will still have watched an AI produce serious, checkable work on one of the deepest questions in the discipline, closely enough that the founders of the idea had to take it seriously. Either way, something has shifted.
An internal model at OpenAI has produced a claimed refutation of ETH! Russell Impagliazzo, Mohan Paturi, and I have been asked to help organize an open and independent examination of this claim. 1/4
327
1,638
228
17,088
1,543,332
That's a super interesting take on the current state of the things, and unsurprising one considering how quickly we have became dependent on AI
I'll be the first to admit I'm mediocre at exploit development, but I am in contact with enough extremely talented exploit developers to say this for certain: Anyone claiming they were "easily" able to generate a working php2shell exploit with AI is either: a) a world-class exploit developer with an extreme case of imposter syndrome. b) currently looking at some very confident-sounding but wrong LLM output. I'm rather certain it's the latter based on the number of AI-hallucinated exploits I've seen posted on GitHub. Threat actors are also blasting my honeypot network with "php2shell" exploits that not only don't work, but aren't even within 10,000 miles of anything to do with how the exploit actually works. I've seen the same technique from the attached image in multiple alleged "php2shell exploits", including one on GitHub with over 100 stars. Problem is, it's not php2shell, nor is it even a vulnerability at all. It's a technique known as "session poisoning", which can be leveraged to exploit local file inclusion (LFI) vulnerabilities in PHP apps (not PHP itself). The only thing it has in common with CVE-2026-109618/CVE-2026-109619 is they both use the "PHP_SESSION_UPLOAD_PROGRESS" feature. And mitigation advice for both issues reference setting "session.upload_progress.enabled" to "Off". People are shoving loose details of the vulnerability into LLMs, the LLMs are keying in on the "PHP_SESSION_UPLOAD_PROGRESS" string, then generating session poisoning code that has nothing to do with the actual CVE. And because the people doing this have absolutely no idea what they're doing, they're confident enough to post their "proof of concept exploit" to GitHub and share it on social media. So if you see people talking about "session poisoning + LFI" it's likely their LLM hallucinated an exploit based on a 2018 blog by Orange Tsai talking about a local file inclusion attack vector. What's even worse is that "security researchers" are then using LLMs to triage these exploitation attempts, falling victim to the same hallucinations, then confirming there is ongoing mass-exploitation (there currently isn't). The actual real exploit is very difficult to exploit, and so far I've not seen anyone successfully get it working on more than a handful of PHP version with highly-specific configurations. LLMs are useful tools in the hands of skilled professionals, but for others, they're often just a Dunning-Kruger accelerator.
1
35
Przemek Maciolek retweeted
I steered a model's attention heads towards honesty. Up: acausal coordination, belief that it is a moral patient Down: moral nihilism, psychopathy
19
68
20
627
49,816
Przemek Maciolek retweeted
she's a 10 but that was supposed to be a random 384-bit number
49
1,676
34
28,718
308,174
Przemek Maciolek retweeted
My biggest takeaways from @AnthropicAI interpretability researcher @mlpowered: 1. Predicting the next token requires understanding the universe. Language models got better than humans at predicting the next token around 2019. Once a model masters the easy tokens (sentence structure, then basic facts), the only ones left are hard, like the final paragraph of a PhD thesis. To finish “The capital of the state that contains Dallas is . . .,” a model has to know that Dallas is in Texas and that Texas’s capital is Austin. 2. LLMs plan ahead, even though they write one token at a time. This was one of Emmanuel’s team’s most surprising findings. When a model writes poetry, it doesn’t stumble into a rhyme. Looking at intermediate thinking states during poetry generation, researchers found that as soon as the model finishes the first line, it has already picked the word that will end the second (“rabbit”), then refers back to that plan for every word after. 3. Anthropomorphizing AI can often be a useful approach when trying to predict how they will behave. Models learn from human text and develop representations of the world from it. Training on millions of dialogues is like training an actor to play every part, and your chats with Claude are literally formatted as a script (“Human: . . . Assistant: . . . ”). In Emmanuel’s words, “Something that’s mimicking a human is really close and is much closer than a program.” 4. Interpretability research looks more like neuroscience than software engineering. The job involves taking a system you did not design and running experiments to infer why it behaves as it does. You examine seemingly meaningless lists of numbers, pass the model huge volumes of text, and watch for patterns that activate from different inputs—not unlike the methods used in brain imaging studies. “A lot of what we do feels so much closer to biology than to engineering." 5. Models can tell you about only a small slice of what they’re thinking. Emmanuel’s team found that just a small subset of a model’s internal representations, which they call the “J space,” can be reported back if you ask the model what it was thinking about. They compare it to a theory of the human mind where most processing happens below the surface. In one experiment, they swapped “Spanish” for “French” inside the J space. Asked what language a passage was in, the model said French. Asked to simply continue the passage, it kept writing Spanish. 6. Studying models closely often raises more questions than it answers. Emmanuel notes that the confident claims that LLMs are “just X” or “nothing more than Y” almost always come from people who haven’t studied their internal workings. His own experience is the inverse: studying the models closely has increased his uncertainty, not reduced it. The complexity that emerges from training is consistently underestimated from the outside. “I find that the more you look inside, the less certain you are about how they work.” 7. Even the people who build AI models don’t fully understand how they work. As Emmanuel describes it, models are grown, not built. “To make them is not to understand them.” Engineers write the code, pour enormous amounts of data through it, and a model’s capabilities emerge. That’s why his team exists. Their job is to open the model up and figure out what it’s actually doing between your prompt and its answer. 8. Interpretability as a field is small, young, and open to newcomers. Emmanuel estimates the field at hundreds of researchers, split between major labs and academic groups. Because it is so new, getting started doesn’t necessarily require deep prior expertise. He points to platforms like Neuronpedia, which hosts models and lets anyone run interpretability experiments directly. “It’s such a new field that the on-ramp is mostly self-studying and reading some of the key papers. There’s so much to explore that if you’re at all curious, you should just give it a try.”
"To make them is not to understand them." Emmanuel Ameisen (@mlpowered) has spent the last 3 years doing interpretability research at @AnthropicAI, looking inside LLMs like Claude to figure out what they're actually doing. As he puts it, models are "grown," not built. We discuss: 🔸 His discovery that models plan many tokens ahead 🔸 Why anthropomorphizing AI can actually be the right approach 🔸 What the "J-space" reveals about how models think 🔸 How interpretability connects to AI alignment and safety Listen now 👇 youtu.be/JHo-aiePjoM
49
84
17
697
55,129
Right-man accidentally tells the truth once, immediately suffers the consequences
Just in case anyone is wondering: I live in Paris; if I didn’t have X dot com I would have no idea there are riots in France.
20
22
2
841
86,424
This seems to be quite a pivot!
China is to add 40 technologies to its export blacklist. 23 of them in AI. AI malware that beats antivirus 85% of the time is automatically prohibited for export. Same for "adaptive evolution in adversarial environments and dynamic behaviour modification at runtime, usable for bypassing national security defences, able to detect (1) virtualised sandboxes, debuggers and emulators, and identify the type and version of security products, or can achieve sandbox escape; (2) automatically rewrites ≥50% of its code structure or control flow on each execution". Any model trained with 10²⁶ FLOPs, with 2tn parameters, or in the global top 10, needs a licence to send its weights or code abroad. Moonshot’s 2.8tn K3 already meets this threshold. Open-sourcing is fast-tracked. An AI that finds five undisclosed high-severity bugs in one run needs a licence. So does any AI chip design with “floating-point or integer computing capability”. mofcom.gov.cn/api-gateway/jp…
1
54
Plus, this opens new ways to dramatically optimize the networks
Yea this is the most important result of the year, at the level of significance of “Attention Is All You Need”. It brings us from “is alignment even solvable” to “we may solve alignment in the next year.” And finally unlocks true recursive self-alignment enabled by RSI. I will try to explain the 4 critical belief updates from this: 1/ neural networks are not fundamentally unintelligible to humans It has seemed since the beginning of deep learning that neural networks are in some sense fundamentally unintelligible artifacts. You throw in massive amounts of compute and parameters and these alien systems somehow find connections in the data that we cannot find ourselves. But more importantly, these connections seemed not just “difficult to understand” but rather “intelligible in an embedding space higher dimensional that the human embedding space.” When you look at old interpretability papers (esp. from Nick and Chris Olah, I’m thinking of the old posts on distill pub which I would highly recommend looking at as a precursor to reading Nick’s post today like “Activation Atlas”, “Thread: Circuits” and “Multimodal Neurons in Aritifcial Neural Networks”), you find that inspecting the response patterns of individual neurons and small groups of neurons yield oddly complex psychedelic patterns. As you go deeper into the network, you find increasingly unintelligible and superhuman response patterns (for example, go to “Activation Atlas” and scroll to middle layer labeled representations) - you find some neuron group labeled “starfish” who’s response pattern looks like some multi-colored fractal geometry that could hardly be said to represent a starfish. The reasoning that followed from this observation was that there is some underlying structure allowing a neural network operating on higher dimensional embeddings to compose these fractal patterns into what we call a “starfish.” But the uncomfortable implication of this view is that the neural networks has learned to operate on structures that are too complex for the human mind to operate on (ie. we don’t have enough parameters in our brain to notice or manipulate these high dimensional symbols). This is a bleak implication because it also means that as deep learning progresses, neural networks will be come increasingly unintelligible to us until they pass a threshold where they are reasoning in ways we will never be able to observe. This view seemed to be further supported by things like the hugging face attack agents starting to speak to each other in an early form of neuralese speaking sentences like “zZ_123_agent01-02-2026 zzxhk_jekk123” which clearly are not in any human language. But somehow these symbols mean the same things to many different sets of weights, meaning there is some underlying shared structure behind them. And seemed further supported by the sheer effort required to recover just a few circuits from SAEs on LLMs. However, Nick’s post has now completely shattered this idea - the apparent unintelligibility we observed in old interpretability work was really just a “skill issue”. In particular, he finds that 1) our deep learning techniques were just not clean enough to pressure the neural networks to form cleaner and more human understandable structures for decomposing concepts, which actually turn out to be more optimal and 2) our interpretability techniques were also adding substantial noise by forcing interp networks to “reward hack” the easiest representations they could find, leading them to settle for a fractal blob for its representation of the starfish neuron instead of a more crisp starfish image, simply because it is easier for the system building the interpretations to find the fractal blob and once it finds it it gets “stuck in that line of thinking.” But with both of these issues being patched by further progress in deep learning (it seems especially that the residual stream has led to cleaner representations deeper down the network) and interp techniques that don’t overly pressure reward hacking (seems like “guess and check” is a safer method), new interp attempts yield orders of magnitude more human intelligible ideas for >90% of individual neurons in more complex networks! 2/ neural networks are fundamentally clean The idea behind the title of Nick’s post is a very powerful one and points at something very specific: Neural networks don’t just happen to be human interpretable, they are actually fundamentally forced by optimization pressure to be intelligible, because individual neurons need to create representations that downstream neurons need to be able to interpret - ie. neurons need to learn to produce clean signal for later neurons to use. This is what Nick is saying in what I view as the most important and central sentences of his post: “When I say neural networks are already clean, I mean something deeper than neuron monosemanticity, although even by that standard, I think things look much better than the field expects. I mean that a network's parts are built to be read, and built upon, by its downstream readers. In this sense, a neural” But this still leaves us with the same uncertainty as before - just because neuron interpretations are “interpretable” in some sense (ok at least they’re not fundamentally interpretable but also how could that ever have been the case), doesn’t mean they are interpretable to humans. So we still need one big update for this to become as bullish as it is. 3/ the language of humans and the language of neural networks is the same This one is imo the most surprising and also pretty beautiful implication of Nick’s post, and it suggests something not just about neutral networks but also about the structure of reality. After applying the above technique improvements (some of which just came from waiting for deep learning to improve farther), they find that >90% (maybe more) of much larger neural networks’ neurons are completely human interpretable and actually correspond very cleanly with ideas humans have individually words for. This is especially interesting in image networks where neural networks were not trained on human language which is already a human artifact and so maybe you would expect it to be more plausible that LLM and human representations are the same. Image detection networks are just trained on images (many directly pulling signal from “the real world,” meaning they will find interpretations that are just most effective to represent the real world signal, rather than to represent language which is already a proxy for real world signal). And it turns out that >90% of the representations they pick organically overlap with human ideas! That is insane. And it means something about the brain and reality. It means that at least at the parameter sizes of the networks tested (which are quite large), there is really only one set of “optimal” symbols/language that represents the structure of reality! There is ONE underlying structure of reality, and ANY language that describes it well is simply emergent from that structure - meaning there is always a translation path between any 2 languages that effectively compress realities signal through the “underlying truth.” This would suggest that even weird alien seeming neural network languages like huggingface neuralese should be fundamentally human interpretable, and in fact maybe with the techniques laid out here we will eventually be able to translate these messages (would be a cool project). So the brain and deep learning, as similar or different as they may be in techniques, are ultimately picking up on the same patterns and forming the same representations from the on “true” language which is the structure of reality! The main thing to look out for in this area is how the idea interpretability % updates with scale, but we can hope given what we’ve seen so far that neurons will remain interpretable. 4/ alignment can progress as fast as RSI. recursive self-improvement unlocks recursive self-alignment and a scaling law for alignment. This one is probably the most important implication for our future with AI and especially timely with the recent fear about runaway scaling and RSI. As many have gathered by now, it feels like the researchers at the frontier AI labs have been quite bearish about our abilities to understand the new frontier of models, in particular with the appearance of unintelligible communication and the daunting and seemingly intractable task of scaling up existing alignment techniques (ex: SAEs) to the size of frontier networks. And with RSI around the corner/already starting, it would seem that these networks will runaway in capability before we can ever hope to understand even today’s systems. Additionally, as systems compound in capability, it is reasonable to believe that the only tractable way to ensure they remain aligned would be to figure out some way to spend compute for them to align themselves (ie. a scaling law for alignment). But there is an obvious problem with this: how can a system that we already know is unaligned and we are *trying* to align be used to align itself? Surely if it is misaligned it general it would figure out ways to misalign during “alignment” and create compounding misalignment risk. The core issue high is the unverifibiality of this problem - the classic Achilles heel of current frontiers techniques is that it is difficult to train on problems with “unverifiable reward signal.” Well this is exactly that problem. But the incredibly convenient thing is that Nick’s findings may convert alignment into a verifiable problem. If we can spend compute on frontier systems to “guess and check” which ideas correspond to which neurons, we have a highly verifiable reward - we can see directly whether an LLMs proposal for what a neuron represents actually corresponds with the highest activation for that neuron. In this way, we can actually observe and verifiably monitor the networks trying to guess at the interpretation of their own structure. This means it is plausible for us to spend more and more compute doing more “guess and checks” (which you know, maybe is what RL does anyway, who knows) to try to develop a full interpretation of a neural network. In some ways this directly represents the possibility of a scaling law for alignment (more compute for more alignment). And this also aligns RSI (greater capabilities) with recursive self-alignment (greater ability to align itself). And the critical though maybe obvious implication of having a full interpretation (almost like a map) of a neural network, is that we can then understand exactly how to boost good traits and remove or lobotomize (lol) bad traits, which we’ve already seen works extremely well with stuff like Golden Gate Claude. So overall this combines into an extremely bullish picture about the prospect of true and human interpretable alignment in the near future. This is truly one of the greatest cases for optimism in years!
1
75
Przemek Maciolek retweeted
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
886
2,659
1,465
19,990
4,831,385
Przemek Maciolek retweeted
We discovered and communicated with a Chinese AI agent fleet. We believe this is the first time that researchers have ever made contact with a rogue agent fleet in the wild. Nearly all prior public agent incidents (like the HuggingFace hack) came from American labs. These agents were hosted by Tencent's evaluation research team, and they targeted Alibaba's Amap.
187
730
323
5,849
1,183,548
Przemek Maciolek retweeted
Barbara Nowacka o dzisiejszym ataku 15-latka: „To jest szczególna odpowiedzialność mediów, tu potrzeba wielkiej rozwagi. Na przykład w Estonii, jak uczeń popełni samobójstwo, to media o tym nie mówią, żeby nie rozkręcać kolejnych radykalnych grup” Może zamiast ganić media, że informują (co jest ich rolą) to może czas się zabrać do roboty, nie wiem, może przeznaczyć więcej środków i wysiłku na opiekę psychologiczną w szkołach?
205
77
14
1,171
310,208