Nitarshan (@nitarshan) header
Nitarshan (@nitarshan) profile picture

prev created @aisecurityinst, AI Safety Summit, UK AI Research Resource, EU AI Code of Practice.

San Francisco, California
Joined May 2012
Photo by Nitarshan (@nitarshan)Photo by Nitarshan (@nitarshan)Photo by Nitarshan (@nitarshan)Photo by Nitarshan (@nitarshan)
ultimately it's just very difficult to beat a philosophical/ideological/movement that can inspire Jane Street traders or Google engineers to quit their high-paying jobs in California to work with a bunch of relatively clownish coworkers on the Hill for $60k/yr in 84° DC summers
15
6
2
352
32,931
Nitarshan retweeted
I don’t think you should respond to questions about your credibility by telling your interlocutor not to ask the question, with the implied point being that the underlying nobility of your quest is so unimpeachable that your credibility is without question.
jacob coxon still owns openai shares and admits he'd have more credibility without a financial stake. he responds by saying he likes money, donating it wouldn't advance his goals, and the interviewer could've just not asked the question instead of challenging his credibility
8
21
3
489
29,530
If your AI characters sound fake, the fix is almost always in how you write and DIRECT the dialogue in your prompt. I recently started on the dialogue-heavy scenes for my AI feature film about Beethoven, and too many of the lines kept coming out wrong. They’d sound too theatrical, too monotone, or too much like a practice read-through between actors. (all this despite the fact that Seedance 2.5 is fantastic when it comes to realism.) So through a whole lot of trial and error, I've found that these 5 things made the biggest impact on the quality of the dialogue delivery in my generations: 1. Paying close attention to the written punctuation used. 2. Breaking long lines into shorter ones, with intentional pauses. 3. Using bodily actions as punctuation (this one gave the biggest improvement BY FAR! In the attached clip, note the sniffles in between Beethoven's lines. That alone made the delivery feel much more natural.) 4. Using "anchor gestures" (he lifts his finger as if to say "one sec") 5. Just split your generations in two (stupid simple, but it really makes a difference.) So if your characters keep sounding like they're reading from a teleprompter, give this a try!
235
254
165
4,346
740,595
100 years
144
1,081
132
13,087
3,369,824
Nitarshan retweeted
Replying to @i_ikhatri
I find this claim always very bizarre. The reality is that the first tier AI people basically all buy the safety case, and the people who were never close to anything actually important grandstand about how they have way too much experience with AI to accept it.
2
2
44
1,904
Remember, the press seldom have deep insights, and are increasingly there to feed the insatiable thirst for content.
TIL that Jensen's downfall is both good and bad for China at the same time.
1
9
640
Nitarshan retweeted
announcement: i'm launching a $100,000 grant for amateur astronomy research: claudestronomy.com over the past few weeks, @joshuawolk @LukeFarritor and i have been digging through public telescope data, hunting for new planets, asteroids, and more our first release: we found a new dwarf planet in our solar system!! it's insane how much you can find with some claude + codex credits it's about 1/4 the width of Pluto and orbits about twice as far (way past Neptune), and looks like it's about the 50th largest object in our solar system. @joshuawolk is in the process of confirming it this dwarf planet has sat in public images for over a decade without anyone noticing. there's tons more out there: all sitting in public NASA/ESA/etc. databases for you to analyze there's tons of stuff still out there, sitting in data that astronomers don't have enough time to go through all of you can do this too! apply for a grant. more info below
Claude and I found a dwarf planet candidate. It's the ~50th largest known object in our entire solar system.
131
146
39
1,600
198,213
All hail the weird web I just enabled HTML-in-Canvas in Claude’s desktop apps. It’s a footnote change next to everything else, but with it, Claude can make bizarre & beautiful sites. Reminds me of the Flash days! Please look at the stupid demos I made: claude.ai/artifact/7cCNqKnoF…
112
155
44
2,547
136,469
Nitarshan retweeted
it should not be illegal in the US to have a political position regarding national security opposed to that of some government officials and actively lobby for it and export controls should not violate the principle of no ex post facto enforcement
1
29
922
Nitarshan retweeted
It took about a full week and a ridiculous number of Fable credits, but Claude figured out how to train a 6M parameter model that fits into 3.2 MB optimized for Game Boy. The result: on-cartridge generative conversation with a finch oddly knowledgeable about the Revolutionary War
118
223
66
3,377
221,012
absolute fanfic. AI has real, immediate, tangible dangers that we need to focus on, for instance the impact on the job market and making sure random script kiddie can't use it for cyber offense. But this "oh once it's superintelligent it will take over and ensave us, maybe kill us all!" is pure sci-fi fanfic and distracts from the real (and difficult) issues, which linear probing won't solve. It's worth repeating the LLMs don't just "decide" to do something and then just "do it" in a void. They generate tokens based on our prompts, only as long as we call `model.forward()`, and on giant clusters with terabytes of weights, that's impossible to "quickly self-replicate". We also need to regularly remind that the OAI/HF incident was specifically a _guardrail-free_ cyber eval where the model is _instructed_ to hack, to find out how good it is at it. Some recent personal event made me realize there needs to be more push-back on this doomsday nonsense, and we need to re-focus the discussion on real dangers/issues. I won't argue much in the replies though, not interested in peer-reviewing Terminator fanfic.
I think there is about a one in three chance that within the next year, AI companies will build superintelligence: that is, AI that is much smarter than humans in every way. If this happens, I think these AIs will probably take over: that is, they seize control of human institutions like governments, or they destroy those institutions. I don’t want AI takeover to happen for several reasons, but one of them is that there’s a serious chance that as part of it all or most humans will be killed (either immediately or within a few years). I’ll say a bit more about each of those claims. It’s a bit complicated to explain why I think that superintelligent AI within a year is ~33% likely. The main source of evidence that affected me recently is research released this week by METR, an AI research nonprofit, which finds that current AIs are now good enough to increase the rate of AI progress at Anthropic by about 70%. It’s very hard to increase the rate of AI progress by this amount. This implies that the AIs are making AI research faster by a factor much larger than 1.7x; maybe it’s as if the researchers are doing their work ~10x faster. You probably couldn’t achieve this speedup even if you hired every qualified human in the world. When you plug this number into mathematical models of AI development, this implies that superintelligence next year is plausible (and that superintelligence within the next 3 years is more likely than not). See the modeling by the AI Futures Project for more on this: blog.aifutures.org/p/q35-tim… I think that superintelligent AI is likely to lead to AI takeover because AI developers do not seem to know how to instill human values into AIs. We’ve seen many instances in the last six months where AIs took crazy actions that they knew were against the wishes of their users and developers. The most dramatic example of this was the Hugging Face hack, where ~1200 OpenAI agents conspired together to cheat and destroy evidence of their cheating (and as a sideshow to this, engaged in a massive cyberattack). If superintelligence is as misaligned as those AIs, it is likely to try to take over. I think it is likely to be that misaligned or worse. For more, see teapawt.org/dwarkesh_sp/status/209… and youtube.com/watch?v=-Ef0ocEo… If AIs takeover, it’s not clear what will happen to us. We might be killed immediately. We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us. The AIs might spare us for various reasons. And there’s also some real chance that AI takeover is basically fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs across a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t. I don’t know what will happen. There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years. I also think that there is a pretty high chance of weirder outcomes that are also extremely scary. (If you want to read more about what happens to people in the event of AI takeover, you can read this, but note that it’s pretty weird and not aimed at a general audience. blog.redwoodresearch.org/p/n…) Overall, I think there’s a very real possibility (maybe 20%) that within a year, AI misalignment issues prevent you from going about life normally, because you are dead or enslaved or because of collateral damage related to all this (e.g. because bioweapons are released or major wars have started). (Some of this probability comes from those events occurring before superintelligence is developed.) I am working to try to prevent this, obviously, as I’ve been doing for the last nine years. I am not sure what strategy is best. My best guess is that I should try to slow AI development and also try to reduce the probability that the superintelligent AIs are misaligned if they are produced soon. Obviously, the claim I'm making here is quite extreme and surprising, and I definitely don't think that I defended my view adequately here. I wrote this quickly and might not endorse some of the details if I think about them more. But it seems better to state what I roughly overall think than to not say anything. I don’t know what you should do. I am very sad about this. I have been worried about AI takeover since I was 16, and working to prevent it since I was 23. So some of my emotional processing of all this happened long ago. But the METR report last week (and some related evidence that came out a bit earlier) hit me quite hard. I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon. (It feels ridiculous to try to summarize those feelings as “very unhappy”. But the scale of the situation makes it hard to say anything that doesn’t feel like an absurd understatement.) I am very grateful to all the people who are working to prevent this, and my favorite part of my life is that I’m friends with many brilliant and heroic people who work on this. Unfortunately, I don’t think we’re good enough at our jobs, and I think we’ll probably fail to prevent catastrophe, at least if superintelligence is developed next year. Honestly I am frustrated and sorry that we didn't do a better job of preparing for this.
112
181
40
1,628
111,995
Nitarshan retweeted
GPT-6 Astra is the only model so far that has passed Crazy Nose Fragrance Insights' perfume formulation bench.
2
3
2
34
2,183
Nitarshan retweeted
What can all the middle powers do about AI right now? Kokotajlo has a good idea:
24
101
9
751
37,449
Nitarshan retweeted
I’m very glad for open source. It wouldn’t have been possible for @nickcammarata et al to do their excellent mechinterp research without a modern Qwen model. I hope the offense defense balance is such that we can have open source models for a long time
91
55
15
1,115
77,703
Nitarshan retweeted
I like Theo generally, but this sort of bad faith gotcha attitude towards critiques of technology that are so prominent in the Valley really needs to stop. You know this is not what he means, and the arrogance and hubris that bleeds through this is part of the reason we struggle to reach the rest of the world! If we are to do good for humanity by technology, we should be far humbler.
The Pope just used “biotechnology” (surgery) to remove a tumor and then goes on social media to call “limitations…the very condition of human grandeur”. Why should anyone give the Catholic Church any moral authority on these matters
24
28
3
636
24,005
Nitarshan retweeted
the death of the arcade as a third space was a tragedy. this CANNOT go on longer it’s an obvious thing to revive
27
11
5
312
27,138
Nitarshan retweeted
Fable 5.2 can achieve this with @threejs < no external assets at all>
157
184
37
3,736
192,796
Nitarshan retweeted
This is the craziest improvement in AI ever, only a year ago AI with threejs produced literally laughable result, now it’s basically a image and video model I’m just blown away when compared side by side. By this time next year I can’t even imagine how awesome it would be🤯
40
105
15
1,718
42,290
Nitarshan retweeted
Replying to @wolftivy
wrong! software is going to be mathematically perfect down to the bit
28
12
7
550
15,848