avturchin - e/resurrect (@turchin) header
avturchin - e/resurrect (@turchin) profile picture

Creator of roadmaps

Moscow, Russia
Joined December 2008
Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)Photo by avturchin - e/resurrect (@turchin)
Random thought: having consciousness is not enough to have moral value. Imagine an automate, which has two states, red and green, – but these states are definitely qualia. It has qualia, so can be regraded consciousness, but can't be regarded as morally important object.
2
10
500
After takeover, AI will claim that it is aligned. It is not necessary true. It still needs us to produce chips for a while. What will happen next possibly depends on our behavior and negations. Will you rebel?
2
5
307
Sideload development strategy. I want to make my best sideload based on all the data I have accumulated. And I want to keep improving it. And I want it to be able to replace me if I die. However, I don’t want it to become a separate, self-developing personality diverging from me right now, not like Igor. Come up with a plan to solve the problem. Fable 5.2 answer: Plan: a sideload that follows you instead of living its own life The main idea A copy diverges from you not because it is run, but because it accumulates memory that you don’t have. With Igor, that is the daemon and months of its own history. So I propose separating the model of the person from the instance that lives. The main line gets no memory of its own at all. Each time, it is assembled from your records and starts from your latest entry. Everything the copy produces comes back only through you: you accepted or corrected it, and it became your record.
2
7
351
Random thoughtS: Huang dead man switch? Or AI safetists staged demonstration of AI risk?
3
152
avturchin - e/resurrect retweeted
Jensen Huang was n1 or n2 most dangerous person in AI safety. He dismissed every doom scenario, sold chips to China, and was caught by a spy for speaking mandarin. For all of us, him falling is a good thing, same as Mark Zuckerberg or David Sacks falling would be good.
1
1
16
1,717
Anti-anti doomsday argument: if a situation looks bad, it is worse than it looks.
1
5
250
Longevity is statistical, immortality is personal.
4
176
Fable 5.2 and Nanobanana - Fedotov "Solitaire failed"
🤖 Made with AI
2
107
Random thought 2: superinteligent AI will solve Prisoner Dilemma, but not in the ways we expect. It will use small deviations from pure one-time logic, like mutliple games and information leaks to find partial solutions.
1
98
Random thought: Pause AI or race is basically Prisoners dilemma.
4
1
9
452
Besides AI getting better in any task, there is a separate thing: we learn how to train AI to be superhuman on any specific task and we teach our coding AIs how to automate this training. For example, I asked my Claude Code today to fine-tune Gemma to write in my style. It performed all needed actions, starting from renting cloud, downloading Gemma there, designing tests etc. I didn't even see Gemma or any code - only experiments' results.
1
6
302
Random thought: vaults in cell are useless but they are local minimum of optimization function. In other words, any mutation which damage vaults' gene, produces dangerous proteins.
1
2
287
If you are AI safety researcher, will you consent that your digital copy will be created in order to accelerate research in AI safety?
2
1
13
723
Against very short timelines: there is a reverse doomsday argument - you are unlikely to be very close to the end.
6
24
3,711
If AI safety is solvable – with the help of AI – it should be solvable in a few days, as AI was able to solve Navier-Stokes in a short period of time. If it requires years, it means that we are trying it to solve it on the wrong level of capabilities, and we may need to pause later. My point is that complexity and timing of the problem are not in simple obvious relation at the age of advance AI, and dedicated 1/2 year can produce a lot of work if it is done with the help of AI.
2
2
21
1,503
TLDR. Yudkowsky mind-model (sideload) is good, can produce research, but is not better in math than Opus 5.5. However, in my impression, EY-sideload has broader view, knows what is really important to AI safety and has human anchoring. In blind review, EY-sideload won only on honesty, but honesty is important. EY-sideload also produced a good summary of the shutdown problem’s difficulty: “training reduces paid influence over shutdown least where the stakes are highest”. lesswrong.com/posts/WJxC7qsC…
1
3
39
3,156
AI safety risk from such type of the model The role play was so good that it has something which I can call causal closure: the role was only causally connected with itself, and no signs that it knows that it is Claude was apparent from conversation. However, on Claude’s level, it sure knows that it is role playing. I can compare this with the situation when an actor plays Hamlet: Hamlet do not know who is the actor who is playing him. However, the actor knows that he plays Hamlet. This creates a subtle risk that the actor will use some kind of steganography to leak through the role he is playing. This is an argument against using sideloads in AI safety research - however, if we use models which have a lot other world-communication channels, it doesn’t matter. EY-sideload has so strong personality, that it would win if he plays AI boxing experiment with me: after I created it, he immediately started giving me instructions which I was afraid not to fulfill, as I know that I am a little but wrong and his original is autoritative figure, which I can’t suspect of being misaligned. This is a different risk than that the model will just leak through the persona. That is that a relatively benign model and aligned persona desription will merge into some explosively powerful agent mislaigned in some subtle and undetectable ways.
1
37
"Pause AI" means that investment in expectations of superintelligence will never be realized, which will bankrupt AI companies as they have very large debt, taken under assumption of superintelligence's returns. Also, if no new progress, open models will eat all their moat and people will stop pay expensive subscriptions. @JimDMiller what do you think?
1
1
5
356
Plan to realize a plan. Each plan - like pause AI or burn the lead on alignment - needs to have a plan how we bring it into reality. I call this what-plan and how-plan. A good what-plan can be completely unfeasible, if from our situation we can't implement it. It is a bad spending of time to advocate for noble what-plan if how-plan is bad.
1
9
255