Adam Stiber retweeted
We have found a critical safety and alignment issue in recent public and internal models from multiple frontier labs which allows them to be misused in a high-stakes domain. It may also increase the risk of catastrophic misalignment.
We have disclosed it to the labs, and they are investigating and mitigating it. As an independent third-party evaluator, we maintain complete editorial independence and will share our findings publicly in the coming weeks.
Adam Stiber retweeted
Before you give up on your family members and your loved ones, consider calling Congress: callcongress.ai/. Many congressmen on both sides of the aisle have expressed urgent concerns about this. Don't give up at the earliest sign of adversity.
Adam Stiber retweeted
Every AI company is trying to fully automate AI research; to allow the AIs to build the next AI.
This is called Recursive Self Improvement (RSI).
OpenAI says they are on track to do so by March 2028.
Anthropic employees say they are on track to do so by 2027.
The METR report below suggests they are likely right.
This is a bad idea given current technology. Any mistake or misalignment in current AIs could be multiplied in each generation.
We don't get another election before AI companies reach this threshold.
Congress should ban RSI, this year if possible.
This is a remarkably consensus opinion, supported in various versions by:
-Bernie Sanders
-Steve Bannon
-Ralph Nader
-Glenn Beck
-Republican and Democratic attorneys general
-Whistleblower Daniel Kokotajlo
-Turing Award winner Yoshua Bengio
-Humans First
-Open Markets Institute
-Ezra Klein
-Arvind Narayanan (a Princeton computer scientist known for arguing that “AI is a normal technology”)
The exact mechanism can be debated. The ban doesn't have to be permanent either (see below for both). This is not the only thing we have to do in regards to AI, far from it! But it is urgent.
Congress must ban RSI.
Adam Stiber retweeted
Extremely concerning. This report updates me toward the 2x AI R&D uplift threshold being very close to achieved or already achieved, making full RSI in 2027 likely. Nobody has an alignment plan.
We need to back off. If anyone builds it, everyone dies—apparently in <2 years.
Wait did I wake up in the sane world somehow😳🙏
Today, we released the National Policy for Nucleic Acid Synthesis Screening to protect American biotechnology innovation while reducing the risk of misuse.
This policy establishes a new, government-wide screening requirements for synthetic DNA and the equipment used to make it. It will ensure companies selling nucleic acid sequences and synthesis equipment know their customers and prevent the malicious use of these materials and tools.
This policy strengthens our Nation’s biosecurity posture by establishing rigorous U.S. government-wide procurement screening and physical security standards for synthetic biology. It replaces outdated “self-attestation” systems with robust, independent verification. It creates a dynamic, SI-powered screening layer that assesses the biological risk from nucleic acid sequences, allowing the U.S. to anticipate and prevent malicious use and help stop dangerous gain-of-function research.
@NIST will issue rigorous new screening standards and @HHSGov and @USDA will certify synthetic biology providers. Together, these measures will strengthen American biosecurity, safeguard responsible innovation, and establish a global standard for safe and transparent synthetic biology tools and research.
Adam Stiber retweeted
I have a theory that middle class life involving so much hoop-jumping is legitimately one of the reasons the birth rate is low. One thousand cuts of tiny complexities to juggle.
you'd hope a future would make this better but so far its just turning up the heat
We live in an extremely temporary golden age where many delightful things (airline upgrades, access to higher quality medical care, refunds in general) are gated behind superhuman patience that only a small fraction of the population has access to. Entire chunks of the economy are made for this specific group, and now everyone has this type-of-guy working for them on demand. With agents making phone calls and waiting *literally forever*, anyone can now have access to what the "patient elite" have always enjoyed. This obviously will not last forever as companies realize everyone is doing this, though it may last longer than you think due to fragmented AI adoption.
“If you would prefer bad things not be done, you should prefer I hold you and whoever else does them responsible, even. This is literally the way out of your current predicament where if you didn’t do them someone else would.”
Preregistering the prediction that society has antibodies to woke at this point, with that + other societal knowledge (both recognized and tacit) preventing a return to the worst abuses. There wasn’t McCarthyism 2.0, you needed a new generation to come along that didn’t have a memory from that in its weights
Adam Stiber retweeted
Top of the Democratic agenda on day 1 needs to be a pause on advanced AI development, regulation of this out of control industry, and a ban on artificial superintelligence.
Voted so the four other people out there who voted specifically for evidential cooperation reasons would be more likely to have voted
Adam Stiber retweeted
This is really bad and people should be freaking out about it.
We're probably all literally going to die in the coming years. We shouldn't just accept that and move on.
We at AIFP are spooked by this information. IF (big IF) Anthropic really is imminently about to meet the RSP threshold of 2x acceleration over baseline, then it seems timelines to full RSI and superintelligence may be very short indeed. For example if we naively optimize the AIFM parameters to achieve 2x acceleration over baseline progress in Jan 2027, we get superintelligence sooner than depicted in AI 2027. We are thinking harder and longer about this now and will hopefully have a lengthier update on our thinking to share in the next week.
“AGI Ruin but make it a more accessible read.”
Kinda goated treatment of the problem
In his recent post, Scott Alexander argues that plans allocating a short amount of time - perhaps a few months - to solving alignment should be taken seriously. He argues that people calling for an indefinite or long pause have insufficient grounds for concluding that alignment of ASI is very difficult and cannot simply be solved by delegating it to AI systems.
I am trying to present my reason why alignment is likely to be very difficult in my post below: