President, @LawReformInst; previously executive director @uniformlaws, attorney @StateDept
McLean, Virginia
Joined January 2016
- Tweets830
- Following717
- Followers566
- Likes8.5K
Pinned Tweet
New from Dan Crane and me in @just_security: an explanation of why the bipartisan Collaboration on Adversarial Threats and Security Risks Act is necessary.
Looks like the rumors were correct, re: Anthropic's founders retaining majority voting control.
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
Anthropic also made disclosures about the antitrust risks it faces based on efforts to address "the responsible pace of AI development."
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
Very interesting that Anthropic highlights that its "future models ... may be able to produce export-controlled software," but they say nothing in that respect about current models, let alone export controlled "technology" or "technical data." How is it not a relevant risk that, at any time, BIS and DDTC could decide to enforce EAR and ITAR restrictions based on models being able to output information necessary for producing/using export-controlled items?
Also interesting that they flagged that "[i]n certain contexts, the provision of our models, platform, and applications may constitute export of defense articles or defense services subject to the ITAR, which can be exported only if we obtain required export licenses or other government authorizations." Glad to see they're not focused solely on the EAR... but still weird that they drew the line in this manner?
Anthropic has filed an S-1 registration statement with the SEC related to its proposed IPO.
Read more: anthropic.com/news/s1-sec
Legal disclaimer: A registration statement relating to these securities has been filed with the SEC but has not yet become effective. These securities may not be sold, nor may offers to buy be accepted, prior to the time the registration statement becomes effective. This post does not constitute an offer to sell or the solicitation of an offer to buy these securities, nor shall there be any sale of these securities in any state or jurisdiction in which such offer, solicitation or sale would be unlawful prior to registration or qualification under the securities laws of any such state or jurisdiction.
I'm bullish on courts using AI, but we're also going to see problematic instances. In some ways, judges can trust the best models (at high reasoning levels) more than (good) clerks, and in some ways still far less. But oversight is needed for both.
[Eugene Volokh] "The District Court—or Perhaps, Being Charitable, Its AI Agent—Ignored and Manipulated Governing Law, Apparently to Satisfy a Policy Agenda" ift.tt/WYdPUtr
Tim Schnabel retweeted
If we really want a surge of investment in cybersecurity ASAP, lame duck should pass a major cyber tax credits & deductions bill
Tim Schnabel retweeted
In other safety-critical industries like nuclear and finance, employees have defined channels for raising safety concerns confidentially with auditors and regulators. AI needs similarly clear rules governing relationships between labs, outside evaluators, and employees.
Calling an evaluator an independent auditor means little unless its access, and employees’ ability to speak candidly with it, are protected. OpenAI says they support embedded auditing but then fires employees for talking to those same auditors.
This case illustrates the weakness of leaving those protections to voluntary arrangements controlled by the labs themselves. Employees who are never clearly told which channels are permitted, and how they can be used, can't comply with the rule, and everyone watching learns that talking to evaluators is risky.
Even if OpenAI does give competent auditors the access they need, I am worried employees will be too afraid to tell the auditors what's really going on.
New: The three researchers OpenAI fired last month spent the summer fighting to keep AI's chain-of-thought process visible.
Then, OpenAI asked to see their Signal chats. A week later, they were let go.
More below from @keachhagey, @berber_jin1, and me.
wsj.com/tech/ai/the-signal-c…
Tim Schnabel retweeted
Best explanation I have for our firing: talking to auditors was too much safety, disappearing messages were too much security, and we should have paced ourselves.
The firings of three OpenAI safety researchers came in the throes of a battle over how to keep AI systems under human control—and who gets to see the secret designs that could determine the winners of the biggest commercial race in history. on.wsj.com/48bYPG8
RIP John Anderson; one of the most recognizable voices in country music. "I was the proudest papa in the USA-- makin' a living on the road somewhere a thousand miles away." youtube.com/watch?v=JgPMRInv…
Tim Schnabel retweeted
Congress has explicitly limited the Exec's export authority around information and software before, it can and should again. The only allowed model-blocking should be narrowly defined by Congress, and with strong procedural protections considering the prior restraint issue.
One thing I find surprising about Anthropic's draft is that it doesn't move the model-blocking authority from ECRA into this new framework. It creates a parallel, difficult-to-use mechanism that the executive branch could ignore in favor of more is-informed letters from BIS.
Anthropic quietly pitches its own AI safety bill to lawmakers politi.co/4zxDFhl
Interesting proposal from @A1Policy, which has been putting out some nuanced pieces on AI policy.
New @A1Policy paper 🚨:
The Hugging Face incident and other recent incidents have led many in the Super Intelligence (SI) industry to wonder whether we should “pace the frontier” of development.
We propose a better solution: “Reorient the Frontier.” Under this plan, SI companies would refrain from playing God by pursuing recursive self-improvement—SI systems running without human input or oversight—and instead reorient their incredibly capable models and their computing power to serving useful products to consumers and creating new science that benefits humanity.
The core of this strategy is a commitment from SI companies to devote the majority of their computing power toward serving real customers.
To usher in the “Reorient the Frontier” approach, the White House should organize a new initiative, leveraging large new compute commitments from SI companies, to cure cancer before the end of this administration.
Glad that Josh is banging this drum-- we're collectively not paying nearly enough attention to this issue. If BIS and DDTC started assessing frontier models' ability to assist in developing export-controlled items, it might force the USG to notice where things are headed.
"increasing centrality of privately controlled dual-use tech" indeed
Tim Schnabel retweeted
Good.
The Government Accountability Office is making the oversight of artificial intelligence and emerging technologies over the next five years a priority, according to its strategic plan released Friday.
As Congress’ watchdog over the federal government, the GAO said in its 2027-2032 plan that it seeks to help the House and Senate “anticipate emerging science and technology trends, opportunities, and risks, including artificial intelligence,” and “monitor federal efforts to ensure the nation’s security from cyberattacks and other threats.” scoopmedia.co/4sjp5rE
Tim Schnabel retweeted
AI policy has almost exclusively focused on setting guardrails on AI companies and their models. This is reasonable, as we have seen some of AI's biggest risks materialize this year.
It's time we start talking about guardrails for government as well.
More in my latest:
If someone thinks there aren't enough viable proposals for something that they want to do, then maybe they just need to think about what they'd do if they had more agency. theinformation.com/articles/…
The idea that OpenAI would have to choose one of these three (flawed) options within the next year, rather than finding a better approach, would suggest a significant lack of imagination/agency, atypical for Sam. Perhaps the actual phrasing was different?xitter.cf/sama/status/1885218481…
@amir and I take you inside last week’s all hands at OpenAI, where CEO Sam Altman outlines 3 industry proposals for pacing the rate of AI development:
- IAEA for AI
- The Demis plan (CERN for AI)
- The Eliezer plan (“melt everyone else’s GPUs”)
theinformation.com/articles/…
This IFP reports estimates that implementing a narrow antitrust safe harbor to let US companies coordinate against adversarial distillation could, on its own, increase the US lead over China by 57%.
Big new IFP report from @GeorgiaCAdamson, @KonstantinPilz, and me on the US-China AI competition.
We spent the last 6 months digging into how Chinese AI companies’ access to compute and distillation of American models translate into the current US-China capabilities gap.
The result is a 23k-word research report, including an interactive analytical model that visualizes changes in the US AI lead under different policy interventions.
Our findings:
1. The US AI lead over China is currently only around 6 months.
2. The US could roughly double this lead, to over a year, using authorities already available to the federal government.
3. On chips: China’s access to AI chips will roughly double over the next year, with chips housed in data centers overseas accounting for roughly half of China’s 2027 compute.
4. On distillation: the current ~16x US training compute advantage would normally yield a 13-month lead. But we estimate adversarial distillation halves this to around 6.5 months.
5. Increasing the lead would have some obvious benefits:
- Give the US broadly a greater cyberdefensive advantage against Chinese threat actors.
- Give US AI companies breathing room to spend more R&D resources on making their systems safer and more secure, without fear of being overtaken by less safety-conscious Chinese competitors.
6. Provide the US government more leverage in negotiations with China about the future of AI.
7. The two most important interventions are:
- For BIS: Create a new end-use control to restrict China’s remote access to training clusters abroad — China’s largest and fastest-growing source of frontier-scale compute.
- For DOJ/FTC: Create a narrow safe harbor to allow US companies to coordinate on countering Chinese adversarial distillation.
8. Both interventions (and all the others we discuss in the report) are things the federal government could do today.
9. To estimate the effects of each policy intervention, we model the US lead as a function of its compute advantage over China and China’s ability to adversarially distill the capabilities of the US frontier.
10. The model makes many simplifying assumptions, but provides a useful way to assess and rank the impact of different policy interventions at a high level.
Read the full report here: ifp.org/how-to-increase-amer…
We urged DOJ and the FTC to provide such guidance earlier this year. xitter.cf/LawReformInst/status/2…
But hopefully this bipartisan legislation will provide a more complete fix than agency guidance could: justsecurity.org/150875/anti…
