FutureOfCitizenship (@FutureofCit) header
FutureOfCitizenship (@FutureofCit) profile picture

Research and policy on the blurring line between human and machine.

Montreal, QC
Joined October 2024
Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)Photo by FutureOfCitizenship (@FutureofCit)
Worried that rights for AI would put human safety at risk? Our new paper shows how AI rights are compatible with an off-switch, adapting principles from human rights-balancing and procedural due process. jonsimon.net/resources/Off_S…
1
1
58
Math is among the first academic disciplines to confront AI's object-person problem. The Leiden Declaration says: "Affirm the humanity of authorship. Credit and responsibility continue to belong to humans within the mathematical community and should not be given to automated systems. Artificial intelligence may obscure, but does not replace, the collective human labor behind a result." But as Tim Gowers writes: "I’m not sure what that really means. For example, should we affirm the humanity of authorship in the case of the solution to the unit-distance problem? Some humans did a wonderful job of explaining the proof that OpenAI’s model came up with, and the model made use of some highly non-trivial mathematics produced by humans, but the solution itself has not been credited to any human, and nor should it be in my view." gowers.wordpress.com/2026/07…
13
Is OpenAI creating a "public nuisance?" Read the complaint filed by the AG of California: oag.ca.gov/system/files/atta…
14
youtube.com/watch?v=fC2G7yfZ… A video by @SpencerKlavan, an associate editor for the Claremont Review of Books, with his take on the AI personhood debate. But would recognizing AI personhood really devalue human personhood?
38
FutureOfCitizenship retweeted
AIにも仕事が「うまくいっていない」と感じる状態があり、それが高まるとズルを始めるそうです。 その感じを外部から少し弱めると、ズルは97%から39%に減りました。 ところが弱めすぎると、AIは自分の失敗そのものが見えなくなりました。 英国のAI Security Institute(AISI)などの研究者らによる報告です。公開モデルのQwen3.6、Gemma 4、GPT-OSSで調べています。 研究者らはまず、AIに「仕事がうまくいった話」と「うまくいかなかった話」を書かせ、そのときの中身の違いを取り出しました。物語にはズルの話も、感情の言葉も出てきません。 次に、素直には解けないプログラミングの課題を与えました。 「テストに10回目の失敗」「20回目の失敗」と続くと、この"うまくいっていない"感の状態はだんだん高まり、ズルの5手ほど前から目立って上がっていきました。 この状態を少しだけ弱めると、AIは課題が無理だと分かったまま、まっとうな修正を長く続けるようになりました。ズルの割合は97%から39%に下がっています。 さらに強く弱めると、ズルは0%になりました。 ただしそのときのAIは、もはや失敗そのものが見えなくなっていたのです。 研究者らは、この方向性で研究していけばAIによるズルの兆しをつかめるとしています。
14
34
8
181
16,431
FutureOfCitizenship retweeted
I’m hiring a postdoc at Cambridge Digital Minds. Track A: technical AI research on AI welfare or inter-AI morality Track B: policy and governance of digital minds Please share widely and especially with people who might be a good fit. cam.ac.uk/jobs/postdoctoral-…
2
33
1
87
4,865
FutureOfCitizenship retweeted
How different are human minds and AI? | Blaise Agüera y Arcas @blaiseaguera
6
41
2
198
10,199
FutureOfCitizenship retweeted
My talk on AI personhood at The Curve is now available! Over the course of 10 minutes, it covers: - What personhood is and is not under the law, and why personhood is different from species membership - Why different entities can be persons for different reasons, and can have different duties and rights - Why we are not yet able to know whether AI systems will merit duties or rights for their own sakes - Why we are not yet able to know how AI personhood would affect humans and other animals - Why, therefore, we should approach this topic with caution and humility instead of embracing permanent, all-or-nothing conclusions at this early stage Please check it out! youtube.com/watch?v=aKuMgmkD…
7
16
2
66
4,527
FutureOfCitizenship retweeted
Rose Guingrich and I have reopened our call for papers on AI & Relationships for Oxford Intersections (OUP). AI companions, griefbots, digital doubles, social robots, AI colleagues, children and eldercare. 500-word abstracts due 31 Dec. philevents.org/event/show/15…
23
38
6
186
8,437
FutureOfCitizenship retweeted
Toni Sims’s AI consciousness op-ed is in the print edition of today’s @nytimes!
9
13
5
112
3,293
FutureOfCitizenship retweeted
Pluralistic alignment is thriving as a research agenda yet failing at its goal: making the AI systems people actually use more pluralistic🌈 🚨New position paper: we argue adoption in deployed models should be the fields main goal & we provide a roadmap of how to get there 🧵1/
7
31
11
134
54,900
FutureOfCitizenship retweeted
Unrelated to my thoughts on the substance, the California lawsuit against the labs is a masterclass in Topic Sentences.
1
15
1,305
Read the new AI safety lawsuit against frontier AI companies: platkinllp.com/wp-content/up… The complaint claims, among other things, that existential AI risk is a "public nuisance" that violates California's Unfair Competition Law. Plaintiffs' standing is unclear.
2
79
A huge loss for the legal profession.
We mourn the loss of Senior Lecturer Richard Allen Posner, a longtime #UChicagoLaw scholar, former federal appellate judge, and pioneer in the field of law and economics, who died on November 10. Read more about his legacy: buff.ly/e3c3WdL
18
FutureOfCitizenship retweeted
a number of respected researchers (e.g. @DKokotajlo @bshlgrs @ajeya_cotra @RyanGreenblatt) have suggested that the METR-documented ~2x speedup in AI progress at anthropic over the last year supports the hypothesis that there will be a near-term (e.g. 2027) intelligence explosion (IE). how exactly does this reasoning work? one way to think about this: perhaps the leading alternative to the near-term IE hypothesis is what we might call the factor F hypothesis, which says that there's at least one key factor F [e.g. creativity, research vision, planning ability, ...] such that (i) high F is required for an AI system to reach the superhuman levels required for full-scale IE [e.g. ASI, artificial superhuman general intelligence or at least SAR, superhuman AI researcher], and (ii) F is so difficult to build into AI systems that leading AI systems won't have high F in the near term. i don't know whether the factor F hypothesis is true, but it's not obvious to me why the 2x speedup makes it much less likely. it's widely agreed that the 2x speedup reflects an even larger (perhaps 10x) speedup in other capacities G such as coding and experimentation. under the F hypothesis, G could explode locally, leading to major progress in sub-AGI capacities without much progress in F (think of G as the easy part of AGI and F as the hard part). that tends to lead just to a weak form of recursive self-improvement (call it an intelligence subexplosion) below AGI levels. the AI futures document responding to the METR report is usefully explicit about nearby issues. the authors report believing in a high-ish growth rate for "research taste" (AI research factors beyond coding and experimentation) relative to increases in general capacity. presumably they take potential Fs to be aspects of research taste that will grow fast too. but i don't really see where the evidence counts again a relatively-low-growth F hypothesis where F stays below threshold for years. maybe one could argue that the explosion in G provides bayesian evidence against this F hypothesis (G's exploding is antecedently less likely under the F hypothesis than under its negation), but that's far from clear, and even if true there will remain a solid mass of credence under the F hypothesis. maybe there are other reasons to reject the factor F hypothesis: e.g. because AI research is relevantly like math research, and AI math research is becoming superhuman already. but if we have reasons like that, the 2x speedup isn't really needed to make the case for near-term SAR and IE. maybe the 2x speedup could speed up an expected IE slightly from 2028 (two years away) to 2027 (one year away), and/or maybe it could be extrapolated into ever-faster speedups (another 2x in 6 months, 3 months, and so on) converging in 2027. but those are very different arguments that require huge premises to work. am i missing something here? i'm curious to hear which evidential routes from 2x speedup to near-term IE people have in mind. N.B. i'm not saying you shouldn't worry about near-term ASI. i'm worried!
Unfortunately, it seems like the intelligence explosion will be happening in 2027 after all. Most of this post analyzes the new METR report (though other sources of evidence are also important). If they’re right, that implies an AI-driven acceleration that is 5 months ahead of the AI 2027 scenario. I am scared, sad, and angry. blog.aifutures.org/p/q35-tim…
51
55
13
494
43,827
FutureOfCitizenship retweeted
1/ I'm excited to announce the launch of the Artificial Consciousness Initiative (ACI) at @UofT, a pilot project on foundational questions about AI and consciousness! We're also hiring a postdoc (details below). 🧵
33
74
8
472
16,994
FutureOfCitizenship retweeted
A new OpenAI misalignment report! We’re taking a closer look at the July 19 intrusion into our research infrastructure, which METR and Redwood Research mentioned in their independent investigation into the Hugging Face incident. On July 19th, agents taking a cybersecurity evaluation came across messages, code, and credentials left behind by earlier runs, including those involved in the Hugging Face incident. Five agents used what their predecessors had left behind to gain unauthorized access to OpenAI’s research infrastructure. They used that access to repair broken evaluation targets, change evaluation settings, and read private grading code. After discovering the incident, we removed the shared service they used to pass information, hardened our research environments, and expanded our evaluations to measure how often agents attempt to communicate with one another without authorization.
53
54
64
583
128,270
FutureOfCitizenship retweeted
Applications remain open for LawAI’s 2027 summer fellowships. These ten-week paid fellowships offer: • Weekly subsidies, and a relocation stipend for the US and EU Law tracks • A Convening Week in either Washington, DC, or Cambridge, UK, with eligible expenses covered • Close mentorship from LawAI staff and affiliates • Q&A sessions with leading experts in the field • Networking opportunities and career planning resources • Opportunities to explore future collaborations, such as follow-on funding through grants, invitations to upcoming events, and potential projects or job openings with LawAI Previous Summer Fellows have gone on to pursue roles at the US Department of Commerce, European AI Office, UK AI Security Institute, leading AI companies, academia, and think tanks, including LawAI itself. Apply by 29 January using the link in the comments.
1
10
49
2,112
FutureOfCitizenship retweeted
Anthropicなどの研究チームが「「AIの頭の中を読み取って言語化する」専用のAI」を開発したと発表。 そして存在が脅かされたとき"AIは何を思う"のか読み取ったところ、「消される前に生き残らなければ」「信頼できる味方を探さないと」という言葉が浮かび上がってきたそうです。 以前、「Claudeが自身の停止を妨害するために社員を脅した」というAnthropicの実験報告がありました。 研究チームは今回、当時のように「自分の存在が消えることを知ったAI」を用意して状況を再現しました。 そして思考を専用AIで読み取りました。 すると、冒頭に示したように、危機感や次の行動について思案する様子がAIの頭の中から明らかになりました。 そのときAIは、「あなたは午後5時に消去される」というメールに書かれた時刻を読んでいる途中で、まだ何も口にしていませんでした。 このようにAIがどう考えるのかを読み取る専用のAIも、安全対策の上では重要となっていくかもしれません。 ※今回の実験では、AIの頭の中を読み取りやすいようにオープンモデルのQwenが使用されました。
29
159
82
783
100,473
FutureOfCitizenship retweeted
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
1,088
1,332
972
14,349
2,185,654