AGI Uncontainability Field Building

Raising $10,000active4 people

Encourage new research and increase distribution of existing research on the impossibility of alignment or control for sufficiently general and powerful AI.

Project Details

Phase 1: Online seminars
Partner with a well respected AI safety platform with a large audience to invite top uncontainability researchers to run live public talks and Q/A. The series of virtual working sessions will convene technical participants from the first Limits to Control workshop [1] and additional researchers in AI safety and control theory [2]. The seminars produce a shared technical understanding among participants before the in-person workshop. They will attract researchers in control engineering and control theory, computability theory, computational complexity, formal methods, neural-network verification, measure theory, information theory, mathematical logic, dynamical/complex systems, decision theory, learning theory, evolutionary biology and ecology, distributed systems, multi-agent systems, AI systems security, and governance/law who could be motivated to work on proofs of AGI uncontainability.

Phase 2: Second Limits to Control Workshop
An in-person, proofs-focused workshop with whiteboard sessions working through formal arguments. Participants engage with proof structure, submit their own formal contributions, and work toward joint assessment of which results hold, which need further work, and what the aggregate formal case implies. Venue could be at the University of Louisville again or perhaps in Cambridge, UK.

[1] 2025 Limits to Control Workshop https://limitstocontrol.org/

[2] Uncontainability researchers:

A. Computability / undecidability

Manuel Alfonseca, Manuel Cebrian, Iyad Rahwan, Antonio Fernández Anta, Lorenzo Coviello, Andrés Abeliuk - Superintelligence Cannot Be Contained: Lessons from Computability Theory (JAIR 2021). Strict containment is undecidable.
Ayushi Agarwal - On the Formal Limits of Alignment Verification. No verifier can be simultaneously sound, general, and tractable.
S. Haider - The Impossibility of AI Containment.
Moritz Englert, Sebastian Siebert & Martin Ziegler - Logical Limitations to Machine Ethics, with Consequences to Lethal Autonomous Weapons.

E. Value-learning & specification impossibility

Rina Panigrahy and Vatsal Sharan - Limitations on Safe, Trusted, Artificial General Intelligence. A safe-and-trusted system provably cannot be an AGI.

D. Evolutionary / substrate

Forrest Landry and Remmelt Ellen - The Formal Impossibility of AGI Safety: The Substrate Needs Convergence Proof.
Richard Everheart - On the Boundaries of Formal Intelligence.

E. Accountability and rule following

Haileleol Tibebu - The Accountability Horizon: An Impossibility Theorem for Governing Human-Agent Collectives.
Jan van Leeuwen and Jǐrí Wiedermann - Impossibility Results for the Online Verification of Ethical and Legal Behaviour of Robots
F. Measuring AGI
Greg Kamradt - President of Arc AGI; why current benchmarks saturate, why ARC-AGI is different, what a serious eval looks like.

F. Caveats and Rebuttals

Gabriel A. de Melo, Marcos R.O.A. Máximo, Nei Y. Soma, Paulo A.L. Castro - On the Undecidability of AI Alignment: Machines that Halt. Checking whether an arbitrary model is aligned is undecidable (Rice's theorem); a constructible aligned set still exists.
Matthew Fox - On Formally Undecidable Traits of Intelligent Machines. Argues against the Rice's-theorem impossibility line in Alfonseca et al.
Matt Luckcuck, Marie Farrell, Louise A. Deinns, Clare Dixon, and Michael Fisher - Formal Specification and Verification of Autonomous Robotic Systems: A Survey
Louise Dennis, Michael Fisher, Marija Slavkovik, Matt Webster - Formal verification of ethical choices in autonomous systems
Samuel Teuber - Provably Safe Neural Network Controllers via Differential Dynamic Logic.

Theory of Impact

AGI may not just be likely or very likely to be harmful, but guaranteed to be harmful. If the formal case for uncontrollability holds, then decision-makers should substantially redirect how safety resources are allocated, what governance frameworks are pursued, and what the public is told about the trajectory of AGI development. We need rigorous, collaborative, and exploratory spaces to scrutinize proofs, understand where they fail, and close gaps in explanation and action where they do not, improving how the field reasons about controllability and how that is communicated to the right actors.

People

Corey Cleland

Remmelt Ellen

Zachary Schlosser

Team Member

Funding Details

Start Date: -
End Date: -
Expected Duration: 6 months
Funding Raised to Date: -
Annual Budget: -
Monthly Burn Rate: -
Current Runway: -
Funding Goal: -
Funding Stage: -
Fiscal Sponsor: Better Giving

Track Record

The team deployed a 2024 SFF grant for AGI Inherent Non- Safety. Forrest Landry drafted proofs of why any autonomous AGI could not be controlled sufficiently to stay safe.

We produced:

Four uncontainability proof drafts being reviewed by math postdocs:
The 1st one establishes the non-amplification theorem [NAT] and defines hashiness.
The 2nd defines abstraction and uses NAT to prove limits of control.
The 3rd shows that these control limits apply to learning systems (especially AGI) to show that AGI, (and the world, through that AGI), is uncontrollable.
The 4th with Richard Everheart shows that, within the cycle of an iterated open-ended system (ie; basically every process, in ecology, economics, etc), there is no process of control that will produce a definite positive outcome.

Grants Received

Add grant

2026 SFF Speculation Grant from Roman Yampolskiy

from Survival and Flourishing Fund

$10,000

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$10,000

Ideal

$50,000

What this ask is

[1] 2025 Limits to Control Workshop https://limitstocontrol.org/

[2] Uncontainability researchers:

A. Computability / undecidability

Manuel Alfonseca, Manuel Cebrian, Iyad Rahwan, Antonio Fernández Anta, Lorenzo Coviello, Andrés Abeliuk - Superintelligence Cannot Be Contained: Lessons from Computability Theory (JAIR 2021). Strict containment is undecidable.
Ayushi Agarwal - On the Formal Limits of Alignment Verification. No verifier can be simultaneously sound, general, and tractable.
S. Haider - The Impossibility of AI Containment.
Moritz Englert, Sebastian Siebert & Martin Ziegler - Logical Limitations to Machine Ethics, with Consequences to Lethal Autonomous Weapons.

E. Value-learning & specification impossibility

Rina Panigrahy and Vatsal Sharan - Limitations on Safe, Trusted, Artificial General Intelligence. A safe-and-trusted system provably cannot be an AGI.

D. Evolutionary / substrate

Forrest Landry and Remmelt Ellen - The Formal Impossibility of AGI Safety: The Substrate Needs Convergence Proof.
Richard Everheart - On the Boundaries of Formal Intelligence.

E. Accountability and rule following

Haileleol Tibebu - The Accountability Horizon: An Impossibility Theorem for Governing Human-Agent Collectives.
Jan van Leeuwen and Jǐrí Wiedermann - Impossibility Results for the Online Verification of Ethical and Legal Behaviour of Robots
F. Measuring AGI
Greg Kamradt - President of Arc AGI; why current benchmarks saturate, why ARC-AGI is different, what a serious eval looks like.

F. Caveats and Rebuttals

Gabriel A. de Melo, Marcos R.O.A. Máximo, Nei Y. Soma, Paulo A.L. Castro - On the Undecidability of AI Alignment: Machines that Halt. Checking whether an arbitrary model is aligned is undecidable (Rice's theorem); a constructible aligned set still exists.
Matthew Fox - On Formally Undecidable Traits of Intelligent Machines. Argues against the Rice's-theorem impossibility line in Alfonseca et al.
Matt Luckcuck, Marie Farrell, Louise A. Deinns, Clare Dixon, and Michael Fisher - Formal Specification and Verification of Autonomous Robotic Systems: A Survey
Louise Dennis, Michael Fisher, Marija Slavkovik, Matt Webster - Formal verification of ethical choices in autonomous systems
Samuel Teuber - Provably Safe Neural Network Controllers via Differential Dynamic Logic.

Theory of impact

ASI may not just be likely or very likely to be harmful, but guaranteed to be harmful. If the formal case for uncontrollability holds, then decision-makers should substantially redirect how safety resources are allocated, what governance frameworks are pursued, and what the public is told about the trajectory of AGI development. We need rigorous, collaborative, and exploratory spaces to scrutinize proofs, understand where they fail, and close gaps in explanation and action where they do not, improving how the field reasons about controllability and how that is communicated to the right actors.

How the money will be spent

$1k Virtual seminars to work through proof premises and logic
$30k Physical workshop to do on whiteboards (venue, stay, food, ops)

$19k Team time

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate12d?