Detailed Threat Models for AI-Enabled Extreme Power Concentration

Raising $20,000upcoming

Detailed models of how AI could allow a small set of actors to gain a decisive strategic advantage over the rest of the world: concrete pathways, required capabilities, quantified likelihoods, and the defenses that bind them.

Endorsed by

Marcus+1

Endorsed by

Marcus+1

Project Details

AI-enabled power concentration currently lacks detailed threat models which rigorously map out concrete pathways that lead from today's systems to entrenched control, what capabilities are required for each of these pathways, and how these different pathways and threats interact with each other. A lot of threat modelling is underspecified and narrowly scoped, in terms of the concrete pressures, incentives, and mechanisms, and also in the interplay between domestic consequences and outlooks and the broader strategic and security landscape.

This project is an attempt to fill this gap and create detailed models of how exactly AI could allow a small set of actors to gain a decisive strategic advantage over the rest of the world. I intend to spend 9 months driving an effort in this direction, drawing upon my own knowledge of technical AI safety, institutional design, nuclear verification, and international politics, and my connections with experts in relevant fields, in order to identify the pathways and relevant cruxes, evaluate the likelihood of these pathways and identify the possible outcomes, and map out possible interventions to defend against these pathways.

I intend to spend approximately 50% of my time, frontloaded into the initial 4-5 months, on creating and refining detailed threat models with concrete pathways, mechanisms, and quantified assessments of likelihood and impacts. About 30% of my time, mostly in months 4-8, will go into mapping existing and designing potential defenses for the specific pathways. About 20% of my time will go into engaging other people in the field and in adjacent expert communities, in order to get feedback, ensure minimal duplication of effort, and build a shared picture, closing with a small workshop. This is a nine-month programme costing roughly $130,000 in total. The funding ask in the grantmaking.ai round is scoped to its first phase.

At a minimum, I will publish six pieces of public-facing writing analysing specific facets of different pathways to power concentration, plus a comprehensive document detailing the pathways and potential defenses, circulated with relevant people in the community and shared publicly if doing so is not too risky. I will work solo and full-time; for relevant expertise, I can draw on my existing collaborations with the Alva Myrdal Centre and the Oxford Martin AI Governance Initiative.

Theory of Impact

AI-enabled power concentration seems to be one of the most critical threats we might face over the coming decade, and one of the ones we are least prepared for. A small set of actors seizing complete and entrenched power would vastly limit the ability of human civilization to prosper and flourish. In addition, the quest to gain this form of decisive advantage would also likely lead to reckless racing, increasing the risks of loss-of-control and potentially great power conflict.

My work is valuable as detailed threat models are the necessary inputs for understanding the nature of the threat, and identifying and prioritizing the potential interventions. Hence, the outputs of this project will be useful to funders and people working in the field, and would contribute to building an actionable agenda in order to rapidly scale future work in an effective manner.

People

Amritanshu Prasad

Team Member

Funding Details

Start Date: Jul 20, 2026
End Date: -
Expected Duration: 9 months
Funding Raised to Date: -
Annual Budget: -
Monthly Burn Rate: -
Current Runway: -
Funding Goal: -
Funding Stage: Seeking initial funding
Fiscal Sponsor: -

Track Record

Evaluating and Understanding Scheming Propensity in LLM Agents (arXiv:2603.01608), co-authored at LASR Labs, supervised by David Lindner (Google DeepMind).
Applying LLMs to Political Challenges in Nuclear Verification (in prep., with Giacomo Cassano, Uppsala). Presented at the Politics of Verification in Frontier AI workshop (Stockholm, April 2026) and the AMC Multidisciplinary Conference 2026.
Opportunities and Risks from Advanced AI in Nuclear Verification, forthcoming chapter with Sophia Hatz in the Routledge Handbook on Nuclear Verification.
CVD-for-AI working paper at Pivotal Research with Robert Trager (To be published)
Ongoing collaboration with Oxford Martin AI Governance Initiative on strategic parameters for AI agreements.
Contributor to ControlArena (UK AISI control evaluations) and the HCAST benchmark (arXiv:2503.17354) at METR.
Public writing on AI and nuclear governance at thenextfrontier.blog

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$20,000

Ideal

$44,000

What this ask is

This project is an attempt to fill this gap and create detailed models of how exactly AI could allow a small set of actors to gain a decisive strategic advantage over the rest of the world. I intend to spend 6 months driving an effort in this direction, drawing upon my own knowledge of technical AI safety, institutional design, nuclear verification, and international politics, and my connections with experts in relevant fields, in order to identify the pathways and relevant cruxes, evaluate the likelihood of these pathways and identify the possible outcomes, and map out possible interventions to defend against these pathways.

I intend to spend most of the initial months creating and refining the threat models, with concrete pathways, mechanisms, and quantified assessments of likelihood and impacts, and the rest on mapping existing and designing potential defenses for the specific pathways, while engaging others in the field and adjacent expert communities throughout, in order to get feedback, ensure minimal duplication of effort, and build a shared picture.

Theory of impact

AI-enabled power concentration seems to be one of the most critical threats we might face over the coming decade, and one of the ones we are least prepared for. A small set of actors seizing complete and entrenched power would vastly limit the ability of human civilization to prosper and flourish. In addition, the quest to gain this form of decisive advantage, would also likely lead to reckless racing, increasing the risks of loss-of-control and potentially great power conflict.

How the money will be spent

Ideal, $44,000 over six months full-time:

Stipend: $36,000 ($6,000/month)
Research compute, API, and AI tooling: $2,000
Travel (two lane conferences or workshops, inter-base flights): $4,000
Visa and setup buffer: $2,000

Minimum, $20,000 over three months full-time: stipend $18,000, compute and tooling $1,000, travel $1,000.

Discussion

SSophiahatzEndorsement3d

We're at a ripe moment to make progress on this problem, starting with detailed threat models. The projected is well-scoped, with clear, realistic and impactful deliverables.