Dean Ball says good AI governance needs democratic input in "what level of catastrophic risk are we willing to tolerate"; we provide that input, and predict the level is far below forecaster estimates, revealing a gap to close.
Ryan+2Dean Ball says good AI governance needs democratic input in "what level of catastrophic risk are we willing to tolerate"; we provide that input, and predict the level is far below forecaster estimates, revealing a gap to close.
Ryan+2See detailed draft protocol here.
What we'll do and why
This project measures the US public's catastrophic risk tolerance. Prior public surveys have focused on attitudes and appetite for regulation. However, the public lack the AI literacy to knowledgeably prioritise risks, or to rate effective regulations. But, given AI imposes involuntary societal risks, the public do likely expect and deserve some say in the level of risk they are exposed to. So, we follow recommendations from Hadfield and Clark to separate the 'technical deficit' from the 'democratic deficit.' We reduce the democratic deficit by assessing the level of catastrophic risk the public are willing to tolerate. While this is hard to measure directly, we plan to triangulate four different methods for assessing their risk tolerance. We'll collected a representative sample of ~4,000 americans of voting age. Our prior work in Australia showed the public expect safety akin to commercial aviation. This would be orders of magnitude safer than expert forecasts imply for AI. If replicated in the USA, and confirmed with more robust methods, it would mean the status quo is intolerable by the public's lights.
Who's involved
The project would be led by the team at the MIT AI Risk Initiative (AIRI): Michael Noetel, Peter Slattery, and Alexander Saeri, supervised by MIT FutureTech lab director, Neil Thompson. AIRI is the number one result for 'AI risk' on Google and in ChatGPT. Our work has been widely cited in research papers, theses, curricula, reports, evidence submissions, presentations, and standards.
This grant would fund the survey recruitment costs and a research assistant to help with project delivery.
Concrete output
As per our Australian SARA2025 survey (self-funded by the researchers; examples from that project are linked), we will produce:
Per our SARA2025 survey, we will publish all data regardless of what we find to maintain integrity and avoid selective reporting.
Our project aims to provide the US government with the catastrophic risk tolerance of their citizens. It aims to assess if there is a gap between that tolerance and the risk perception of experts. If there is a gap, then we aim to share the gap with those policymakers to provide an impetus to close it. This would increase the political will for mitigations until the desired risk tolerances are met. We ultimately hope it provides democratic support for policy decisions that increase safety, like mandatory third-party supervision of frontier AI development.
Team Member
Only visible to verified funders, reviewers, and admins.
Email hi@grantmaking.ai to get verifiedAIRI is the top-ranked source for "AI risk" on Google and in ChatGPT. Our MIT AI Risk Initiative work has been widely cited in research papers, theses, curricula, reports, , , , and . In 2025, we received more website visitors (111k) than (), which has over 30 employees and a budget over USD . Despite being a new and small team, our reach already exceeds that of institutes with much larger teams and budgets. Our work has been featured in major media outlets including , , , and received the most of any research produced by MIT over 2024–2025. It has been adopted by the AI Incident Database, IBM, and Credo AI, with a testimonial from Yoshua Bengio. The team has been invited to brief or engage with governance bodies including the and .
Only visible to verified funders, reviewers, and admins.
Email hi@grantmaking.ai to get verifiedThe min would cover the survey fielding costs for 4,000 US voters, and research assistant costs for 8 weeks.
The ideal also covers a UK sample and an additional 4 weeks of research assistant time.
Spreadsheet here
It seems reasonable to expect the team to succeed in this research plan, given their track record, and providing robust evidence around gaps in perception of safety supports informed public debate and policy deliberations. Endorse.
We have excellent supervision and support to help us have an impact. We’re part of MIT FutureTech, the largest lab in MIT’s Computer Science and AI Laboratory (CSAIL). FutureTech’s director, Neil Thompson, has been invited to present his work to congressional staffers (House and Senate), the US Federal Reserve, the Pentagon, National Security Staff, the Department of Commerce, the Department of Energy, Brookings Institution, and recently presented at a World Summit on the same program as the Prime Minister of India and Former Prime Ministers of England and Australia. He has been instrumental in helping us get similar opportunities to translate our research into policy.
Each of us brings unique strengths. Alexander Saeri directs the initiative, led the first Survey Assessing Risks from AI, and has a track record of translating complex research into practical tools for decision-makers. Michael Noetel is a highly cited Associate Professor at the University of Queensland, chair of Effective Altruism Australia, and a Faculty Member of the Future of Life AI Existential Safety Community. Peter Slattery has 68,000+ LinkedIn followers, received almost 4 million impressions in 2025–2026, and was ranked as the 58th most influential LinkedIn voice in AI. Simon Mylius has built LLM-enabled infrastructure for tracking and classifying AI incidents and AI governance. Jess Graham and Kun Zhao have driven large swathes of our work. Each of us has been deeply engaged with catastrophic risk, with the knowledge and skills to make important contributions. Our plan with this grant is to deploy those skills alongside AIRI’s accumulated credibility.
I like this idea and I value approaches that provide evidence for informed policy decisions.
I think there's value in being able to compare the results from your earlier Australian research with a comparable US / UK survey - and perhaps a formalised survey methodology that could be utilised in other countries to help build out some less English speaking / Western findings.
I'm not convinced that the public is good at estimating small risks, which is one of the issues that your methdology notes, and I agree that the findings are most useful in terms of direction.
I think your approach of making comparisons with other fields (eg regulation of air travel) is great, both as a way of making the risks and risk treatments more approachable for people, and also as potential media soundbites.