AI safety projects actively seeking funding: what they’re working on and how much they need.
Showing 51-100 of 454 Top rated
The Sydney AI Safety Space is a co-working hub in Sydney, Australia, offering free office space for people working in the field of AI safety. Read more
+4Software to match people with shared AIS interests into teams that complete projects, increasing the odds that those beyond narrow talent pipelines upskill, develop credibility, and turn their ideas into output. Read more
+2Doom Debates is a modern "infotainment" show that functions as a mainstream-accessible forum for top thinkers to have a high-quality conversation & debate about AI extinction risk. Read more
+2AI risk assessment currently checks few threat models and doesn't compose them into the aggregate risk that matters. We'll build a tool mapping what frontier system cards cover and omit, plus a paper on what the assessments miss. Read more
+1Probe an open-weight model’s activations under biasing/cue conditions to test whether chain-of-thought explanations match internal reasoning, releasing paper, code, and datasets. Read more
+1TLDR: A representative survey with Yougov of the American public on questions about AI futures, including space governance, successionism, values. Read more
+1Paying 1-8 FTE to scale ControlAI's work meeting and briefing lawmakers about extinction risk from AI and building a coalition to prohibit superintelligence across the G7 (excl. US). Read more
An independent, multi-lineage panel of AI models, tested on whether it can identify welfare concerns in AI evaluations with opinions, dissents, and proposed modifications published in a public registry Read more
A 3 / 6 month builders fellowship where mentor-builder pods ship practical AI safety tools, not papers. Read more
Hosting a full day conference based in Sydney, Australia where young, aspiring students in senior high school and university interested in AI Safety can connect and share ideas Read more
An evaluation suite to identify a model’s legal values (e.g., anti-tech-regulation) relative to well-known actors (e.g., Ruth Bader Ginsburg) and an assessment of how language in a model’s constitution impacts the extent to which these values are human-aligned. Read more
A SCORE-style computational reproducibility audit of empirical AI safety research that estimates the field's base rate of reproducibility and generates a taxonomy of its failure modes. Read more
Map how undesired behaviors can silently spread between models during training, and which parts of the pipeline have the greatest risk — starting with the feedback processes used to align models. Read more
Auditing the social media presence of the top 25 AI safety organizations across major platforms to quantify the public communication gap and publishing a full gap analysis, then giving recommendations to the orgs for improvement. Read more
Research and plan advocacy for creating a UK Minister for Human Autonomy, including foundational research, project management, and drafting policy recommendations to safeguard human autonomy amid AI. Read more
Alien Cub is a production company focused on stories that inform a wide audience about risks from transformative AI. Read more
A specification-driven architecture for building more controllable and reliable long-horizon AI agents. Read more
Development of a sampling based zero knowledge proof for frontier AI training with 10% overhead. Read more
A video game that teaches people about AI alignment and race dynamics. Read more
We are the UK's civic movement dedicated to averting the risks of superhuman artificial intelligence. Read more
+4We have validated the artistic value of our show; this grant would test whether it can become a scalable and repeatable form of AI-safety outreach. Read more
Auditing production-style safety probes for silent failure on the world's other languages, and what fixing it costs. Read more
A six-month effort to build and pilot an online course that equips cross-sector AI professionals and other stakeholders with a deep understanding of military AI technologies and the limits and opportunities for oversight. Read more
Liaise helps close the generalist talent gap in AI safety by helping motivated non‑technical people enter with fluency, alignment, and a trusted network. Read more
A participant-led unconference gathering the global AI safety community, online and at local sites, Nov 20-22, 2026. Read more
It's La Jetée (1962), the time-traveling film later adapted as 12 Monkeys (1995), but it's about X-risk and inspired by AI 2027 and it's directed by, and starring, myself and @p8stie. Ergo, La P8stée Read more
Identifying reasoning pathologies/disingenuous behavior in reasoning traces based on activation dynamics rather than apparent semantics Read more
Short-term residencies (2–4 weeks) that bring high-context AI safety researchers to work at AI Safety Berlin’s coworking hub to connect with local researchers and career-transitioners and seed a European relocation pipeline. Read more
Career support for communications professionals who are looking to get into AI Safety. Includes regular coaching, referrals to roles, and introductions to people in the AI Safety space. Read more
Could cheap realignment instructions correct a drifting model in one turn — and if so, can the correction be made persistent? Allow me to test the idea against major frontier models, and publish the results! Read more
Testing whether activation probes recover safety signals from reasoning models as their chain-of-thought becomes illegible. Read more
Studying how structure in weight space, i.e., low-dimensional LoRA update geometry and sparse parameter subnetworks allow narrow fine-tuning to induce broad, unintended behaviours such as emergent misalignment, subliminal learning Read more
I have an automated control research scaffold. I'll use it to find the best control protocols to protect against various attacks and publish reports. Read more
Seeking a top-up grant to prioritize AI Safety Career Coaching. Read more
Building resilience in nucleic acid screening systems against split orders by integrating assembly pipelines into screening systems and validating resilience through test sets Read more
A quarterly Techplomacy Conversations series that turns AI x-risk research into direct, actionable recommendations for foreign ministries, UN missions, and AI companies. Read more
Benchmark for agents spending human money - paybench.org Read more
Building computational tools to identify and address pathological behavioral states in conversational and autonomous AI Read more
A high-production visual journalism series combining primary-source data and motion graphics to break down the mechanics, safety crises, linguistic biases, and Arab-world implications of transformative AI. Read more
Feature-length documentary that captures the intellectual and cultural zeitgeist surrounding frontier AI at the moment before AGI, and communicates X-Risk ideas to a broad range of elites. Read more
Multi-agent cooperative contractual obligation framework. Read more
Extend recently developed deception detection+steering method to larger and more diverse open-weight models, releasing the tooling that makes frontier-scale honesty interventions reproducible. Read more
An open-source library that both detects self-chosen deception in open-weight LLMs and steers the model back toward honesty at inference — the correction half that current honesty tools lack. Read more
An open benchmark measuring how model honesty survives long, pressured conversations and multi-agent interaction - lying, sycophancy, and calibration tracked turn by turn Read more
*Comprehension audits* are a novel development-process assurance mechanism to verify human understanding of AI research outputs to act as a gate to slow automation of AI R&D. Read more
6 months of salary support for my research and operations work to help found a quasi-governmental AI safety institution, the [Estonian AI Security Institute](https://www.aisi.ee/) (Turvalise Tehisaru Teadmuskeskus T3). Read more
Free, open, forkable safety infrastructure for agentic AI: a peer-reviewed pathology nosology, a safety runtime with reproducible benchmarks, and enforcement gating every agent tool call against human-authored policy. Read more
An open benchmark and causal interpretability study of when individually power-motivated LLM agents compete, form coalitions, collude, or betray—and whether internal signals reveal these shifts before behavior does. Read more
Drone swarms hunting people in the woods, as a less-dry video explainer of risks from AI. Read more
Neutral, reproducible benchmark measuring whether AI memory systems update correctly when facts change — every major system in one open table, October 2026. Read more
