AI safety projects actively seeking funding: what they’re working on and how much they need.
Showing 1-50 of 59 Top rated
SF based accelerator for communicators educating the public about the transformational impacts of AI. Read more
Funding for venue and catering for the first full-day convening, where Europe's AI safety institutions and researchers gather and coordinate on what comes next Read more
Fund demonstrated/rigorous quantitative researcher (already run reproduction/audit pipelines on published economics) for 6-month AI safety transition, shipping concrete safety eval audits and positioning for top fellowships. Read more
Safeguarding open-weight genomic foundation models through weight lock against adversarial finetuning Read more
I study how training processes produce models that behave deceptively and pursue hidden objectives, with scheming as the most consequential case. Read more
Implementing different types of unlearning methods for genomic and protein language models to remove sensitive biological information (e.g. pathogen virulence) while preserving predictive performance and scientific utility. Read more
Developing a practical evaluation framework to identify governance failures in frontier AI systems during elections, strengthening democratic legitimacy and the institutional capacity needed to reduce catastrophic risks from AI. Read more
AI safety for builder hackathon in India - to build tools, products, etc. (culminating into a fellowship) Read more
Agent Island places agents in a rich social setting, similar to reality competitions like Survivor, to study multiagent interactions and the consequences of learning pressure in competitive settings. Read more
Sapiens First organizes voters to fight against concentration of power and for AI safety Read more
A six-month effort to build and pilot an online course that equips cross-sector AI professionals and other stakeholders with a deep understanding of military AI technologies and the limits and opportunities for oversight. Read more
A quarterly Techplomacy Conversations series that turns AI x-risk research into direct, actionable recommendations for foreign ministries, UN missions, and AI companies. Read more
A pilot to find, screen and support overlooked African ML talent into frontier AI safety programs such as MATS and ARENA, adding new researchers to the alignment field’s talent pipeline. Read more
A first‑principles architecture for drift‑free, auditable, and safe AI agents built on governed runtimes. Read more
Measuring homogenization and social bias in LLMs and developing interventions to promote diversity. Read more
An AI safety community that trains professionals and researchers to become competent AI safety contributors within industries and the academia, by learning and doing something. Read more
Build an open adversarial benchmark and evaluation harness to stress-test model reversal/unlearning methods and diagnose whether unsafe capabilities are genuinely removed or merely suppressed. Read more
Develop long-term learning and memory retention in neuron-culture biocomputing via multi-day training protocols on Cortical Labs’ platform and an open-source light-microscope scanner to track structural changes. Read more
Security Gateway for AI Execution. Read more
Training survivors of human trafficking in ethical AI skills and connecting them to paid apprenticeships, turning technical training into sustainable tech careers. Read more
An open, honest way to test whether AI-agent memory systems spread false or unverifiable claims, plus a provenance standard that lets an agent's memory be audited, corrected, and forgotten. Read more
We make trusted regulations, standards, and scientific evidence machine-readable so claims can be checked automatically against authoritative sources. Read more
Real-time dynamical hidden-state safety telemetry for language models, predicting recursive collapse from latent geometry before it manifests in outputs. Read more
Building and validating a governance-first AI architecture that aims to reduce unsafe decisions under uncertainty, corruption, and conflicting evidence while preserving predictive performance. Read more
Tools and services to help organisations comply with the EU AI Act for High-Risk AI Systems Read more
Expand Humanity Tomorrow with a multilingual, jargon-free AI existential-risk section featuring a comprehensive FAQ, field map, action recommendations, and supporting visuals/audio, plus outreach and maintenance. Read more
Building Malawi’s AI Safety Youth Pipeline by training secondary school students to become the next generation of responsible AI researchers. Read more
A legal and policy decision-making platform driven by AI provides transparency and uses verifiable data to support its recommendations. It will reduce the number of times LLMs hallucinate and will allow for the safe use of LLMs in Read more
An open interpretability platform that enables researchers to inspect model internals, analyze latent representations, and detect hallucination or deceptive behavior in open-weight language models. Read more
An expert-elicited estimate of how far AI has closed the gap to superintelligence, scored by people with no financial stake in the answer, plus plain-language monitoring of what the six major labs actually change. Read more
A website for mentees and mentors to connect with each other to write papers and grow, like linkedin+github merged to one Read more
Alignment Infrastructure Routing (AIR) is an open-source, local-first network that connects AI safety labs, so they can scale talent and operations through shared, verifiable coordination standards. Read more
An online debate platform that offers cash purses to winning responses around topics covering the most prescient issues of our time. Read more
An open architecture for deterministic AI governance that separates policy enforcement from model behavior, enabling trustworthy deployment across providers. Read more
A fast, comprehensive directory of the people and orgs in AI safety: search, filter, and match. Read more
Requesting $1.2k to cover remaining travel, visa, and living costs to attend the Human-Aligned AI Summer School (HAAISS) in Prague to transition into technical AI safety research. Read more
One year of bootstrapped development, four patent filings, seeking support to continue. Read more
Mapping the attention heads that push LLMs toward refusal vs. compliance, and building an inference-time defense against both single- and multi-turn jailbreaks. Read more
Develop a training-time method for transformers that puts concepts where you can find them, so removal has predictable efficacy and bounded side-effects. Read more
A publication about the institutions we need for powerful AI. Read more
Scale and causally validate live monitors (EmotionMonitor) that detect and steer pre-commitment and frustration signals in thinking models, testing jailbreak relevance and attack robustness across model families. Read more
Produce and distribute a feature documentary using expert interviews and explanations of RSI, alignment, and AI x-risk/governance to educate the public and policymakers about frontier AI. Read more
Assessing whether Persona drift be detected in conditionally misaligned models using the assistant axis and potentially analyzing the reasons for failure Read more
Sapiens First organizes voters to fight concentration of power and for AI safety Read more
Translating formal verification into physical gravity-biased safety interlocks. We have a working prototype; funding builds 10-15 MIL-SPEC Beta units for red-te Read more
An AI social media project from a diverse content creator who has accumulated over 100k followers on 2 separate accounts Read more
EU-China AI governance and cooperation forum in Shanghai and Future of AI for Democracy workshop in Seoul Read more
Develop and validate PLM-embedding toxin screening that detects ProteinMPNN/RFdiffusion redesigns and maps the evasion boundary (black-box vs gradient access) using probes, SAEs, and attribution. Read more
I'm developing a system to keep advanced AI models safe, I've done some tests, but need more compute to do deeper tests. Read more
