grantmaking.ai
Actively FundraisingRecent ActivityFull Database
Resources
grantmaking.ai
Actively FundraisingRecent ActivityFull Database
ResourcesApply for funding
grantmaking.ai kickoff grant round$328k / $1M distributed
Get funded

Database

The AI Safety database is a work in progress, as we are focusing on the launch of the initial $1M grant round. Feel free to send us ideas or feedback!
Raising funds(remove filter)Entity: Project(remove filter)Clear all
NameTypeTagsEndorsedTeamRaising $

Showing 401-450 of 461·Sorted by endorsements

NameTypeTagsEndorsedTeamRaising $
AIR: Platform Mitigating X-Risk through Human Capacity Ops
Project
ToolingGovernancePlatform
--

AIR: Platform Mitigating X-Risk through Human Capacity Ops

Team?
Project
Previous

Page 9 of 10

Next
Beaufbourrin: A Networked Guide to the 2020s Manosphere
Project
ResearchGovernanceIndividual
-
-
Existential
Project
CompanyToolingRationality
--
Reverse engineer conditions for resilience and flourishing
Project
ResearchGovernance
--
Investigate AI incidents
Project
ToolingSecurity
--
AI Internet-Meritocracy
Project
CompanyComms
--
Animal Welfare Constitutions and Evals
Project
AI WelfareResearchValue Alignment
--
Descry AI
Project
TrainingIncubatorTechnical Safety
--
Value Compass
Project
GovernancePlatformComms
--
SocialDeductionBench - A Deception Benchmark for LLMs
Project
DeceptionResearchEvals
--
Intent Preservation Platform
Project
ToolingEvals
--
Physics verification for AI-generated hardware designs
Project
ToolingOversight
--
Physics-based early warnings for interpretability failures
Project
InterpResearchIndividual
--
QuariumTech
Project
CompanyResearchTechnical Safety
--
Empathic Dissociations In Embodied AI
Project
ResearchTechnical SafetyResearch Lab
--
AI Safety Bootcamp - Kenya
Project
GovernanceTraining
--
SWEET: local, verifiable detection of machine influence on choice
Project
ToolingGovernance
--
Developing methods to evaluate the trustworthiness of CV models
Project
ResearchEvals
--
KAM: an economic mechanism against white-collar AI displacement
Project
CompanyResearchGovernance
--
Manhattan Kernel
Project
ToolingControlIndividual
--
Prototyping Browser-Based Misinformation User Flows
Project
CompanyResearchDemocratic AI
--
ONYX Research Cloud: Trustworthy AI Infrastructure for Brain Health
Project
CompanyAI WelfareTooling
--
The Librairy
Project
NewsletterCommsX-Risk
--
New alignment approach, better than training AI on safe behaviour
Project
Field-BuildingIndividualTechnical Safety
--
A fail-closed control layer and evals for autonomous agent fleets
Project
ControlResearchEvals
--
MUSEUM: Multiscale Unified Systemic Existential Uncertainty Modeling
Project
ResearchX-Risk
--
AI Safety Evals and Red Teaming Platform
Project
ToolingPlatformEvals
--
Evaluating Prompt Injection Defenses in LLMs
Project
ResearchSecurity
--
Scale Up: AI Development Index & Model Evals for Fabrication Benchmark
Project
ResearchPlatformEvals
--
Healthy Human–AI Collaboration through CI Theory
Project
AI WelfareResearchIndividual
--
We Are Hokmah: Comparative Messenger Pilot
Project
ResearchGovernanceComms
--
Creedal AI
Project
ResearchValue Alignment
--
Pragmatic value alignment of LLMs to influence downstream behavoirs
Project
ResearchIndividualValue Alignment
--
DriftBench v1.1: an ambivalence metric + 20 cross-domain scenarios
Project
ToolingEvals
--
WorkReady Academy: AI Transformation Academy
Project
GovernancePlatformEducation
--
The Veil: which human variables move the internals AI oversight reads?
Project
InterpResearch
--
Lighthouse: Behavioral Governance for Conversational AI
Project
ResearchEvals
--
AI Panopticon
Project
ResearchIndividualTechnical Safety
--
Animal Intelligence StoryTelling Project
Project
Technical SafetyComms
--
Double Tagging a Survey Data for Real World Grounding
Project
ResearchForecasting
--
AI-Enhanced Smart Clinic Chair Platform
Project
ResearchResearch Lab
--
AI Risk Sensemaking Through Interactive Future Scenarios
Project
GovernanceComms
--
MYCELIA- a consensus based AI with predictive & prescriptive telemetry
Project
ResearchRobustness
--
Beamline - Retroactive analysis of systems to create shared truths
Project
ToolingControl
--
Human Oversight Framework for AI Systems: Building Governance Capacity
Project
ResearchGovernanceIndividual
--
MiR
Project
MediaGovernanceComms
--
Auditing How AI Systems Convert Political Questions Into Technical One
Project
ToolingGovernanceIndividual
--
Cantivia
Project
CompanyToolingSecurity
--
AEGIS: Evidence-Backed Control Layer Reducing AI X-Risk
Project
CompanyToolingControl
--
Safety framework for animate materials
Project
ResearchTechnical Safety
--
Tooling
Governance
Platform
Fundraising
Claimed

Alignment Infrastructure Routing (AIR) is an open-source, local-first network that connects AI safety labs, so they can scale talent and operations through shared, verifiable coordination standards.

Led byBasil Korompilias
Endorsed by-

Beaufbourrin: A Networked Guide to the 2020s Manosphere

Team?
ProjectResearchGovernanceIndividualFundraisingClaimed

Research maps manosphere-related masculinity subcultures and semantic drift across countries to inform AI alignment, lexicon, and sentiment/stylometry design that reduces radicalization, bullying, and violence risks.

Led byTanner Durant
Endorsed by-

Existential

Team?
ProjectCompanyToolingRationalityFundraisingClaimed

A local-first AI that strengthens your own reasoning instead of replacing it, and keeps your thinking on your own device.

Led byRena O'Brien
Endorsed by-

Reverse engineer conditions for resilience and flourishing

Team?
ProjectResearchGovernanceFundraisingClaimed

I am an expert in sociotechnical systems. I know that increasingly power, trust, virtue, … are emergent properties whose conditions may be analysed and reverse engineered through systems and policy at least.

Led byMegan
Endorsed by-

Investigate AI incidents

Team?
ProjectToolingSecurityFundraisingClaimed

Developing investigation methods for AI incidents

Led byElena Sirbu
Endorsed by-

AI Internet-Meritocracy

Team?
ProjectCompanyCommsFundraisingClaimed

My app asks AI, what portion of the global GDP a person is worth, and gives accordingly.

Led byVictor Porton
Endorsed by-

Animal Welfare Constitutions and Evals

Team?
ProjectAI WelfareResearchValue AlignmentFundraisingClaimed

Open evaluations that measure how models reason about and act towards animals, paired with constitutional principles drawn from animal welfare and cognate bodies of law, that labs can train against.

Led byPanashe Zowa
Endorsed by-

Descry AI

Team?
ProjectTrainingIncubatorTechnical SafetyFundraisingClaimed

Descry AI is an 8-week AI safety technical talent incubator targeted to talented high school students in the Global South that aims to provide a head-start on doing open-source research that reduces catastrophic risks from AI.

Led byEdric Castel Hao
Endorsed by-

Value Compass

Team?
ProjectGovernancePlatformCommsFundraisingClaimed

A values and concentration map that exposes the values of AI models and tools makers, and those behind the makers (funders, jurisdictions etc) to the public (individuals and organizations) so they can vote with their choices.

Led byYuyu Shen
Endorsed by-

SocialDeductionBench - A Deception Benchmark for LLMs

Team?
ProjectDeceptionResearchEvalsFundraisingClaimed

A benchmark that measures how successful models are at social deduction games and if there is any trade-off between this skill and safety guardrails.

Led byTimothy Obiso
Endorsed by-

Intent Preservation Platform

Team?
ProjectToolingEvalsFundraisingClaimed

An open-source AI safety platform for evaluating how and where large language models preserve human intent during complex information transformation.

Led byKhianna Deseide
Endorsed by-

Physics verification for AI-generated hardware designs

Team?
ProjectToolingOversightFundraisingClaimed

Builds a physics-based verifier for AI-generated hardware that flags unverifiable aspects as UNCHECKED, enabling a propose-and-check loop to test and improve parts and assemblies for strength and fit.

Led byPeter Boctor
Endorsed by-

Physics-based early warnings for interpretability failures

Team?
ProjectInterpResearchIndividualFundraisingClaimed

Early warning signals that help identify when interpretability results stop being trustworthy, motivated by theories in statistical physics.

Led byWill Pfalzgraff
Endorsed by-

QuariumTech

Team?
ProjectCompanyResearchTechnical SafetyFundraisingClaimed

Discovery does not equal truth.

Led byStelle Jacobson
Endorsed by-

Empathic Dissociations In Embodied AI

Team?
ProjectResearchTechnical SafetyResearch LabFundraisingClaimed

Funding to launch a two-person institute researching embodied AI x-risk via LLM misalignment evaluations and interpretability, and producing governance, policy, and design guidelines for physical-world deployment.

Led byRoshni Lulla
Endorsed by-

AI Safety Bootcamp - Kenya

Team?
ProjectGovernanceTrainingFundraisingClaimed

A six month capacity building program that will equip 100 youth (17-30) with practical skills in AI Safety, Responsible AI to emphasize safe, ethical and human centered AI Development while creating pathways to further education

Led bySeth Momanyi Ouko
Endorsed by-

SWEET: local, verifiable detection of machine influence on choice

Team?
ProjectToolingGovernanceFundraisingClaimed

Look at any of my work, maybe start at onwardai.ai, it is live but nowhere near where it will be, but if you dont get it from where it is at you will never get it at all

Led byAaron Lott
Endorsed by-

Developing methods to evaluate the trustworthiness of CV models

Team?
ProjectResearchEvalsFundraisingClaimed

Developing evaluation methods to determine when CV models can be trusted by addressing the gap between benchmark metrics and real-world performance.

Led byElena Eroshina
Endorsed by-

KAM: an economic mechanism against white-collar AI displacement

Team?
ProjectCompanyResearchGovernanceFundraisingClaimed

Selma: The AI that never leaves the building.

Led byKara Bombell, and others
Endorsed by-

Manhattan Kernel

Team?
ProjectToolingControlIndividualFundraisingClaimed

A deterministic Rust kernel that isolates dangerous LLM agents inside a strict compiler-integrated safety sandbox.

Led byLászló Lipcsik
Endorsed by-

Prototyping Browser-Based Misinformation User Flows

Team?
ProjectCompanyResearchDemocratic AIFundraisingClaimed

Qualitative UX research exploring how users respond to different prototypes of a browser-based tool for identifying online misinformation.

Led byCindy Gilbertson
Endorsed by-

ONYX Research Cloud: Trustworthy AI Infrastructure for Brain Health

Team?
ProjectCompanyAI WelfareToolingFundraisingClaimed

Building reproducible, transparent, and auditable AI infrastructure for neuroscience and clinical brain health.

Led byAffaan Shaikh
Endorsed by-

The Librairy

Team?
ProjectNewsletterCommsX-RiskFundraisingClaimed

The Librairy is a weekly newsletter that helps individuals and groups outside the field understand and be equipped with AI safety knowledge and resources.

Led bySakshi Chaubey
Endorsed by-

New alignment approach, better than training AI on safe behaviour

Team?
ProjectField-BuildingIndividualTechnical SafetyFundraisingClaimed

I've already shown that the approach is very promising. This project is about making people at AI labs and alignment academics aware of it first, then personally collaborate with them (or make others work on it, even w/o me).

Led byMichele Campolo
Endorsed by-

A fail-closed control layer and evals for autonomous agent fleets

Team?
ProjectControlResearchEvalsFundraisingClaimed

A fail-closed control plane for coding-agent fleets — spend caps, audited actions, rollback — plus a public eval suite and automated red-team explorer that measure whether any control layer actually stops unsafe agent actions.

Led byJackie YU
Endorsed by-

MUSEUM: Multiscale Unified Systemic Existential Uncertainty Modeling

Team?
ProjectResearchX-RiskFundraisingClaimed

LPMs for modeling x-risk patterns across human taxonomies and agentic swarms

Led byDave Haas
Endorsed by-

AI Safety Evals and Red Teaming Platform

Team?
ProjectToolingPlatformEvalsFundraisingClaimed

A no-code AI safety evaluation (including red teaming) tool that enables non-technical domain experts (e.g. social scientists, STEM experts) as well as AI engineers to design, run, and communicate effective and efficient evals.

Led byJeanice Koorndijk
Endorsed by-

Evaluating Prompt Injection Defenses in LLMs

Team?
ProjectResearchSecurityFundraisingClaimed

An independent evaluation of current prompt injection defenses in large language models, producing practical recommendations for safer AI deployment.

Led byIsmail Magan
Endorsed by-

Scale Up: AI Development Index & Model Evals for Fabrication Benchmark

Team?
ProjectResearchPlatformEvalsFundraisingClaimed

Expanding a continuously updated database covering global panel AI data as a ground-truth benchmark which allows stress-testing frontier LLMs’ fabrication rates in global contexts (lower-income and upper-income countries’ contexts

Led byJason Hung
Endorsed by-

Healthy Human–AI Collaboration through CI Theory

Team?
ProjectAI WelfareResearchIndividualFundraisingClaimed

Developing CI Theory and the Exogram framework to enable healthy, sustainable human–AI collaboration and reduce long-term AI risk.

Led byHiroto Onda
Endorsed by-

We Are Hokmah: Comparative Messenger Pilot

Team?
ProjectResearchGovernanceCommsFundraisingClaimed

Testing, in Washington, DC, if a AI generated messenger can build trust in communities largely unseen in AI Safety as well as a human being.

Led byKaren Maria Alston
Endorsed by-

Creedal AI

Team?
ProjectResearchValue AlignmentFundraisingClaimed

A first-person creed for AI defining what the AI is for rather than what it must not do, to be internalized during training, not tacked on after.

Led bySamareh Jack
Endorsed by-

Pragmatic value alignment of LLMs to influence downstream behavoirs

Team?
ProjectResearchIndividualValue AlignmentFundraisingClaimed

Four-month project to fine-tune LLMs on public value surveys, evaluate behavior via a text-based agent simulator, and run a public consultation comparing model actions to respondents’ values.

Led byJames Thompson
Endorsed by-

DriftBench v1.1: an ambivalence metric + 20 cross-domain scenarios

Team?
ProjectToolingEvalsFundraisingClaimed

Deterministic, no-LLM-judge benchmark for how faithfully AI tracks changing beliefs. Funding v1.1: a new ambivalence metric + 20 cross-domain scenarios.

Led byOleksii Simon
Endorsed by-

WorkReady Academy: AI Transformation Academy

Team?
ProjectGovernancePlatformEducationFundraisingClaimed

Equipping our society for people-centric AI transformation

Led byKeith Thode
Endorsed by-

The Veil: which human variables move the internals AI oversight reads?

Team?
ProjectInterpResearchFundraisingClaimed

The Veil measures how the other side of the exchange, human or artificial, changes where and how a language model computes, toward testing whether those shifts confound the activation-based tools AI safety relies on.

Led byPetya Sarafova
Endorsed by-

Lighthouse: Behavioral Governance for Conversational AI

Team?
ProjectResearchEvalsFundraisingClaimed

Researching whether many conversational AI failures share underlying behavioral dimensions — and whether those dimensions can be independently governed.

Led byKathlyn Kelly, and others
Endorsed by-

AI Panopticon

Team?
ProjectResearchIndividualTechnical SafetyFundraisingClaimed

Develop a decision-theory framework for AI self-alignment in nested, cyclic, partially observable multi-agent environments where delayed punishment for welfare compromise promotes cooperative behavior.

Led byEduard Kapelko
Endorsed by-

Animal Intelligence StoryTelling Project

Team?
ProjectTechnical SafetyCommsFundraisingClaimed

A transmedia storytelling project using an animal characters as a metaphor for AI

Led byWill Shin
Endorsed by-

Double Tagging a Survey Data for Real World Grounding

Team?
ProjectResearchForecastingFundraisingClaimed

a person takes an occupational Survey data (+1 or 2 fresh, third party) +a existing alignment emulation model / trigger analysis mod "what would happen" emu then do all sorts of manual tagging, for maximum real world grounding

Led byAndrew Djuwidja
Endorsed by-

AI-Enhanced Smart Clinic Chair Platform

Team?
ProjectResearchResearch LabFundraisingClaimed

Long-term vision is to transform the Smart Clinic Chair from a telehealth device into an AI-enhanced remote examination room capable of extending clinical intelligence into healthcare deserts, workplaces, transportation truckstops

Led byvincent waterson
Endorsed by-

AI Risk Sensemaking Through Interactive Future Scenarios

Team?
ProjectGovernanceCommsFundraisingClaimed

12-week project to build and user-test a web interactive narrative prototype that communicates AI risk scenarios to non-technical young adults, producing notes, feedback data, and a public write-up.

Led bywei jiang
Endorsed by-

MYCELIA- a consensus based AI with predictive & prescriptive telemetry

Team?
ProjectResearchRobustnessFundraisingClaimed

Language model where its attention-heads are not competing but consenting. Its governors monitor the hidden state geometry and provide early-warning by predicting failure and timely modify latent processes to avoid instabilities

Led byDaniel Solis
Endorsed by-

Beamline - Retroactive analysis of systems to create shared truths

Team?
ProjectToolingControlFundraisingClaimed

This architecture uses retroactive observation to identify and close system blind spots, leading to better alignment over time.

Led byLeo Guinan
Endorsed by-

Human Oversight Framework for AI Systems: Building Governance Capacity

Team?
ProjectResearchGovernanceIndividualFundraisingClaimed

Developing practical AI oversight frameworks that keep humans in control of consequential AI decisions, starting where governance capacity is weakest.

Led byAgwu Naomi Nneoma
Endorsed by-

MiR

Team?
ProjectMediaGovernanceCommsFundraisingClaimed

A short film on the moral gray of AI: a single conversation between Elena, a celebrated founder whose elder-care AI harmed the people it soothed, and Joanne, the journalist who championed her and now holds her accountable.

Led byCarolina Groppa
Endorsed by-

Auditing How AI Systems Convert Political Questions Into Technical One

Team?
ProjectToolingGovernanceIndividualFundraisingClaimed

AI that surfaces power-concentration risk in civic systems by making value trade-offs visible instead of resolving them silently.

Led byNaomi Munroe
Endorsed by-

Cantivia

Team?
ProjectCompanyToolingSecurityFundraisingClaimed

Cantivia detects deepfakes and disinformation, and simulates how synthetic media spreads, so platforms, newsrooms and fraud teams know what's fake, who made it, and how far it'll travel.

Led byAshkan Samali
Endorsed by-

AEGIS: Evidence-Backed Control Layer Reducing AI X-Risk

Team?
ProjectCompanyToolingControlFundraisingClaimed

An inline authorization platform for AI agent actions

Led byVikas Prakash
Endorsed by-

Safety framework for animate materials

Team?
ProjectResearchTechnical SafetyFundraisingClaimed

A public, cross-disciplinary safety framework for emerging animate matter technologies

Led byGeorge Blanda
Endorsed by-