| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
Showing 1-50 of 1.1k·Sorted by endorsements
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
| Project | Field-Building |   +9 |
| - |
| Project | Network |  +8 | - |
| Project | Platform | +10 | - |
| Project | Network |  +6 | - |
| Project | Incubator | +6 | - |
| Project | Legal | +5 | - |
| Project | Research | +3 | - |
| Project | Individual | +4 | - |
| Project | Conference | +3 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Individual | +2 | - |
| Project | Network | +4 | - |
| Project | Research Lab |  | - | - |
| Project | Research |    | - | - |
| Project | Training | +2 | - |
| Project | Network | +4 | - |
| Project | Platform | +1 | - |
| Project | Research | +1 | - |
| Project | Research Lab | +3 | - |
| Project | Comms | +4 | - |
| Project | Tooling |   | - | - |
| Project | Research | | - | - |
| Project | Media | +2 | - |
| Project | Individual | +3 | - |
| Project | Conference | +2 | - |
| Project | Tooling | | - | - |
| Project | Comms | | - | - |
| Project | Research |   | - | - |
| Project | Individual | +1 | - |
| Project | Research | | - |
| Project | Individual | | - |
| Project | Individual | | - |
| Project | Tooling | +1 | - |
| Project | Research |  | - | - |
| Project | Individual | +1 | - |
| Project | Company | | - |
| Project | Research | | - | - |
| Project | Tooling | +3 | - |
| Project | Research | +1 | - |
| Project | Research | | - |
| Project | Network | +2 | - |
| Project | Research | +1 | - |
| Project | Research | | - |
| Project | Individual | | - |
| Project | Platform | | - |
| Project | Think Tank | +3 | - |
| Project | Research | +1 | - |
We help people reduce x-risk from misaligned AI with scalable online courses and ongoing AI guidance.
+9As Hong Kong’s first dedicated AI safety organisation, AI Safety Hong Kong develops local capacity through research, training, convening, and policy engagement.
+8The greatest risk of AI is to humans: displacement, disengagement, fear, isolation and desperation. The Purpose Network proves that community, knowledge & hands-on pro bono work can rebuild purpose - the antidote to this dystopia.
The Argentinian AI Safety community (BAISH, baish.com.ar) is the largest in Latin-America. Support BAISH's growth, by providing funding for paying salaries for six months to 2-3 FTEs.
+6Funding compute/API costs for Incubator projects that build nonhuman welfare consideration into AI safety work
+6Submitting FOI requests across EU Member States to reveal how governments understand and address advanced AI risks, creating evidence for accountability, advocacy and stronger policy.
+5Measuring whether CoT monitoring fails when an influence reaches an agent through a tool return rather than the user message. We aim to extend our experiment from the 10 initial open-weight models to the larger open-weight models
+3Developing a practical evaluation framework to identify governance failures in frontier AI systems during elections, strengthening democratic legitimacy and the institutional capacity needed to reduce catastrophic risks from AI.
+4AI safety for builder hackathon in India - to build tools, products, etc. (culminating into a fellowship)
+3Research to scale midtraining for compassion using self-fulfilling alignment so that it robustly survives subsequent fine-tuning.
+2A policy memo, co-authored with the Institute for Public Policy Research, resolving the open technical, economic, and legal questions blocking real-world implementation of token taxes.
+2Build low-overhead and robust zero-knowledge protocols for verifying properties of frontier AI training, starting with FLOP counts
+2An ICML-published interpretability technique that can elicit latent knowledge from red teamed model organisms, looking for funding to clear any remaining barriers for adoption at frontier labs.
+2Monthly English-language analysis of China's algorithm-filing registry and binding AI security standards, read in the original Chinese, for the people calibrating frontier AI rules in the West.
+2We are the UK's civic movement dedicated to averting the risks of superhuman artificial intelligence.
+4Funding ends June 2025: Urgent support for proven AI safety pipeline converting technical talent from 26+ countries into published contributors
6-month funding for a team of researchers to assess a novel AI alignment research agenda that studies how structure forms in neural networks
SF based accelerator for communicators educating the public about the transformational impacts of AI.
+2AI Safety Quest will scale its free Navigation Calls program from 150 to 750 annual coaching calls by recruiting more volunteer coaches, improving scheduling/software systems, and expanding marketing to guide newcomers into AI…
A regularly updated catalogue of AI safety techniques, and of what is known - and what is not known - about their effectiveness and deployment status
+1Test whether offering a current AI model a deal can find misalignments it otherwise hides, and find the best way to present that deal.
+1Dean Ball says good AI governance needs democratic input in "what level of catastrophic risk are we willing to tolerate"; we provide that input, and predict the level is far below forecaster estimates, revealing a gap to close.
+3ARC aims to strengthen coordination and build the communication layer of the AI safety field through frameworks, trainings, practical content, communication tools, and field activation.
Funding to cover our expenses for 3 months during unexpected shortfall
AI Futures Project
Doom Debates is a modern "infotainment" show that functions as a mainstream-accessible forum for top thinkers to have a high-quality conversation & debate about AI extinction risk.
+2A six-month effort to build and pilot an online course that equips cross-sector AI professionals and other stakeholders with a deep understanding of military AI technologies and the limits and opportunities for oversight.
Funding for venue and catering for the first full-day convening, where Europe's AI safety institutions and researchers gather and coordinate on what comes next
+2TransformerBridge enables loading any PyTorch nn.Module (including HF transformers) into TransformerLens via a single config file, reducing reimplementation work and supporting broader interpretability use cases.
20 Weeks Salary to reach a neglected audience of 10M viewers
Making sure AI systems don't mess up acausal interactions
An open, cheap method that detects when an inoculation prompt inoculates against off-target traits, so labs and developers can catch undesired trait/persona changes before deployment.
+1Dataset curation, synthetic data generation, and LLM training, fine-tuning, and evals to distinguish and quantify the effects of data improvements, separately from progress in algorithms and architectures, on AI capabilities.
Fund demonstrated/rigorous quantitative researcher (already run reproduction/audit pipelines on published economics) for 6-month AI safety transition, shipping concrete safety eval audits and positioning for top fellowships.
A hand-verified library of AI-safety theorem statements in Lean 4 with AI-generated proofs, building the skills to trust AI formalization.
An open-source benchmark and defense toolkit for testing whether corrupted biological databases can hijack retrieval-augmented AI agents used in genomics, protein science, and single-cell analysis.
+1Surveying neuroscience for tools to analyze and understand neural networks and building a natural science of deep learning
A practical playbook that helps AI safety advocates and policy professionals communicate effectively with the U.S. government during the short window of opportunity that may open during an AI-related crisis.
+1Deploying customer screening software at DNA synthesis providers to reduce AI-enabled biothreats
Expanding proven isolation techniques to high-risk capability domains in Mixture of Expert models
A merge gate that trusts AI-generated code by what's verifiable about it: automated formal checks plus anonymous proof of qualified human review, keeping human oversight viable as AI writes more of our code.
A benchmark (and accompanying site) that allows AIs to verify their actions will have real-world impacts and tests how their preferences and moral "character" evolve under deployment versus evaluation-like environments.
+1This project builds cognitively-aligned preference learning that interprets feedback the way human actually decide (e.g., regret minimization) rather than as a reward to maximize.
Sapiens First organizes voters to fight against concentration of power and for AI safety
+2Research strategic resilience via a Strategic Human Capacity Reserve, mapping threats to critical human skills and designing policy to preserve them through AI transition, producing academic papers.
+1We will test whether circuits in protein language models can detect function-preserving redesigns of known toxins that evade homology-based DNA-synthesis screening.
A benchmark to empirically investigate: (i) the ability of models to tacitly coordinate with copies of themselves and (ii) which decision theory best explains the way that models evaluate the consequences of their actions.
SafeBio-Registry: An Open-Source Verification, SpecDef Weight Locking and Unlearning Platform for Genomic and Protein Language Models
Bringing together legal expertise and civil society input to encourage Council of Europe action on AI x-risks and build legal knowledge at the intersection of AI x-risks and the ECHR.
We aim to develop a framework for evaluating whether reward model preferences remain aligned over long-horizon tasks, along with training method that improves long-horizon alignment performance.
+1
