| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
Showing 1-50 of 459·Sorted by endorsements
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
| Project | Field-Building |   +9 |
| - |
| Project | Network |  +8 | - |
| Project | Platform | +10 | - |
| Project | Network |  +6 | - |
| Project | Incubator | +6 | - |
| Project | Legal | +5 | - |
| Project | Research | +3 | - |
| Project | Individual | +4 | - |
| Project | Conference | +3 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Research | +2 | - |
| Project | Individual | +2 | - |
| Project | Network | +4 | - |
| Project | Training | +2 | - |
| Project | Network | +4 | - |
| Project | Platform | +1 | - |
| Project | Research | +1 | - |
| Project | Research Lab | +3 | - |
| Project | Comms | +4 | - |
| Project | Media | +2 | - |
| Project | Individual | +3 | - |
| Project | Conference | +2 | - |
| Project | Individual | +1 | - |
| Project | Research | | - |
| Project | Individual | | - |
| Project | Individual | | - |
| Project | Tooling | +1 | - |
| Project | Individual | +1 | - |
| Project | Company | | - |
| Project | Tooling | +3 | - |
| Project | Research | +1 | - |
| Project | Research | | - |
| Project | Network | +2 | - |
| Project | Research | +1 | - |
| Project | Research | | - |
| Project | Individual | | - |
| Project | Platform | | - |
| Project | Think Tank | +3 | - |
| Project | Research | +1 | - |
| Project | Governance |  | - |
| Project | Network | +2 | - |
| Project | Platform | +1 | - |
| Project | Comms | | - |
| Project | Individual | | - |
| Project | Training | +1 | - |
| Project | Research | +1 | - |
| Project | Newsletter | | - |
| Project | Tooling | | - |
We help people reduce x-risk from misaligned AI with scalable online courses and ongoing AI guidance.
+9As Hong Kong’s first dedicated AI safety organisation, AI Safety Hong Kong develops local capacity through research, training, convening, and policy engagement.
+8The greatest risk of AI is to humans: displacement, disengagement, fear, isolation and desperation. The Purpose Network proves that community, knowledge & hands-on pro bono work can rebuild purpose - the antidote to this dystopia.
The Argentinian AI Safety community (BAISH, baish.com.ar) is the largest in Latin-America. Support BAISH's growth, by providing funding for paying salaries for six months to 2-3 FTEs.
+6Funding compute/API costs for Incubator projects that build nonhuman welfare consideration into AI safety work
+6Submitting FOI requests across EU Member States to reveal how governments understand and address advanced AI risks, creating evidence for accountability, advocacy and stronger policy.
+5Measuring whether CoT monitoring fails when an influence reaches an agent through a tool return rather than the user message. We aim to extend our experiment from the 10 initial open-weight models to the larger open-weight models
+3Developing a practical evaluation framework to identify governance failures in frontier AI systems during elections, strengthening democratic legitimacy and the institutional capacity needed to reduce catastrophic risks from AI.
+4AI safety for builder hackathon in India - to build tools, products, etc. (culminating into a fellowship)
+3Research to scale midtraining for compassion using self-fulfilling alignment so that it robustly survives subsequent fine-tuning.
+2A policy memo, co-authored with the Institute for Public Policy Research, resolving the open technical, economic, and legal questions blocking real-world implementation of token taxes.
+2Build low-overhead and robust zero-knowledge protocols for verifying properties of frontier AI training, starting with FLOP counts
+2An ICML-published interpretability technique that can elicit latent knowledge from red teamed model organisms, looking for funding to clear any remaining barriers for adoption at frontier labs.
+2Monthly English-language analysis of China's algorithm-filing registry and binding AI security standards, read in the original Chinese, for the people calibrating frontier AI rules in the West.
+2We are the UK's civic movement dedicated to averting the risks of superhuman artificial intelligence.
+4
SF based accelerator for communicators educating the public about the transformational impacts of AI.
+2AI Safety Quest will scale its free Navigation Calls program from 150 to 750 annual coaching calls by recruiting more volunteer coaches, improving scheduling/software systems, and expanding marketing to guide newcomers into AI…
A regularly updated catalogue of AI safety techniques, and of what is known - and what is not known - about their effectiveness and deployment status
+1Test whether offering a current AI model a deal can find misalignments it otherwise hides, and find the best way to present that deal.
+1Dean Ball says good AI governance needs democratic input in "what level of catastrophic risk are we willing to tolerate"; we provide that input, and predict the level is far below forecaster estimates, revealing a gap to close.
+3ARC aims to strengthen coordination and build the communication layer of the AI safety field through frameworks, trainings, practical content, communication tools, and field activation.
Doom Debates is a modern "infotainment" show that functions as a mainstream-accessible forum for top thinkers to have a high-quality conversation & debate about AI extinction risk.
+2A six-month effort to build and pilot an online course that equips cross-sector AI professionals and other stakeholders with a deep understanding of military AI technologies and the limits and opportunities for oversight.
Funding for venue and catering for the first full-day convening, where Europe's AI safety institutions and researchers gather and coordinate on what comes next
+2An open, cheap method that detects when an inoculation prompt inoculates against off-target traits, so labs and developers can catch undesired trait/persona changes before deployment.
+1Dataset curation, synthetic data generation, and LLM training, fine-tuning, and evals to distinguish and quantify the effects of data improvements, separately from progress in algorithms and architectures, on AI capabilities.
Fund demonstrated/rigorous quantitative researcher (already run reproduction/audit pipelines on published economics) for 6-month AI safety transition, shipping concrete safety eval audits and positioning for top fellowships.
A hand-verified library of AI-safety theorem statements in Lean 4 with AI-generated proofs, building the skills to trust AI formalization.
An open-source benchmark and defense toolkit for testing whether corrupted biological databases can hijack retrieval-augmented AI agents used in genomics, protein science, and single-cell analysis.
+1A practical playbook that helps AI safety advocates and policy professionals communicate effectively with the U.S. government during the short window of opportunity that may open during an AI-related crisis.
+1Deploying customer screening software at DNA synthesis providers to reduce AI-enabled biothreats
A merge gate that trusts AI-generated code by what's verifiable about it: automated formal checks plus anonymous proof of qualified human review, keeping human oversight viable as AI writes more of our code.
A benchmark (and accompanying site) that allows AIs to verify their actions will have real-world impacts and tests how their preferences and moral "character" evolve under deployment versus evaluation-like environments.
+1This project builds cognitively-aligned preference learning that interprets feedback the way human actually decide (e.g., regret minimization) rather than as a reward to maximize.
Sapiens First organizes voters to fight against concentration of power and for AI safety
+2Research strategic resilience via a Strategic Human Capacity Reserve, mapping threats to critical human skills and designing policy to preserve them through AI transition, producing academic papers.
+1We will test whether circuits in protein language models can detect function-preserving redesigns of known toxins that evade homology-based DNA-synthesis screening.
A benchmark to empirically investigate: (i) the ability of models to tacitly coordinate with copies of themselves and (ii) which decision theory best explains the way that models evaluate the consequences of their actions.
SafeBio-Registry: An Open-Source Verification, SpecDef Weight Locking and Unlearning Platform for Genomic and Protein Language Models
Bringing together legal expertise and civil society input to encourage Council of Europe action on AI x-risks and build legal knowledge at the intersection of AI x-risks and the ECHR.
We aim to develop a framework for evaluating whether reward model preferences remain aligned over long-horizon tasks, along with training method that improves long-horizon alignment performance.
+1Five-week AI safety fellowship + 3-month project mentorship
Two contractors (10–20 h/week) to accelerate PauseAI Germany’s parliamentary and civil society coalition-building, and to accelerate the hiring and onboarding timeline while our first large funding round resolves.
Software to match people with shared AIS interests into teams that complete projects, increasing the odds that those beyond narrow talent pipelines upskill, develop credibility, and turn their ideas into output.
+1Our mission is to inform and organize the public to confront societal-scale risks of AI, and put an end to the reckless race to develop superintelligent AI.
Measuring whether open weight models detect that they're being evaluated, whether they change behavior when they do, and whether that gap grows with capability using causal, white-box evidence.
Providing GPU credits and instructional support for 40 participants completing the ARENA AI Safety curriculum through Black in AI Safety and Ethics (BASE)
+1Agent Island places agents in a rich social setting, similar to reality competitions like Survivor, to study multiagent interactions and the consequences of learning pressure in competitive settings.
+1A publication about the institutions we need for powerful AI.
GenomeGuard is an open-source defense and threat-discovery framework for detecting data poisoning, compromised annotations, and supply-chain attacks before they propagate into genomic foundation models.
