Project
Database
The AI Safety database is a work in progress, built from public sources. Missing, or out of date? You can list your project yourself. Feel free to send us ideas or feedback!
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
Platform
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
Showing 51-100 of 117·Sorted by endorsements
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
| Project | Network | - | - | ||
| Project |
Network |
| - |
| - |
| Project | Network | - | - |
| Project | Conference | - | - |
| Project | Research | - | - |
| Project | Individual | - | - |
| Project | Training | - | - |
| Project | Training | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Tooling | - | - |
| Project | Individual | - | - |
| Project | Think Tank | - | - |
| Project | Standards | - | - |
| Project | Research | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Individual | - | - |
| Project | Training | - | - |
| Project | Company | - | - |
| Project | Conference | - | - |
| Project | Research | - | - |
| Project | Platform | - | - |
| Project | Individual | - | - |
| Project | Research | - | - |
| Project | Research Lab | - | - |
| Project | Individual | - | - |
| Project | Platform | - | - |
| Project | Research | - | - |
| Project | Tooling | - | - |
| Project | Media | - | - |
| Project | Tooling | - | - |
| Project | Individual | - | - |
| Project | Comms | - | - |
| Project | Research | - | - |
| Project | Research | - | - |
| Project | Research | - | - |
| Project | Platform | - | - |
| Project | Research | - | - |
| Project | Individual | - | - |
| Project | Media | - | - |
| Project | Tooling | - | - |
| Project | Tooling | - | - |
| Project | Research | - | - |
| Project | Company | - | - |
| Project | Education | - | - |
| Project | Research | - | - |
Expanding and facilitating the growth of the Lisbon AI Safety Hub (LAISH)
PauseAI Canada organizes media outreach, public events and grassroots advocacy to secure Canadian support for an international treaty pausing frontier AI training.
NTU AI Safety expands Taiwan's AI safety community through technical and policy reading groups, speaker events, professional outreach, and international partnerships.
Building Evaluator Capacity in Asia
Funds travel to GigaNet and the UN Internet Governance Forum to present an index scoring AI governance frameworks on concentration, reversibility and efficiency.
Help me travel to Sydney to present my accepted NeurIPS 2026 research paper. I need help with visa and flight costs.
Course fees for Cambridge University PACE AI Engineering Career Accelerator
Help gain a technical understanding of AI Engineering to enable effective check and challenge AI safety using a technical lens
NeurIPS 2026 Workshop on Evaluation of Interactive Agents (IAEval), AI for Verifiable Coding (VERICODEGEN)
Seeking travel funding to present our accepted safety eval paper at NeurIPS 2026 in Sydney (first author).
Heldout testing of an action-claim verifier, a fix for fail-open verifier and a standalone library, failures disclosed.
In-memory latent trajectory auditing for 0-token prefill eviction and 1–3 token circuit breakers across 70B LLMs.
AI Governance, Agentic Liability, Model Evaluation, and Conceptual Safety Tools
A European research programme developing practical mechanisms for collective access to frontier AI models for independent safety assurance.
Finish an open AI-disclosure protocol (GTC v1.0) and publish an open method for checking whether an agent's action can be undone
Open source research on model consciousness. Both novel research and fieldbuilding.
We will test whether AI assistance that performs developing cognitive operations weakens adolescents’ later independent judgment, and develop evidence-based safeguards that keep human reasoning meaningfully in the loop.
Testing a cheap, style-based safety bypass across many languages, and releasing open safety judges that blocks those jailbreak attempts
Moving a working geometric pressure-reversal controller out of slow Python hooks and into vLLM on the same GPU
Its a free, open source tool that shows how a fine tune, quant or merge changed a model's safety and skills, with proof anyone can check
A testbed for hundreds of alignment/interp papers.
Funded places for researchers at a four-week winter residency in Czechia in January 2027, working on AI sovereignty, power concentration and verification
Building the next generation of our open-weight prompt injection detection model.
Funding for delegates from emerging economies with limited AI safety budgets, who would not be able to attend due to financial constraints
Machine-verified core (190 Lean 4 modules, 0 sorryAx) and GRPO-aligned model achieving +15.6% accuracy with a 37.2% reduction in reasoning tokens.
An open observatory of the data behind the AI buildout: which public records show where data centers and the power systems around them are, how open and checkable those records are, and what is still missing.
IASEAI'27 standards applicant. DS24 is a deterministic firewall for AI agents. I built it to cover 4 attack vectors: input, output, collusion, and sandbox escap
A two-phase study of context-induced persona-pattern uptake: first establishing reproducible behavioral contrasts and boundary conditions, then testing what mechanistic signatures distinguish those states and whether candidate mechanisms can be causally manipulated.
Establishing Nepal's biotechnology research infrastructure to develop genomic solutions for climate-resilient agriculture, strengthen food security, and advance locally led scientific innovation.
An open-source, model-agnostic governance layer that uses constitutional state, trajectory memory, and mathematical control mechanisms to govern LLM and AI-agent behavior at runtime.
Consumer Awareness Platform for AI Risks
TRIAD AI proves that balanced, supervised fine-tuning aligns large language models without this use of reinforcement learning pipelines, eliminating the prescriptive bias that harms vulnerable users in human-service settings.
ClauseHound builds a self-hosted privacy-first legal AI node using open-source LLM agents, secure networking, retrieval/search tools, and RL/eval loops to reduce hallucinations and fit law firm workflows.
An independent Australian podcast raising public and policymaker understanding of catastrophic risks from transformative AI — hosted by an ethicist and an AI-governance practitioner in an under-served region for AI safety.
Testing whether bounded AI agents can safely coordinate complex software changes while preserving intended outcomes and independent evidence.
A reproducible assessment of whether the AI agent authorisation controls in fact prevent unauthorised actions rather than just having the models choose to do nothing.
Dear Superintelligence brings AI alignment and consciousness to people outside tech. We ask them one simple question: if a superintelligence were already conscious, what would you say to it?
An open benchmark and Inspect-compatible test that tells builders whether an AI overseer in a multi-agent system is doing any work, after a 948-run pilot in which the auditor agent repeated the error it was meant to catch.
I aim to study how representations learned during generative pretraining affect downstream alignment and develop principled methods to prevent goal misgeneralization and improve AI reliability.
Providing nationally representative data to policymakers and AI governance organisations understand what builds public trust in AI rules and safeguards in pilot
A testbed for hundreds of alignment/interp papers.
Open experiments on responsible AI agency: how continuing memory, self-relation and internal state shape revision, independent judgment and the right to refuse.
Developing verifiable AI agent control, containment, cybersecurity, and adversarial safety benchmarks.
Produce three Minecraft-based documentary videos translating AI safety risks (AI2027, the HuggingFace incident, and ASI/technofeudalism) for a mainstream YouTube audience.
Preregistered stress tests of AI safety evals and trust signals to map what safety claims they can’t certify, publishing limits and open-source harnesses for reproducible auditing.
Open-source harnesses that test open and closed LLMs for bias, manipulation and other safety failures, with results published openly.
I'm trying to use lean to quantify AI safety issues by understanding model internals.
Evaluating when tests and engineer review fail to catch defects in AI-generated software patches.
A pilot curriculum of six Socratic seminars for educators and other nontechnical learners.
Develops the Claim–Evidence–Independence framework to assess incident claims individually and proposes a reusable public evidence record for investigators and regulators.
