| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
Showing 1-50 of 103·Sorted by endorsements
| Name | Type | Tags | Endorsed | Team | Raising $ |
|---|
| Project | Community |  +4 |
| - |
| Project | Training |  +2 | - |
| Project | Research | +3 | - |
| Project | Media | +2 | - |
| Project | Research |  +2 | - |
| Project | Training | +3 | - |
| Project | Tooling | +3 | - |
| Project | Training | | - |
| Project | Interp |  | - |
| Project | Research | | - |
| Project | Governance | +2 | - |
| Project | Individual | | - |
| Project | Media | | - |
| Project | Research | - |
| Project | Research | +1 | - |
| Project | Research | +1 | - |
| Project | Community | - | - |
| Project | Governance | - | - |
| Project | Community | - | - |
| Project | Research | - | - |
| Project | Governance | - | - |
| Project | Research | - | - |
| Project | Individual | - | - |
| Project | Media | - | - |
| Project | Community | - | - |
| Project | Research | - | - |
| Project | Newsletter | - | - |
| Project | Tooling | - | - |
| Project | Research | - | - |
| Project | Platform | - | - |
| Project | Interp | - | - |
| Project | Research | - | - |
| Project | Research | - | - |
| Project | Research | - | - |
| Project | Tooling | - | - |
| Project | Governance | - | - |
| Project | Tooling | - | - |
| Project | Tooling | - | - |
| Project | Media | - | - |
| Project | Field-Building | - | - |
| Project | Research | - | - |
| Project | Tooling | - | - |
| Project | Tooling | - | - |
| Project | Research | - | - |
| Project | AI Welfare | - | - |
| Project | Control | - | - |
| Project | Interp | - | - |
| Project | AI Welfare | - | - |
| Project | Control | - | - |
| Project | Deception | - | - |
The Sydney AI Safety Space is a co-working hub in Sydney, Australia, offering free office space for people working in the field of AI safety.
+4Creating a Stronger Pipeline into Technical AI Safety Research
+2How do you map your old concepts to new concepts when your model of the world gets updated? How would you translate concepts you care about to new ontology? I plan to adress this via extending "Condensation: Theory of Concepts" - Eisenstat, (2025)
Doom Debates is a modern "infotainment" show that functions as a mainstream-accessible forum for top thinkers to have a high-quality conversation & debate about AI extinction risk.
+2Detailed models of how AI could allow a small set of actors to gain a decisive strategic advantage over the rest of the world: concrete pathways, required capabilities, quantified likelihoods, and the defenses that bind them.
+2A graduate student in mathematical ML and a postdoc in technology law, trained together at the Fields Institute and uOttawa's CLTS, on research aimed squarely at reducing existential risk from AI.
Building an independent runtime governance layer that enables organizations to deploy autonomous AI with authorization, human oversight, auditability and cross-model governance.
A scalable fellowship training researchers to develop interventions for achieving high-value long-term futures
LLM agents collaborate to discover and formally verify theorems about the internal computations of transformers, beginning with a simple pilot question: how many attention heads are needed to represent a Boolean function?
How do the options agents present to human researchers alter which research paths are explored, and do steered researchers notice or feel less in control?
A dedicated role at Pause IA to brief French policymakers on existential and catastrophic risks from advanced AI, and to train our volunteer network to do the same with their own representatives.
Auditing the social media presence of the top 25 AI safety organizations across major platforms to quantify the public communication gap and publishing a full gap analysis, then giving recommendations to the orgs for improvement.
Alien Cub is a production company focused on stories that inform a wide audience about risks from transformative AI.
AIs keep going wild on the web, what are they up to?
Testing whether the safety circuits in language models actually work in Igbo (Nigerian Language) and building evaluations that distinguish real refusals from accidental ones.
consolidate, extend, and manually validate an AI-alignment field map, close gaps, perform backtests, post Metaculus questions, and write LessWrong posts about the map.
Requesting $750 to fund travel, lodging, and expenses for an AI Safety Nepal representative to attend Swiss AI Safety Days 2026 in Zurich to network and learn field-building practices.
AI Safety Influencer Comms Agency
AISHK will run an in‑person weekend sprint hub for elite finance talent and host the winning team during the Apart Fellowship.
An LLM benchmark for emergency response communications in existential catastrophes.
Educating the world about AI Risk, bringing marginalized voices into the global AI conversation to create a future where AI risk is significantly reduced.
Combining, iterating and refining best-practice epistemic democratic AI Governance with a documentary and international strategy to scale a UK citizen assembly
Write 30 blog posts over 2 months on AI alignment and related technical topics (e.g., LLM agency, preferences, chaos/nanotech), using funding to cover living expenses while writing.
Produce a 4-episode technical AI alignment podcast interviewing researchers about their agendas, funding equipment, travel/space, and the host’s time.
Funds Parveen Sheikh to travel from India to present an AI safety governance poster at Swiss AI Safety Days and attend EAGxBerlin for networking and collaborations.
Research and write a history-of-science analysis of persistent disagreement (e.g., caloric vs kinetic heat) to draw lessons for current AI alignment disputes and progress.
Expand "Humanity Tomorrow" website with a multilingual AI Risks & Mitigations Map connecting risks, causal pathways, mitigation strategies, remaining gaps, and project ideas, while maintaining the existing AI Risks FAQ.
Open-source hidden state control telemetry, reading and steering the black box, to prevent latent AI collapse before output, protecting Man and Beast
Measuring Human–AI Inquiry Over Time
red teaming and educational ai agent platform
A two-phase study of context-induced persona-pattern uptake: first establishing reproducible behavioral contrasts and boundary conditions, then testing what internal representations distinguish those states and whether they can be causally manipulated.
Opensource infrastructure to test whether increasingly capable AI systems remain controllable when they actively attempt to bypass the safety mechanisms designed to constrain them
Evaluating a tool to deeply understand misaligned agent behavior post-hoc.
We will test whether AI assistance that performs developing cognitive operations weakens adolescents’ later independent judgment, and develop evidence-based safeguards that keep human reasoning meaningfully in the loop.
An open-source, model-agnostic governance layer that uses constitutional state, trajectory memory, and mathematical control mechanisms to govern LLM and AI-agent behavior at runtime.
Consumer Awareness Platform for AI Risks
A public county-level tracker of AI compute buildout against real power infrastructure, showing where infrastructure can realistically expand and where energy constraints become the limiting factor.
ClauseHound builds a self-hosted privacy-first legal AI node using open-source LLM agents, secure networking, retrieval/search tools, and RL/eval loops to reduce hallucinations and fit law firm workflows.
An independent Australian podcast raising public and policymaker understanding of catastrophic risks from transformative AI — hosted by an ethicist and an AI-governance practitioner in an under-served region for AI safety.
APEC, Bangkok 2026
I borrowed from friends for two years to transition into AI governance research. It worked. Now I need the debts cleared so I can work in peace.
AI-kernel-kill-switch provides a last-resort shutdown mechanism for self-hosted LLMs via AES-256-GCM-authenticated payloads embedded in ordinary prompts.
Builds a blind-audit harness and bilingual Vietnamese local-harms benchmark to evaluate multilingual refusal gaps in frontier AI models without exposing private weights, data, or code.
Tests a weights-only projection for removing hidden AI loyalties, finding it rarely cures backdoors while often erasing evidence, including in third-party models.
Tests activation-steering methods in Gemma 2 9B to improve language models’ self-reporting of internal concepts, finding gradient-based steering boosts detection but not identification.
Develops a behavioral comparison protocol using independently generated solutions and multi-round interrogation to detect injected backdoors that evade weak-model code monitoring.
Uses mechanistic interpretability to study how LLMs process harmful requests during tool use, and proposes a safety monitor based on a new activation direction.
Proposes testing whether instrumental versus relational interaction styles influence experimental signals used to study possible digital minds.
ActionLens is a pre-execution control protocol that helps AI agents safely perform shell and file actions by combining environment probes, simulations, diffs, sensitivity scans, and history.
Develops a black-box test for detecting hidden, trigger-activated model behaviors by measuring ordinary-prompt dormancy and response dynamic range across prompt axes.
