Project Details
Updated 07/13/26 Ā· Edited by orgWhat youāll do
Sentient Futures is building the interface between AI safety & nonhuman welfare. We think developing this intersection is a neglected yet promising route to mitigating AI-related X-risk through such mechanisms as:
-
Improving methods for detecting & correcting species and substrate bias might help prevent AI systems from learning that moral consideration is contingent & instrumental.
-
Probing internal subjective states could enable humans to understand & respond to the experiences of potentially conscious or sentient AI systems.
-
Forecasting societal responses to the rise of so-called digital minds may better inform anticipatory governance frameworks.
We seek funding ($12-24k) for compute/API support to accelerate such research via our Project Incubator: a 10-week program (modeled after SPAR, FIG) matching motivated individuals with expert mentors to work on well-scoped projects on topics related to longtermism & future risks, artificial sentience, post-AGI transition, & macrostrategy/cause prioritization.
Track record
Our spring 2026 run brought together 34 mentors & 93 mentees across 52 projects. Notable outcomes span research exploring implications of artificial sentience for society at large, as well as technical work on aligning AI with nonhuman welfare:
-
Do emotional prompts affect the potential capabilities of LLMs?: Experiment testing whether AI systems operating under suffering-like adverse conditions show measurable performance degradation, making AI welfare an engineering problem ā not just philosophy.
-
What would convince you of AI consciousness?: Qualitative interviews with consciousness & AI safety experts mapping key cruxes with respect to the possibility/likelihood of AI consciousness.
-
Multi-Agent Alignment Game: Frontier AI models compete & cooperate under simulated US-China dynamics. Incentive design, rather than model choice, drives cooperative vs. competitive behavior.
-
AGI Loss of Control Pathways: Participatory backcasting to map social & institutional routes by which humanity loses control over AGI. Under review by EMNLP.
-
Travel Agent Compassion (TAC): Benchmark testing whether AI travel agents avoid booking animal exploitation (e.g. bullfights); merged into UK AISI Inspect Evals.
-
Hyperstition for Good: Writing competition curating fiction of safe, compassionate-to-nonhuman AIs as high-quality mid/pre-training data for labs ā betting that stories about good AI make good AI.
See published projects here (filter for AI Safety & Alignment).
Concrete output
Fall 2026 Incubator run with ~100 mentors & 180-200 mentees across 75-100 projects.
Whoās involved
-
Constance Li (Executive Director & Cofounder)
-
Jay Luong (Incubator Lead)
-
Brody McManus (Fellowship Coordinator)
-
Zoe Lu (Community Lead)
-
Martyna Wielopolska (Research Manager)
Mentors include researchers/staff from MATS, Anthropic (incoming fellow), RAND, Rethink Priorities, Forethought, Cambridge AI Safety Hub, & LASR.
Browse current mentors here.
Theory of Impact
Updated 07/13/26 Ā· By grantmaking.aiThe Project Incubator generates field growth, research, & talent that would not otherwise exist, closing the gap between nonhuman welfare & AI safety & ultimately mitigating X-risk.
-
Field growth: The field of AI safety could be too narrowly focused & may benefit from a greater diversity of approaches & perspectives. Our mentor network attracts a broad array of experts from animal welfare, philosophy, & the psychological sciences into AI safety work. These recruitment pipelines are not as well-covered by other similar online mentor-based project programs like SPAR.
-
Research: Mentored projects produce evals, benchmarks, & empirical work on key areas of importance to both nonhuman welfare & AI safety, like value extrapolation & internal state detection. Adoption by labs & governments leads to safer systems.
Two benchmarks from our Spring 2026 cohort were merged into UK AISI's Inspect Evals, & another project reached EMNLP submission.
People
Updated 07/13/26 Ā· Edited by orgTeam Member
Incubator Lead
Funding Details
- Jun 1, 2026
- -
- -
- -
- -
- -
- -
- -
- Partially funded
- -
Track Record
Our spring 2026 run brought together 34 mentors & 93 mentees across 52 projects. Notable outcomes span research exploring implications of artificial sentience for society at large, as well as technical work on aligning AI with nonhuman welfare:
-
Do emotional prompts affect the potential capabilities of LLMs?: Experiment testing whether AI systems operating under suffering-like adverse conditions show measurable performance degradation, making AI welfare an engineering problem ā not just philosophy.
-
What would convince you of AI consciousness?: Qualitative interviews with consciousness & AI safety experts mapping key cruxes with respect to the possibility/likelihood of AI consciousness.
-
: Frontier AI models compete & cooperate under simulated US-China dynamics. Incentive design, rather than model choice, drives cooperative vs. competitive behavior.
Discussion
this is a hugely important and neglected area, and more work needs to be done around this. This is a valuable field-building effort and past sentient futures' projects been incredibly well-organized and competently executed.
Many of CaMLs papers, benchmarks and the Hyperstition project were only able to happen due to connections made within the Sentient Futures community or their incubator. We fully support them as field builders.
Very important and neglected. Some tractable low hanging fruits. š
Private comment. Only shown to approved funders and grant reviewers.
Sentient Futures is doing incredible work in a space that I believe will only become more valuable as advanced AI systems grow in capabilities and deployments. The Incubator is a one-of-a-kind program offering mentors and mentees the opportunity to work on impactful projects that address issues many other large fellowships neglect.
SF supports good research!
SF Project Incubator is the only program that could look like "MATS for digital minds" right now. As a mentor, I'm very excited for this and want my mentees to have the compute they need to do quality science!
The theory of change is only tangentially x-risk, but all the grantmakers are excited about this application, as well as many others in comments, so I'm excited to approve the $20,000 grant!
I hope the Lightcone Commons funds the full request
@Constance Li, two quick questions:
- Did you receive funding from anywhere since submitting this application, or has the funding ask changed for any other reason?
- Please confirm your commitment to post quarterly updates on how the project is going.
Hey Anton! Appreciate you approving the grant despite our work not being obviously within the scope for x-risk.
To answer your questions:
- We received a 20k speculation grant from SFF which is for our entire org and will not overlap with this grant and we have referred 10 projects to sign up for Silico by Goodfire for compute and if that is successful, they could each receive 250 research credits worth 2.5k each. The latter is not confirmed.
- Yes
Thank you Anton! This is Jay, Incubator Lead. How would you like the quarterly updates? Delivered on Grantmaking.ai ā or will you send us a form later?
Iād like them posted in the comments here
@Anton Makiievskyi šø Happy to do that. Let me know if there's anything specific re: content/form that you'd like. Also, we are currently running the Incubator twice a year. Counting from August, the updates would be:
- November 1: This would be just as the upcoming (2nd) round is wrapping up.
- February 1: We expect to be gearing up for the following (3rd) round about this time.
- May 1: We expect the 3rd round to be wrapping up around this time.
- August 1: We expect to be gearing up for the 4th round about this time.
@Constance Li I don't assume your responsibility to updates for a whole year given that our grant is $10k and would be spent a lot quicker, but I'd definitely be excited to see as many updates as you feel like sharing
I'm interested in highlights of what was done (and lowlights too), so that I know if the grant was a good bet, and to feel connected to the impact you'll produce
@Anton Makiievskyi šø For sure. It's $20k right? That's what's stated in your first comment & in the fundraising progress bar.
We can certainly arrange for the updates. I personally think the best timing would be:
- Update 1 ā December/January: Gives time after the fall cohort ends (November 9) for us to do things like collect all the relevant exit survey data in one place, process submissions & update our website listings, & edit & upload presentation recordings.
- Update 2 ā March/April: Updates on mentor recruitment for the spring 2027 run, by which time we hope to attract even more strong technical mentors having been able to offer compute/API support in the fall round.
@Constance Li Yes, sorry, it's $20k
Yes, these updates would be awesome!
I think this is a neglected research and cause area inside the broader field, and one where unique harms are likely to emerge unless we take exploratory and mitigation measures. Increasing the pipeline for diverse perspectives is especially worthwhile.