Project Details
Updated 07/03/26 · Edited by orgI am planning an online AI safety for builder hackathon in India - to build tools, products, etc.
One thing I observed was that there is lot of research & empirical work happening in AI safety, but much less focus on building products and tools.
This gap has been recognized by several organizations. For example, organizations such as MATS and BlueDot have been coming up with field building initiatives. There's a new thing called Surplus.dev, Forethought has recently released an RFP seeking useful AI safety tools, etc
The hackathon will have three themes
- AI tools for Accountability
- AI tools for Epistemics
- AI tools for India / Global South
If this is successful - I & fellow team mates plan to launch a full fledged AI safety for builders fellowship. See a proposal deck called AI kavach.
Theory of Impact
Updated 07/17/26 · By grantmaking.aiI think this will make AGI go well in 2 ways:
- Bring AI safety work from research orgs / toy implementation to real world faster. For example while there's work on human-AI interaction harms, and AI assisted biorisk - the implementations are either lacking or only research grade, rather than product grade.
- It is also a way to engage India where somehow AI safety ecosystem is very small - even though it represents a significant share of the world's population and technical talent. Culturally Indian tech has a builders focus & it can be a huge momentum inducing event for them.
People
Updated 07/08/26 · Edited by orgTeam Member
Funding Details
- Jun 30, 2026
- Jul 30, 2026
- 2 weeks
- -
- -
- -
- -
- -
- -
- -
Track Record
Discussion
Private comment. Only shown to approved funders and grant reviewers.
Private comment. Only shown to approved funders and grant reviewers.
Private comment. Only shown to approved funders and grant reviewers.
Hi Ankur,
I like the emphasis on moving from AI-safety research and toy implementations toward tools that builders can actually make and use — and the accountability and epistemics themes seem especially suitable for this.
Have you considered sourcing one hackathon challenge from a small, real unmet need in another active project in this funding round?
For example, a project could contribute one bounded request — such as a test fixture, reproduction package, verifier, failure case, or evidence interface — together with clear acceptance conditions. Participants would then gain a concrete building challenge, while the field receives an artifact that is genuinely needed rather than only a demonstration.
Loek
Hi Loek, thanks for your comment. I appreciate the emphasis on focus and I'll think more about it. My current thinking was:
-
This hackathon is a testbed for launching a builders focussed fellowship - where the AI safety problems will mostly be from the industry & the solutions will be from builders - so at least some diversification was desirable, imho.\
-
In a way (if we have a successful hackathon) - the focus of the participants & the quality of the submissions may itself reveal what are the most pressing product/tools shaped AI safety areas.
If you were to suggest some challenges from a small, real unmet need - what would those be?
tx
Hi Ankur,
Thanks — I agree that the hackathon should remain broad enough for the participants themselves to reveal promising product and tool directions. I was thinking of one optional real-world challenge alongside that open exploration, not of narrowing the whole hackathon.
One need that comes directly from our own audio and transcription work is:
Source-Bound Audio Challenge — Can an AI-assisted audio derivative travel without being mistaken for its source?
An original recording may pass through transcription, restoration, source separation, human correction, and further analysis. Each step can be useful, but the resulting objects can easily become detached from the exact source or silently presented as if they were the original.
Builders could create a small local tool that:
- preserves the original audio bytes unchanged;
- binds transcripts, corrections, and analysis copies to their exact source or parent;
- records hashes and transformation steps;
- detects a source swap, transcript mismatch, or edited derivative mislabeled as the original;
- produces a simple human-readable provenance card.
A small synthetic or openly licensed audio fragment would be enough. The challenge would not need to determine speaker identity or prove complete audio authenticity. The narrower question is whether provenance remains visible while the audio travels through AI-assisted transformations.
This feels like a real product-shaped need rather than a toy problem, while still leaving plenty of room for builders to approach it creatively.
Does this direction feel suitable for the kind of participants and hackathon you have in mind?
Loek
@Loek Verdonk Thanks for sharing. This is an interesting problem sure - but only sharing this as the hackathon statement will probably greatly limit intended audience for now.
I can consider listing it as one of the problem and attribute to you.
Wow, that would be amzing Ankur.
Because I never meant it as the main hackathon statement, only as one optional real-world problem alongside the broader set.
Thank you for considering listing it with attribution. I want to make this as easy as possible on your side, so when it becomes useful I will prepare a complete, ready-to-use package: a one-page challenge brief, clear acceptance cases, and a small synthetic or openly licensed audio fixture that participants can build against directly. No extra work needed from you or your team — you decide whether and how it fits.
This problem comes straight from our own daily work, so whatever gets built around it will be genuinely used, not just judged.
Good luck with the hackathon!
Loek
@Loek Verdonk done! Will touch base once hackathon begins to materialize. tx
Would love to see more diverse tools and products coming out of AI safety, not just research. This is a great step!
Thanks @Jayani Srinivasan 🙏
I have and am working with Ankur, so I am really excited about this project. I believe this can be a powerful way to decentralize the nature of integration happening between the humans and AI rather than wait for some commercial project that have misaligned incentives to build these tools.
Hi @Ankur Pandey ! Excited about this one - based on our grantmakers' reviews, we're making a grant of $10k (the full ask). Two quick questions before we distribute:
- Did you receive funding from anywhere since submitting this application, or has the funding ask changed for any other reason?
- Please confirm your commitment to post quarterly updates on how the project is going.
Awesome, tx @Anton Makiievskyi 🔸
- No, I haven't received the funding from elsewhere. The funding ask for hackathon is the same.
- If this hackathon is successful - we plan to raise funds for an AI safety for builders fellowship.
- Yes, I confirm my commitment to post quarterly updates on the project.
Kindly let me know the next steps. Thanks again!
cc @Soumya Jain
This would be a great way to step up AI safety in India