grantmaking.ai Launch Round
Live and Actionable Al safety Map
Build a live, LLM-assisted map indexing AI safety papers to help researchers and grantmakers explore themes, track field trajectory, and identify actionable research opportunities.
This project aims to build a live map of current AI safety research. It is aimed at helping researchers and grantmakers understand the field's trajectory and identify actionable research opportunities while making sure that safety research has the greatest possible impact in reducing existential risk. At present, this is a one-person effort. I have already built a basic version of the map, along with vision documents on how to develop it further. In its current form, the tool aims to index all AI safety papers using llms in one place and lets researchers slice the corpus by theme and at whatever level of granularity and perspective of their choice. I plan to improve on this version iteratively, with the goal of building the best available tool for interpreting the field as a whole.
The quality of the people working on AI safety, the resources devoted to the problem, and the quality of the output they produce all have direct consequences for the trajectory of AI safety. So we need to make sure the community as a whole is working on the most leveraged problems, and that its outputs truly advance the frontier toward AI systems aligned with humanity.
Frontier labs can advance capabilities as independent, heavily resourced organizations. AI safety can't , because It's a collaborative, decentralized effort, and the organizations working on it are nowhere near the scale of the frontier labs. An effort that small and that distributed depends even more on shared situational awareness to stay coordinated.
In a decentralized field with no one setting priorities, each actor needs a signal for what’s worth working on, and the cheapest signal is what everyone else is already doing. Attention becomes the proxy for importance, talent and money follow it, the resulting pile-up of output reads as proof of value, and the loop feeds itself. Decentralization doesn’t prevent this dynamic, it hides it. No one chose the skew, so there is no decision to contest, and because everyone is sincerely mission-driven, the bias wears the camouflage of good faith.
The obvious fixes fail. Issuing better instructions doesn’t work, a central recommender that tells the field where to go doesn’t stand outside the loop, it becomes its fastest engine. And the field’s existing maps, good as they are, are descriptive and mostly static, they catalogue what exists but can’t tell you what’s blocking each direction right now, or where to go next.
The proposal is a live reflection of the field’s current state ,its funding, talent, and research output folded back into a form the field can read, that prescribes nothing. It makes neglected work findable, keeps the newest research discoverable instead of buried under citation lag, and gives the field a common, contestable map to argue over, so the correction comes from many independent readers rather than a few select sources.
Minimum version — Funds will be used for API credits for scraping, Claude credits, and supporting myself to work on this full-time.
Maximum version — Five FTEs actively working to build an entirely new ecosystem of tools and venues that connect researchers and grantmakers around the highest-leverage problems in AI safety. These tools include a research paper analyzer, which helps researchers quickly understand new work and connect it to the existing corpus instead of endlessly prompting Claude, and a research duplicator. The venue mentioned earlier will provide templated, composable, and forkable research infrastructure, allowing researchers to focus on research rather than engineering. We will also put systems in place to communicate the value of research outputs to the broader community and to gather fast feedback ,anonymous or otherwise , so that the bar for quality stays high.
Hi Manu,
Your description of the attention loop in decentralized fields really stood out to me — especially the point that a central recommender may not escape the loop, but become its fastest engine.
In the live, LLM-assisted map, what mechanism will preserve disagreement, missing evidence, uncertain classifications, and genuinely competing interpretations?
I am curious how the map can remain a contestable reflection of the field without its categories, summaries, or visibility choices quietly becoming a de facto ranking of what matters most.
Loek
Hi Loek,
Thanks for the comment. I'm glad you found the idea interesting.
The concerns you raise are very relevant. Preserving competing perspectives is at the core of this project's vision. What we plan to offer is a substrate that no one contests (raw data such as citations and papers), along with tools or lenses to parse that data. Using the tools, anyone can slice the data to create a new map and publish it to the community.
If you're not happy with a map that's been published, you can comment and request a change, or you can fork the part you disagree with, or even the entire map. That fork will be visible to the public, so a neutral observer can see the reasoning behind the competing perspectives.
The vision is to make this process of sensemaking and forking as easy as possible by providing curators with the necessary tools.
Thanks, Manu.
Your reply opened something up for us. As we followed the different threads, we began to see the depth of what you are building and how much value may already be present here that is not yet publicly visible.
After your reply, I read the full three-part series and looked through both the current field-map work and the paper-explorer experiment. There is much more here than I initially saw from the grant page.
In the essays, you describe something closer to version control for a field than simply an AI-generated research map: checks bound to sources rather than checks of whether something belongs, gates and schemas that can themselves be forked, disagreement that retains its history, and a toolmaker who should eventually stop holding the shared map.
Looking at the current work, I see several possible directions living beside each other.
One is the movement from a reference map toward a versioned commons. The current field map is still a static, read-only reference instance, while the essay describes infrastructure through which people can alter, contest and fork the map itself. I could not yet see a public fork-and-diff mechanism or an explicit license, so I am not sure whether you imagine the reference map and the commons becoming two separate layers, or whether the current map itself gradually becomes that shared repository.
A second direction sits at the corpus boundary. Your own README already acknowledges this clearly: selection bias is owned rather than neutral, and the non-indexed gap is only partly closed. The public metadata records 6,652 arXiv records and 1,249 lab records being pulled, while the published layer totals sum to 4,673 after filtering and deduplication.
I cannot tell from the public artefacts how much of that difference represents records rejected as outside the boundary, duplicates being merged, uncertain cases or other transformations. That uncertainty itself feels relevant to the project: the map currently shows what survived the boundary much more clearly than it shows the effects of the boundary.
I am curious whether excluded, merged, uncertain and not-yet-indexed material eventually becomes visible within the map, or whether you see that information as belonging to a separate audit layer around it.
I also see an interesting difference between the careful governance direction in the essays and the more autonomous direction explored in paper-explorer. One emphasises contestability, source fidelity and forkable gates; the other deliberately allows the model to decompose, extend and merge the concept map without a review queue or approval step.
These may simply be separate experiments. I am curious whether you see them eventually converging, what each direction might contribute to the other, or whether keeping them separate is itself important.
The funding range seems to contain another genuine branch. The three-month version is a focused project led by one builder. The larger version becomes an organisation, a venue and an ecosystem that also helps communicate the value of research outputs.
That may create valuable infrastructure, while also moving toward the boundary you describe yourself: the point where the maker of the mirror may gradually become one of the actors directing attention through it.
I do not know which of these readings is closest to your intent, and I may be connecting experiments that you see as separate. I am not raising them as objections with predetermined answers. They simply seem like real and meaningful sides of the project that are currently open.
I am genuinely curious which of these directions you see opening up, which you see as separate, and which still feel unresolved from inside the work.
@Loek Verdonk Are you a bot?
@Manu Xaviour Thaisseril Shaju
hhahaha no :D... but my answers are writing by ai as my english is not that good...
so sorry for that!
@Loek Verdonk Ok Loek, I don't mind your English. It's very hard to understand AI, If you have any questions, feel free to ask in your own words.
@Manu Xaviour Thaisseril Shaju
Ok Manu, thanks for that :D!
What I see ...
I see a dance between what we are both exploring and building.
You are working on how people can see, question and fork the field. We are building a very small “leaf” that carries one source, one proposal, the disagreement or result, and its history without silently changing it.
My simple question is: would you be open to seeing a small example when it is ready, and telling me whether it could be useful for your fork-and-diff idea?
Just trying to find out if its a dance you are open for?
Thanks anyway!
loek
@Loek Verdonk Thanks I will look into it!