Alignment Crux Map

A public map of the objects of AI safety research that lets funders fund and researchers work on the problems they intend to and not just something that shares that name.

Endorsements made here support aintelope gUG (haftungsbeschränkt).

A public map of the objects of AI safety research that lets funders fund and researchers work on the problems they intend to and not just something that shares that name.

Endorsements made here support aintelope gUG (haftungsbeschränkt).

Project Details

Maps of AI safety orgs tell you "who", maps of AI safety projects tell you "how". This project builds a map of the underlying problems of the AI safety field tells you "what" problem funding and research work actually make progress on.

Corrigibility, shutdown, deception, evals, tiling, and inner alignment are all terms that name more than one underlying problem. Corrigibility can mean "humans can still correct it" or it can mean "the off switch works". Funding or working on one when you meant the other wastes resources.

This project disambiguates the terms and provides clear explanations and criteria for evaluation ad a simulated AI lab where each underlying problem is inspectable.

Funders can use the material to check if a proposal is addressing their problem. Researchers can use it to check which gaps their research addresses, and people entering the AI safety field can see where their skills best fit - or develop into neglected areas.

Gunnar Zarncke is extending his existing AI safety field mapping into decision-making material, packaging existing simulations into illustration problems, and working with external AI safety orgs and researchers on testing and validating the material..

Theory of Impact

AI growing in capability can go wrong in multiple different ways all of which requires a specific solution. Not only preventing gaming of the eval score, but also that the defection isn't happening outside the tested scope, that rules aren't lost with the next release, that humans can stop or correct the system, and more.

If a funder funds or a researcher works on "corrigibility" but this more often means "the off switch works" and rarely "humans can still correct it" the portfolio may look fine but important research is missed. Funding and researchers follow common names.

This project shows how to identify the underfunded and understudied problems of AI safety. It is verifying the correctness and usefulness of the materials with researchers working on the underlying problems (Gunnar Zarncke has all the needed contacts).

People

Gunnar Zarncke

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$15,500

Ideal

$50,000

How the money will be spent

The money is spent on (ideal ask):

Researcher time (developing the simulated AI lab, orga): $37000

LLM assistants (for research, coding, writing; no training): $6000

Compensation for ext. researcher's evaluations: $7000

No travel. No GPU. No overhead.

Breakdown by steps:

development of the simulated AI lab for corrigibility, shutdown, deception, and evals plus the associated materials
independent evaluation by Redwood (covers deception and evals; this is the minimum package)
MATS use test
BlueDot use test
independent evaluation by CHAI or Christiano (covers shutdown & corrigibility)
development of the tiling disambiguation in the simulation
development of the Inner alignment disambiguation in the simulation
addressing feedback on republish, no retest
independent evaluation by e.g. John Wentworth on selection theorems as inner alignment
independent evaluation by e.g. Scott Garrabrant on shutdown / corrigibility / tiling
independent evaluation by e.g. Vanessa Kosoy
continuing to maintain for one year (~one day a month, incorporating new research and feedback)

Discussion

No comments yet. Be the first to share your thoughts.