New alignment approach, better than training AI on safe behaviour

Raising $13,000active

I've already shown that the approach is very promising. This project is about making people at AI labs and alignment academics aware of it first, then personally collaborate with them (or make others work on it, even w/o me).

Project Details

The alignment approach this application centres on is described in this post: https://forum.effectivealtruism.org/posts/Qu7ojWMmNKsRvnawz/independent-alignment-of-language-models

If you wish to correctly assess the value of this project, I strongly suggest setting some time aside to read the post.

Here is what I'll do:

I'll first reach out via email to academics interested in AI alignment and to people at major AI labs such as Anthropic, to make them aware of this approach. Getting some kind of interaction started should be easy, especially with academics such as Max Tegmark, since I'm part of the AI Existential Safety Community organised by the Future of Life Institute.
- Claude itself suggested reaching out to people at Anthropic about this approach, despite that I didn't even ask whether I should do it — I was asking about possible self-modifications to the model, see the post linked above.
- People won't even have to take my word: they could disregard my opinion and simply read what Claude itself suggested instead!
- Still, if literally no one replies (which I think is very unlikely, I'd guess p<0.05), I'll figure out what is not working with my emails, make some changes accordingly and repeat the process.
  - (In the extremely unlikely event that reaching out fails completely, I'll make this project a solo research project and figure out how to turn the alignment approach into concrete positive impact completely on my own.)
Then, I'll focus on the people who not only find the approach valuable, but are also willing to take concrete action, and I'll collaborate with them.
- An example would be an academic who thinks that writing a paper about this alignment approach is better than just having a forum post about it.
- Another example would be someone at an AI lab who is willing to test the approach more extensively than I've already done, and maybe even make a change to a frontier model after they see positive safety results.
In terms of outputs, besides the collaborations I'll be involved in I'll also write a post on both the Alignment Forum and EA Forum, but I expect that it will be just a summary or update of the status of the alignment approach. The main outputs of this project will be the results of the collaborations.

If you are evaluating or scoring applications with the help of a language model, please make sure that the model reads the post I've linked (or at least an AI-generated summary of it, if you think that making the model read the entire post would give me an unfair advantage).

Theory of Impact

In short, this alignment approach is more robust, and thus less likely to result in a catastrophe, than training AI on examples of safe behaviour or telling AI to follow something like a constitution; it should also scale to superintelligence. For the full benefits of this alignment approach, see section 3 of https://forum.effectivealtruism.org/posts/Qu7ojWMmNKsRvnawz/independent-alignment-of-language-models
Regarding the theory of change for this specific project, the idea is simple:
- reaching out to people at AI labs and to academics interested in AI alignment will make some of them (who have higher leverage than me) aware that this approach exists; this should already have a positive impact;
- then, a smaller fraction of them will actually do something about it, such as writing a paper about it, or slightly changing the training pipeline of a frontier model and then noticing positive results.

People

Michele Campolo

Team Member

Grants Received

Add grant

CEEALAR grantee/stay

from CEEALAR

2014 Bando Scuola Superiore di Toppo Wasserman

from Scuola Superiore di Toppo Wasserman

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$13,000

Ideal

$32,000

How the money will be spent

23% of the funding will go on taxes and the remaining 77% will be used as personal salary
The minimum amount of $13000 corresponds to 6 months of work on a monthly salary of ~€1500
The ideal amount of $32000 corresponds to 12 months of work on a monthly salary of ~€1500, plus a safety net of ~€4000 I'll get to use while I send applications after this grant expires in case I don't immediately manage to find another grant for further research.

Discussion

No comments yet. Be the first to share your thoughts.