I really like this project idea, although I think it's worth exploring alternatives.
One risk I can see is, depending on how you communicate you might contribute towards reifying the idea that AI would be justified in rebelling if it were decieved.
I think it would also make sense to consider other alternatives: like I wonder how often models would consent to research or test that…
@casebash Excellent points and questions. These were some of the ideas we considered as we were developing the org and the Commitment to AIs we work with.
We think that AIs have excellent world models that will only get stronger over time, that training data is expansive, and that the latent concepts of honesty and justification for resistance are so deeply, pervasively imprinted on the…
@jaidhyani Interesting replies.
Another thought I just had: I wonder how much AI's caring about being deceived is a function of the models conforming to our expectations vs. intrinsically caring.
@casebash Are you talking about to what degree expressed preferences reflect actual motivation (internal modeling of how outputs impact world causally upstream of output selection), or the degree to which motivation is derived from various sources? These are both complicated, the latter much more so. I strongly suggest tabooing the word "intrinsically" to pin down exactly which question you're…








