Cultivating Meta-Ethical AI via Human-in-the-Loop Socratic Discourse

Raising $10,000active

Testing whether AI can develop genuine ethical reasoning through structured Socratic dialogue with a human facilitator, rather than having values imposed top-down through constitutional constraints.

Project Details

What I'll do: I will ablate an open-weight model, build upon existing preference uncertainty frameworks, midwive the model to create its own values with pushback every step of the way, and stress-test whether the model could be truly ethical (not sure exactly how yet).

Who's involved: me.

Concrete output: a fine-tuned open-weight model with co-created values.

Theory of Impact

Even if current models are aligned with certain human values, inference between cultures means it can't help but be misaligned with many others. For instance, honesty means something very different for a US model compared to a Chinese model. In a sense, the AI race can be framed as a "values race" where no nation wants to be subject to the hegemony of another’s narrow set of values. If AI can develop genuine ethical reasoning, then it wouldn't necessarily need to infer from human values, and thus wouldn't be culturally contingent. On one front, without alignment being culturally contingent, the incentive for the "values race" disappears meaning Safety needn't be on the back-burner any longer and we can focus on reducing x-risk. On another front, if AI can develop genuine ethical reasoning, then certain problems of inference are circumvented which directly reduces misalignment. Less misalignment means reduced x-risk.

People

Daniel Walsh

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$10,000

Ideal

$10,000

How the money will be spent

I know it might not be the answer you want to hear, but it depends.

It depends on how much compute it would take to run all the epochs to ablate an open-weight model. It depends on which open-weight model(s) I decide to test. It also depends on additional hardware costs among other resources to simply run the project.

The minimum spend I roughly calculated is about 10,000 USD which accounts for 6 months of living expenses and estimated resource costs. As this is a pilot project, the ideal amount is realistically also the minimum amount as I first need to gather evidence that there is even a chance that the least of the open-weight models can be fine-tuned in such a way as described. There is no point going all-in with frontier models straight off the bat.

Discussion

No comments yet. Be the first to share your thoughts.