Winning the Long Game of Human Flourishing after AI

Boston, MA, USA

A deep dive into what victory actually means for AI safety, a set of written materials, slides, org pitches, etc. that disseminate the ethos widely into the community, and how to shape the field so it ends in human flourishing.

Project Details

I want to spend a few months reading and thinking and writing about the problem of a victory condition for AI safety (see https://firstscattering.com/p/you-need-a-theory-of-victory). In particular I need to think very hard about the dichotomy between making AGI go well or not making AGI happen at all. This should result in a blog post, a slide deck, and a talk series I can easily give to multiple groups at multiple venues.

It could become something larger and I want to think through that as well (course curriculum, book, an organization, etc.) while talking to as many stakeholders and AI Safety personnel as I can during this time. In short, I need to shape and write my thinking up around the problem of how to make AI Safety go well in our society and economy, and produce systems thinking and a thought environment that will help maximize human flourishing.

I’ve given a beta version of a talk at CBAI based on the above blog post as well as a few other readings about aligning incentives with organization structure (e.g. Manifund’s blog and Eric Ries’s Incorruptible), and it went well. More importantly, the strongest takeaways people got from the group talk was the impetus for this ask. I’ve read a lot on this recently, whilst also conducting technical AI safety research with David Bau, Malihe Alikhani, and Jayson Lynch/FutureTech. Most recently I was a CBAI AI Safety Fellow (Summer 2026). I’ve coached, consulted, and presented frequently in my capacity as an industry software engineer consultant, and I have a nascent blog (https://averyyen.dev), but reading, public speaking, and writing (human-generated) are skills I want to level up significantly while not worrying about producing Technical AI Research (e.g. shepherding AI agents to code up my experiments while I wait and read other things).

I have substantial community organization and cultural-flourishing activities in my past; in particular I’ve been an active creator and scene-builder for local Boston area competitive grassroots gaming (Smash Bros.) as well as an avid community orchestra musician and donor (Boston Youth Symphony and Mercury Orchestra).

Theory of Impact

AI Safety needs an understanding of the victory conditions that exist. Quoting directly from Jason Hausenloy, policy lead at the Center for AI Safety and formerly of AI 2027,

To achieve a good outcome, people normally advocate for “theories of change”, which tend to take the form: “here’s my best guess for a causal chain for my action X causes Y.” X is a well-defined action (start an organization, run this media campaign etc.), and Y is a broadly-accepted “good thing” (“increases awareness of AI risks” “increases the probability of rational decision-making”)
This is, however, importantly distinct from a “Theory of Victory”, which I would understand as the full (causal) story of the steps that need to go right for us to “win” (where, borrowing the broadly-agreeable understanding of “win” from earlier -- we don’t die + the benefits of AI are broadly distributed.)

If we can better understand what winning looks like for AI Safety, we will better be able to direct resources into activities that actually lead to the possible win conditions. I think there are quite possibly multiple equilibria here, but understanding the conditions that allow us to reach them is necessary to inform the AI Safety activity (research, grant-making, policy, writing) that actually occurs in the field.

People

Avery Yen

Team Member

Funding Details

Start Date: -
End Date: -
Expected Duration: 3-6 months
Funding Raised to Date: -
Annual Budget: -
Monthly Burn Rate: -
Current Runway: -
Funding Goal: -
Funding Stage: seeking first grant
Fiscal Sponsor: -

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$20,000

Ideal

$40,000

Theory of impact

How the money will be spent

Living expenses and stipend, e.g. food, housing, clothing, healthcare, premiums etc. to help support a family of four. I don’t need compute or in-kind or other things, I have sources for that. But I won’t be working on anything else during this time. If I work on things I can’t immediately monetize like a curriculum, book, or founding an org, that will also require more capital (later).

Discussion

No comments yet. Be the first to share your thoughts.