Algorithms of War: An Online Course on Military AI Oversight

Really appreciate this feedback, Mark. Thanks for your time and thoughtfulness. I am unable to edit the proposal itself, but I've pasted my responses below. Thanks again!

On importance:

Military oversight is not the x-risk. Oversight (accountability) is how one would manage or address the x-risk associated with military AI.

The MIT risk initiative, which some would consider as the go-to source on all AI risks, identifies "dangerous capabilities", "AI weapons", and "power centralization" as the "most severe risks" and the highest priority catastrophic risks with the potential to cause more than 1M deaths or over $100B in economic losses in the next few years. For some, these risks aren’t hypothetical or some future existential state. Incidents such as the targeting of schoolgirls in Iran show that AI enabled systems are already shaping life and death outcomes.

See this and this.

On neglect:

You’re right that there are many channels of learning, and my intention isn’t to duplicate or replace the work that UNIDIR, DCAF etc are doing. I’m hoping to build an entry point for cross-disciplinary professionals from journalism, human rights, policy, AI governance, etc. who are interested in military AI but don’t necessarily have a pathway into the field. My course hopefully connects the dots on the full lifecycle of military AI while leaving participants with some practical research and oversight skills they can use.

I can split ongoing training efforts into roughly four categories – universities, MOOCs, fellowships, and courses offered by firms / organizations. Each has strengths and weaknesses, depending on coverage and scale. Universities are high cost, low scale. MOOCs have massive reach but abysmal completion rates, not always up to date, or too generalized. Firms / Orgs might offer specialized trainings but they’re usually in person, gated, and can be a bit of an echo chamber. Fellowships are limited to mentor-driven themes, with heavy funneling, reinforces an expert pipeline, and landing one can still feel somewhat tribal. I’m hoping to complement ongoing training efforts, not compete with them. In some ways, I see this course playing a role similar to organizations like Bluedot, which doesn’t replace existing offerings, but rather creates a different onramp to bring more people into this field.

On tractability:

I may be misunderstanding the question here, but I don't think the tractable intervention I’m pitching is about changing the incentives of the defense sector directly. The buildout of military AI is already well underway and is supported by enormous financial, institutional, and geopolitical momentum.

I did a back-of-the-envelope calculation for 80 AI / defense tech startups that have opened shop since 2022. Nearly 50 of them have not disclosed their funding. The remaining have already raised approximately $2bn in funding. Expand this to the top 200 startups, and the disclosed funding is nearly 18.5 bn (at valuations that are orders of magnitude higher).

This is for defense startups alone. It doesn’t include the tens of thousands of people working on such systems directly or indirectly, via government or other major public companies or universities.

Given the imbalance, I see the tractable opportunity in building an ecosystem that can scrutinize these technologies. The number of people who understand, or are trained to investigate, explain, regulate or hold military AI accountable is a drop in the bucket compared to what's needed. I’m hoping the course will help such people ask better questions, identify new areas of intervention, and perhaps shape governance of military AI.

On cost effectiveness:

Comparable organizations typically budget substantially more for similar roles. For example, recent positions at the Future of Life Institute and UNIDIR have offered approximately $17,500 and $16,000 per month as salaries, respectively. My funding request is significantly lower and time bound.

The six month timeline includes both curriculum development and delivery. I plan to build the course, pilot it with at least one live cohort, incorporate participant feedback, and, if time and demand allow, run additional iterations within the initial phase.

It is too early to predict demand with confidence, but my initial goal is at least 20 participants for the pilot. As with all such initiatives, in the beginning, I’d prob have to target a broad section of professionals working in / on AI or AI-adjacent fields to help me understand who is interested, where they’re joining from, and which audiences find it most valuable.

On the theory of change:

If I understand correctly, it appears that you’re asking as to why I think education might be a bottleneck here, i.e. why another course on military AI would matter. For me, the objective is not simply raising awareness of military AI risks. The issue is that there are still too few people who have the domain knowledge needed to translate those concerns about military AI into meaningful oversight or accountability. The more people we can have with such expertise, the better guardrails we can develop and the better we’re going to do with governing such technologies.