IMA Existential Safety Focus: Math-ML and Tech Law

Raising $35,000active

A graduate student in mathematical ML and a postdoc in technology law, trained together at the Fields Institute and uOttawa's CLTS, on research aimed squarely at reducing existential risk from AI.

Project Details

Inter-Math-AI (IMA) is an interdisciplinary training program that I run across 4 Canadian universities, together with team member Florian Martin-Bariteau, who also directs uOttawa's Centre for Law, Technology and Society (CLTS), and program coordinator Vincent Létourneau. IMA, funded for 2022–2028 by NSERC-CREATE, provides enriched training in AI safety, ethics and social impact to selected graduate and postdoctoral students working at the intersection of mathematics and machine learning. Students are admitted for 2 years; we have just admitted our final CREATE-funded cohort, who will complete their training in 2028.

Building on our proven track record (confirmed by rigorous NSERC-CREATE auditing over the last 4 years), we now want to pivot IMA to focus on mitigating existential risk from AI. Specifically, we plan to recruit graduate and postdoctoral trainees as we've done each year, primarily in mathematical ML, but now also, in smaller numbers, in technology law. We will look for thesis/research topics in AI technical safety or AI policy/governance, specifically aimed at reducing existential risk.

IMA is a "piggy-back" program: only students already enrolled in relevant thesis or postdoctoral work at an affiliated university may apply, and once admitted we bring them together for significant enriched activities and opportunities (an overview is on our webpage). This lets us recruit the new existential-risk cohort as soon as funding is in place, for admissions starting in 2026–27, rather than waiting on a multi-year cycle. We will subsequently pursue further funding to extend to larger cohorts beyond this transition.

We are thus asking in this round for funds to support two trainees focused on x-risk: one graduate student in mathematical ML and one postdoc in tech law. They would embed with current IMA trainees and with two active research communities: the Fields Institute's Centre for Mathematical AI, whose steering committee I sit on, and which has just admitted a postdoc cohort focused on x-risk (funded by PrincInt); and, for the governance side, the CLTS. IMA support does not replace supervisors' support; it buys trainees time for our activities (rather than, say, TA duties) and funds their travel to IMA events. The concrete output is two trained researchers, and a first increment establishing a graduate-level, governance-linked analog of the Fields purely mathematical postdoc cohort. Our aim is to build a crucible for forging mathematical ML x-risk researchers and leveraging their work to advance existential safety.

Theory of Impact

Our proposed new focus of IMA takes a tandem approach to mitigating x-risk—namely, via progress in technical AI knowledge/algorithms and progress in AI governance/policy. The need for this tandem was prominently outlined by Bengio, Hinton et al. in their landmark 2024 Science paper, "Managing extreme AI risks amid rapid progress". As that paper stresses, institutions charged with oversight need strong technical expertise—and, in its words, "…far greater funding and talent." Our proposal directly builds this talent, training dialogue across the technical-expertise and legal-policy gap.

IMA’s mission, since its founding in 2022, has already been close in spirit, albeit more generally for AI safety and social impact. Our proposed x-risk focus for IMA leverages this foundation and aims it squarely at mitigating x-risk. Our proposed math-ML track supports trainees on the design front, embedded at the Fields Centre for Mathematical AI alongside its x-risk postdoc cohort; our tech-law track supports trainees on the policy and governance front at CLTS, where live questions run from regulatory oversight to coordinated restriction, closely informed by technical results.

People

Maia Fraser

Team Member

Grants Received

Add grant

NSERC-CREATE (awarded to Inter-Math-AI)

from NSERC CREATEwww.uottawa.ca

$1,171,178

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$35,000

Ideal

$100,000

How the money will be spent

IMA support to trainees has been and will continue to be paid in CAD so, below, we first describe the targeted figures in CAD, then convert to USD to build our request. Conversions done here use CAD 1 = USD 0.71 (July 2026). Actual disbursements will be the CAD equivalents of the USD amounts awarded, at the exchange rate at the time of transfer.

Basic amounts: The current standard IMA support for a PhD in Math/ML is CAD 20,000, to which we add CAD 2,000 for travel and conferences. The IMA (partial) support for a Postdoc is between CAD 20,000 and CAD 30,000 depending on the portion of the trainee's time devoted to IMA, and in either case we add CAD 2,000 for travel and conferences. The postdoc’s base salary will come from CLTS and teaching, while the IMA portion of their support is administered by the Fields Institute, which will host extensive interfacing with the PhD trainee(s). The basic amount for the first trainee is thus CAD 22,000 and for the second trainee CAD 22,000 to CAD 32,000.

Fields event organization and handling: Running the IMA-x-risk activities/trainings and payments at the Fields Institute requires an additional 10% overhead (by the Fields Institute, which has agreed to manage the funds similar to its current arrangement with PrincInt).

Minimal total request of USD 35,000: The low-end basic amounts listed above, CAD 22,000 + CAD 22,000, add to CAD 44,000, plus 10%. This is CAD 48,400 which--at the current exchange rate—is approximately USD 34,360, and which we round up to USD 35,000, our minimal request.

Preferred total amount of USD 42,500: This would use the values CAD 22,000 + CAD 32,000 which add to CAD 54,000, plus 10%. This is CAD 59,400 which—at the current exchange rate—is approximately USD 42,170, and which we round up to USD 42,500.

Ideal amount of USD 100,000: This scales the same two-track design to a fuller first cohort: not only the tech law postdoc, but also 3 PhD students paid at a higher rate of CAD 30,000/student, to be able to recruit more competitive students, especially in Computer Science (this is higher than the standard IMA amounts of CAD 20,000 which are typical for Math students but only partially cover competitive ML students). The total is thus CAD 120,000 plus CAD 8,000 for travel and conferences, i.e. CAD 128,000, plus 10% overhead, giving CAD 140,800. This corresponds to USD 99,958 at current exchange rate, and we round to USD 100,000.

Discussion

No comments yet. Be the first to share your thoughts.