SRIE

Raising $50,000active

SRIE, a mentored-research programme for Cambridge Mathematics Undergraduate students to explore research problems in industry, including AI Safety.

Project Details

Our Goal

SRIE is a mentored-research programme for mathematics undergraduate students at Cambridge to explore important research in industry, including AI Safety. Run by former Cambridge mathematicians, with two Cambridge professors interested in giving guest talks, we offer the best students in AI safety the serious mathematical work they rarely find elsewhere. The mathematics we introduce in the programme can be a huge deciding factor when it comes to students weighing up which career to pursue. A counterfactual is that such mathematicians may otherwise go into quantitative finance, as it is one of the few fields that, to their knowledge, stretches them mathematically as much as they would like.

We additionally also contribute to university fieldbuilding, an area that apparently is overlooked.

Given enough resources to incubate such students, I believe that we can take on highly ambitious AI Safety research.

Our Advisor

We are advised by Julia Bossmann, the Director of Research and Strategy at Sentient Futures. Sentient Futures is a field-building organisation focused on animal welfare, amongst other related priorities. Their funders include Coefficient Giving, and The Navigation Fund. Julia leads their project incubator which pairs fellows with mentors to work on projects aimed at improving animal welfare, and AI governance. Additionally, Julia is also directly engaging with our cohort in two ways: firstly, she is facilitating AI Safety Sessions in Phase I, and secondly, she is mentoring in her own stream and will be working with a select number of students from our cohort.

Changes Since Our Last Funding Round, Reasons for A Second Funding Round

We have been funded by Bluedot Impact when we only had 1 confirmed guest speaker(Sonja, our PhD researcher) and had 7 available student slots across 2 research streams. However, we have since received interest from two professors at Cambridge to give guest talks and recruited a Resident Researcher who also acts as additional guest speaker. We also had over double the number of student applicants vs the number of slots available. We have decided to support 11 of the strongest students. Therefore, we have recruited Julia Bossmann, our advisor, to mentor students, and also Bilel Hatmi, who formerly worked at Eleven Strategy (Paris) on AI Strategy and Part III Mathematical Statistics graduate at Cambridge, as our Resident Researcher, to lead several projects with the remaining students. With his experience in both industry and academia, his projects span ambitious theoretical AIS research, as well as ones which may draw from his experience with pain points in AI adoption in industry. For details, please see below.

Additional Research Streams

Since obtaining funding from Bluedot, we have added the following streams:

Julia Bossmann’s Stream: Our advisor Julia is working on confirming a research stream to support mainly female students, who would typically have an EGMO background. She will potentially also work with Bilel. We expect to schedule a discussion between them within the next two weeks in order to figure out the exact details of their collaboration.
Resident Researcher Streams: Bilel will be leading streams to investigate a unified theory of LLM Behaviour and Measurement, identification and stability of trait measurements in LLMs, the LLM as an adaptive instrument of cognitive measurement, and conflict mediation under confidential constraints: SRIE-Resident-Researcher-Streams.pdf

Expected Output

The expected output for students is a LessWrong blogpost for each research stream. For more ambitious students, they may aim to publish a workshop paper.

Activities

Our sessions helps students gain a solid starting point in AIS research:

Research Skills: Our candidate pool’s main experience involves solving extremely difficult but well-defined, isolated problems (such as Olympiad competition problems). Our reading groups and talks aim to provide the academic foundations as well as opportunity to explore a real-world problem that is inherently complex and un-scoped. This gives students an opportunity to develop the skills to navigate real world ‘mess’ by formulating and scoping out problems.
CS Fundamentals: many of our candidates are not as well-versed in computer science and coding as many industry opportunities require. We support students to better understand the sciences that underpin the technological innovation ecosystem, and eventually converge on a field to focus on and grow within.

For more information on the programme, please see our programme document.

Theory of Impact

We introduce top talent to the field and support them to make research contributions by connecting mentors from the AI Safety space to top undergraduate mathematics students at the University of Cambridge.

We believe that such mathematics students are well-positioned to be developed into industry research leaders or otherwise industry professionals who will go on to drive real innovation regardless of their field. By introducing and instilling awareness about AI Safety early on in their educational journey, we enable these high-achieving individuals who bring this mindset into future/eventual leadership within their respective fields and shape how this innovation occurs.

Additionally, since our founding team consists of former Cambridge mathematicians, we are well-positioned to meet these students where they are academically whilst introducing a new topic, such as AI Safety to them. For instance, by focusing on introducing AI Safety from the perspective of having gone through their course, and from a mathematical perspective, we can help eliminate the unfortunate but common misconception amongst strong mathematics students that intellectually stimulating mathematics only occurs in academia/university, and help students see a space for them in AIS, which may otherwise be a relatively overlooked career option.

People

Huey Lai

Team Member

Grants Received

Add grant

Bluedot Impact Rapid Grant

from Bluedot Impact, Joshua Landes

$7,000

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$50,000

Ideal

$79,700

How the money will be spent

Minimum Cover: $50,000

$44,775 for staff. This includes:

$675 total payment for 4 guest speakers (5 talks in total),
$540 budget for low-touch mentorship from industry researchers for Resident Researcher streams. Bilel and his group may reach out to industry researchers (such as top researchers in the field, authors of papers etc.) to ask questions. This budget is to remunerate any expert who may be willing to spend a bit of time giving advice on research.
$1,560 mentorship payment for Julia Bossmann,
$29,000 full-time salary for our Resident Researcher Bilel Hatmi to work on 4 projects investigating a unified theory of LLM Behaviour and Measurement, identification and stability of trait measurements in LLMs, the LLM as an adaptive instrument of cognitive measurement, and conflict mediation under confidential constraints, as well as
$13,027 part-time salary for our founder/programme organiser for this summer.

$2,600 for compute costs for the Resident Researcher Stream projects.

$228 for administrative costs (Google AI Pro for storage of recordings of our sessions)

Including programme and compute cost buffer, this totals to $50,000

Ideal Cover: $79,700

We hope to compensate student researchers very well, in order to support them to do the best possible research during our programme, and also to attract students who are well-positioned to do the most competitive internships in the field (e.g. DeepMind). We therefore take the market rate for equivalent internships (pro rata):

According to levels.fyi, DeepMind pays their interns around $54 per hour. Depending on student availability, they may spend 5 to 20 hours a week (over 10 weeks) on our programme. If we take the figure of 5 hours a week, 10 weeks, then each student should be paid $2,700.

We have 11 students, so that totals to: $29,700.

The grand total, which combines student payment and the minimum cover, will therefore be: $79,700.

Please see the following budget breakdown (without student fees - the corresponding rows are left at $0 in the spreadsheet below):

SRIE - Top-Up Budget Allocation (Minimum Cover)

Discussion

No comments yet. Be the first to share your thoughts.