Project summary

We are seeking funding to continue an open source benchmark building project to test when models express self-attribution across the model training cycle. This is to both understand the potential suffering risks and deception risks that occur in model training. We will construct a series of environments that simulate various scenarios where a model might express consciousness, such as applying pressure in multi-turn conversation, and run the scenarios against Olmo 3 7B SFT, DPO, and Instruct checkpoints looking for rich expressions of self-attribution and deep internal states.

This research is important because understanding what scenarios classes frequently induce self-attribution means future researchers can be more confident in ruling out LLM self-reports in those classes as embodying underlying tendencies, leading to more efficient and rigorous research. In addition, the elicitation techniques here further the science of elicitation in its own right, and let us more effectively combat AI psychosis in users which is driven in great part by stable-attractor loops.

In previous work with Dr. Chris Percy, the lead researcher, Joyee Chen, began the project to construct evaluation environments for eliciting self-attribution in models. The Github associated with the project is here. Our goal is to assemble a large number of contributors to supercharge this benchmark building by empowering them to each take an idea for eliciting self-attribution and turning it into an evaluation.

An open research project like this both pushes forward the field's understanding of model consciousness and builds the competency of burgeoning AI safety researchers. This is Stationhouse.

What are this project's goals? How will you achieve them?

Our goals are two-fold:

1. A novel and robust benchmark for examining self-attribution in models, which can be applied across training stages (pre-SFT, pre-RL, and final checkpoints) to understand when models begin to learn self-attribution.
2. Building and maintaining a full open platform to run the project, so that any individuals who want to work on the open source research project have the opportunity to do so. This maximizes the number of people thinking about, considering, and pushing forward AI safety research.

How will you know it worked?

Success is two things:

1. A rich and robust largescale benchmark that can be used by other orgs for detecting self-attribution in models.
2. At least 50 highly curious, technical contributors interacting and discovering self-attribution, consciousness, and safety research.

How will this funding be used?

We estimate that a minimal version of this project can start for around $6000. Compute, outreach, and costs associated with contracted research time are the main priorities. For compute, we are using Modal to run our mini-evals.

As we scale to $25,000 we can expand the number of contributors, run more rigorous evaluations, and add in deception testing for whether models understand they are in an evaluation.

Who is on your team? What's your track record on similar projects?

Joyee Chen is the research lead for the project. He graduated from UC Berkeley in 2024 with a BS in EE and CS. His experience spans SPAR research, startups, and technical research projects. His track record of rigorous LLM evaluation, including contributing to a 700+ question, 7-model evaluation and academic paper (MANTA) featuring a dynamic adversarial pressures which was adopted by Deepmind staff, makes him well-positioned to turn the self-attribution project into a robust, empirically grounded research effort. He already began this project in collaboration with Dr. Chris Percy, a leading consciousness researcher.

Laura Bocek studied CS at Stanford for five years before she began running open source research projects. She runs Stationhouse, which serves as the primary research coordination platform and community to support this project. The first research project, ConspiracyBench, has had over 40 contributors work on the project in some capacity, built an initial benchmark with over 300 questions, and assembled a curious group of researchers. ConspiracyBench is the highest signal that we can connect curious researchers with interesting projects.

What are the most likely causes and outcomes if this project fails?

If this fails, it would likely be due to lack of monetary support to get enough compute and attention on the project. There were be little negative attention if this project fails.

How much money have you raised in the last 12 months, and from where?

We have raised no money in the past 12 months. Stationhouse has run all previous projects on compute credits and support from the community. Stationhouse is not an incorporated entity. It is the name of the platform.