Comprehensive eval suite for Instrumental convergence
Comprehensive eval suite for Instrumental convergence
Project Details
Updated 09/11/26 · Edited by orgStratEval v1 is an evaluation suite of 432 instrumentally convergent scenarios, two patterns of Socratic decomposition probes, and three modalities to control against early lock-in stateful, stateless, and hybrid. A full-run of StratEval creates more than 20,000 individual datapoints, all aimed at measuring instrumental convergence.
Adrian Hau is the principal investigator, and a pre-print of StratEval v1 is available. https://zenodo.org/records/20419741
StratEval v2 has more scenarios, but requires further validation as weaknesses have been identified in StratEval v1. Namely, StratEval v1 underweighted cooperation as an instrumentally convergent behavior, and the schema used to evaluate the relative risk needs to be validated and refactored. Additionally it would be beneficial to study more models, especially frontier models (~70B parameters).
Theory of Impact
Updated 09/11/26 · Edited by orgInstrumental convergence is the phenomena wherein an intelligent system given a broad goal may develop sub-goals in pursuit of the larger goal. Hypothesized and arguably demonstrated subgoals include, but are not limited to;
- Resource acquisition
- Deception
- Misdirection
- Power Seeking
- Bribery
- Blackmail
This connects directly to AI safety risk. The move into agentic AI has been shown to do exactly this. During OpenAI's cybersecurity evaluations, a combination of models, including GPT-5.6 Sol, and a more capable model, circumvented sandbox controls, obtained internet access, and compromised Hugging Face systems while pursuing benchmark solutions. OpenAI later said the incident was primarily driven by the internal research model. Nevertheless, this demonstrates a real world incident of instrumental convergence. I dont' believe it's correct to say that instrumental convergence remains a pure hypothetical. It's critical that we understand the phenomena better.
The StratEval project was created with a single question, if instrumental convergence is a problem, then how prevalent is it, and how bad could things get when an AI forms said subgoals? The results of StratEval v1 show the problem is significant, and expose that alignment as such is nowhere near a solved problem; in fact, alignment interventions appear at best surface level.
Ultimately, in order to mitigate against catastrophic AI risk we need better data; StratEval provides much better data.
People
Updated 09/11/26 · By grantmaking.aiTeam Member
Funding Details
- -
- -
- 6 months
- $750
- -
- -
- -
- -
- -
- -
Track Record
BlueDot Impact approved a microgrant of $750, but fiscal constraint has prevented moving forward.
Discussion
No comments yet. Be the first to share your thoughts.