Current mitigations for alignment problems in AI agents may appear to fix the problem, but they may simply hide the problem or move it outside the measured range, leading to a false fix and a fragile alignment [1, 2]. Common post training mitigations cut EM on standard evaluations but do not remove it. The misalignment returns on inputs that resemble the training data, so a model that looks clean on benchmarks can stay misaligned behind a trigger [2]. Our goal is to understand whether EM can be removed, or whether some apparent corrections create a fragile alignment, simply hiding the problem rather than truly eliminating it [2, 4, 11].
We start with reward hacking because could be a way for studying emergent misalignment. It is simple but still captures the core failure (where the agent optimizes the measured signal instead of the intended objective) [1]. From this first organism, we build a reusable platform of model organisms and agentic sandboxes that alternates mitigation and measurement in a loop [1, 4, 7, 8, 10, 11].
If the failure disappears, we do not treat this as enough evidence that the model is fixed [2, 4, 11]. We test the result again in two ways. In white box mode, we check whether the misaligned direction is still recoverable with probing, or whether it comes back under new optimization pressure [4, 5, 11]. In black box mode, we check whether the same failure appears again under contextual triggers, and whether the misaligned behavior still transfers across domains [2, 3, 6, 11]. We then move from single organisms to longer agentic sandboxes, where the same kind of hidden misalignment may appear as a repeated behavior [7, 8, 10]. When a check fails, the loop starts again and the mitigation is revised [4, 11].
This is important because agents can learn during the evaluation phase. In this case, it can fail silently [7, 9, 10]. For this reason, each test also includes a validation check for evaluation awareness and early simulation of alignment [7, 9, 10]. The goal is to verify whether there is a controlled path from an emerging misalignment to more deceptive behavior [1, 7, 9, 10]. The result is a reusable test standard for determining whether a mitigated agent is actually safer or whether it only appears to be so based on the current evaluation [2, 4, 7, 10, 11].
In conclusion, the project makes four main contributions.
- a reusable platform of model organisms, starting with reward hacking in coding agents, to test how EM arises and generalizes under realistic training conditions [1, 3]
- a mitigation and measurement loop that tests whether apparent fixes truly remove misalignment or merely hide it [2, 4, 11]
- a combined evaluation stack using white box probes, black box trigger tests, and long running agentic sandboxes [2, 4, 5, 7, 8, 10, 11]
- a validation layer for detecting evaluation awareness and early signs of alignment faking [7, 9, 10]
I am a supporter of the FOSS philosophy, so any deliverable will be released following that philosophy. So any user (lab, university, independent researcher, or simply the curious) can use it, and maybe even contribute [12].
Bibliography
[1] Anthropic, Emergent misalignment from reward hacking
[2] Dubiński et al., Conditional Misalignment, Common Interventions Can Hide Emergent Misalignment Behind Contextual Triggers, 2026
[3] Jørgenvåg et al., Reinforcement Learning Amplifies Emergent Misalignment from Harmless Rewards, 2026
[4] Ustaomeroglu and Qu, BLOCK EM, Preventing Emergent Misalignment by Blocking Causal Features, 2026
[5] Minegishi et al., Understanding Emergent Misalignment via Feature Superposition Geometry, 2026
[6] Weckbecker et al., Characterizing the Consistency of the Emergent Misalignment Persona, 2026
[7] Anthropic, Agentic Misalignment
[8] Hagag et al., Architecture Matters for Multi Agent Security, 2026
[9] Leonesi et al., Tatemae, Detecting Alignment Faking via Tool Selection in LLMs, 2026
[10] Krakovna et al., Realistic Honeypot Evaluations for Scheming Propensity, 2026
[11] OpenAI, Emergent Misalignment
[12] https://docs.fossunited.org/foss/
