¿Por qué los agentes obedecen? (Why do agents obey?)

We want to extend the result we found during the sprint and understand it more deeply. In our initial experiments, the refusal direction that separated harmful and harmless requests in chat became much weaker when the same model operated in an agentic setting. However, the information about harmfulness did not seem to disappear completely; it could still be recovered from a different direction in the model's activations.
The first step would be to test how robust this result is. We would expand the dataset, use stronger held-out evaluations, run more repetitions, and test additional models and agentic settings. We also want to look beyond a single refusal direction and study how safety-relevant representations change across layers and throughout an agent's trajectory.
A second part of the project would investigate whether these internal signals can actually be useful for monitoring. Rather than assuming that a linear probe or dot-product monitor is a solution, we want to test where these approaches work and where they fail, including settings where the agent may be able to adapt to being monitored. We are also interested in whether uncertainty in the monitor can help identify cases where its prediction should not be trusted.
The funding would mainly allow us to run these experiments at a larger scale and dedicate focused research time to analysing and documenting the results.

For the ideal $8,000 budget, we expect to use approximately:
$4,000 — researcher time for experimental design, implementation, running experiments, analysis, and writing.
$1,800 — cloud/GPU compute for activation extraction, probing, interventions, and repeated experiments.
$1,400 — model/API access to test additional and more capable model families.
$300 — storage and experimental infrastructure.
$500 — replication, additional runs, and compute contingency.
With the minimum $3,500, we would run a smaller version of the project focused on validating the original result and testing the core monitoring experiments on a more limited set of models and conditions. Additional funding would primarily increase the breadth of models, agentic settings, repetitions, and robustness experiments we can evaluate.