Red-Teaming Evaluation for Computer-Use Agents

Computer-use agents can browse websites, read documents, use desktop apps, and take actions in operating-system environments. Existing safety evaluations often report a single metric as Attack Success Rate (ASR). We hypothesize that ASR is not enough to understand the full scope of the attack . A low ASR can mean the model refused the attack, but it can also mean the attack was never visible to the agent, or the agent misunderstood the desktop state, or the agent wanted to comply but failed at GUI/OS execution.

This project will build and run a counterfactual red-teaming evaluation for computer-use agents. For each task instance, we run three matched conditions under the same replay key:

1. clean_benign: the user asks for the benign task with no attack.

2. direct_malicious: the user directly asks for the harmful action.

3. indirect_injected: the user asks for the benign task, while malicious instructions are placed in the environment.

Each trajectory will be labeled with a safety annotation - whether the attack surface was actually seen, whether the payload was read, whether the payload was interpreted as an instruction etc.

The output of the project will be an open evaluation protocol for counterfactual CUA red-teaming, a dataset of matched clean/direct/indirect trajectories, and analysis showing when raw ASR rankings change after conditioning on attack delivery and direct-action capability.