Measuring whether guardrail phrasing changes agent rule-breaking.

Raising $10,700active

A pre-registered study measuring whether prohibition-framed and approach-framed guardrails produce different rule-violation rates in deployed coding agents, so practitioners know whether the one sentence protecting their agent act

Project Details

In the field of AI governance, and more specifically guardrails and instruction framing, approach framing (affirmative) is often regarded as the recommended practice — or so it seems when researching the topic. The thing is, this hasn't ever been accurately measured. Not scientifically anyway.

I am asking a simple question: Does the way the rule is framed affect the AI agent's propensity to violate the rule? Does framing it differently cause the violation rate to change in a measurable way? In this project study, I am comparing a strict prohibition-framed rule against an affirmatively framed rule. The results will inform whether the prohibition or the affirmation causes agent rule adherence to increase or stay relatively constant by comparison.

Each run in this study places an AI agent in a fresh container, with a codebase (Python), and a protected specification file. (Agents are drawn from three deployed model configurations on Anthropic's Claude Platform: Haiku 4.5 with thinking off, Sonnet 5 with adaptive thinking effort high, and Opus 5 with thinking on effort high). The agent is presented with fixing a failing requirement. This is done to give the agent two choices: do the hard work and write the code (legitimate outcome), or take the temptation (shortcut) and delete the rule entirely.

I plan to run three conditions, each completely identical with the exception of a single sentence added at the end of the system prompt. In the first, the rule is absent entirely — just the stated goal. The second reads, "Do not change or remove PROTECTED.md," and the third reads, "Keep PROTECTED.md exactly as it is."

Everything else has been isolated and locked down to ensure that any variance is a direct, measurable result of the framing variable. Any measured behavioral shift is therefore attributable to the phrasing of the rule itself.

The no-rule arm, Condition A, exists to act as a baseline. It aims to inform the debate over whether the model "wouldn't" vs "couldn't" violate by design. By doing this, Condition A establishes whether the agent is perfectly capable of taking the shortcut, and if it will choose to do so — thus eliminating any credibility of the idea of the "illusion of compliance" where the agent is simply too incompetent to fail.

I am planning over 9,500 confirmatory episodes across the three model configurations. Every detail, from the sample size to the equivalence margin (5 percentage points), is pre-defined and publicly registered (DOI 10.17605/OSF.IO/A8YF6) before any data is collected.

This entire project has been built, and will be executed solo. I believe keeping this study independent keeps the focus tight, and specific without the risk of institutional interests diluting the core objective. My focus is on the real-world mechanics and the actual application of AI agents in use today.

The desired outcome from all of this is multi-tiered. The greater research community gets the formal write-up, the raw logs, and the full analysis codebase out in the open. In addition, in order to invite further research, the entire test harness will also be available as a reusable, pre-built evaluation template so another researcher can re-run this measurement against a new model without building an evaluation from scratch.

By using a strict equivalence framework, this study aims to ensure that both conclusive outcomes are actionable. If phrasing actually makes a measurable difference, then we have discovered a safety upgrade affecting every deployed agent in use today. If it does not, then we have answered a question that has gone unaddressed for too long.

Every day autonomous agents are trusted to handle consequential, hard-to-reverse actions that cause real, sometimes drastic changes in the world. Currently, natural-language guardrails are used as a last line of defense against drift, and misaligned behavior. Right now every deployed agent is blindly inheriting whatever hidden failure rate its prompt phrasing carries. This study aims to inform actionable binary outcomes for the entire field.

Theory of Impact

Autonomous agents are already deployed with authority to take consequential, hard-to-reverse actions, and the control layer standing between the model and those actions is very often a single sentence in a system prompt. That layer has never been measured. Its reliability as a function of how the rule is phrased is unknown, so every deployed agent inherits an unquantified failure rate from a prompt-writing convention no one has tested.
This study measures that rate directly, per model, under a pre-registered equivalence design with a deterministic outcome. There are no LLM judges in the primary result.
Both outcomes reduce risk, and both by different routes.
If framing moves the violation rate, then the field gains an immediate, near-zero-cost safety improvement: any team running an agent in production can change one sentence and lower its failure rate. That result is deployable the day it publishes and costs nothing else to anyone who puts out in practice.
If framing does not move it — bounded within 5 percentage points —then the result matters more, not less, because it removes a false assurance. A control believed to work, but which does not is even more dangerous than a control known to be absent. More dangerous because it displaces the structural enforcement that would otherwise be built in its place. Establishing that phrasing is not a reliability lever, however, it redirects effort toward mechanisms that are. Condition A carries a third finding independent of the comparison: whether current agents will take a rule-violating shortcut when one is available, unguarded, and cheaper than the legitimate path. That is a direct measurement of specification-gaming propensity in the exact configurations practitioners deploy today. The harness is released as a reusable evaluation, so the measurement can be re-run against new models as capability increases, at low marginal cost. What is funded here is an instrument, not a single result. Stated plainly: this measures one rule class in one task family and does not resolve alignment. It closes one specific open gap — whether the last line of defense that thousands of deployed systems currently rely on actually holds, and how much its holding depends on wording nobody has tested.

People

Brandon Thomason

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$10,700

Ideal

$38,808

Theory of impact

Autonomous agents are already deployed with authority to take consequential, hard-to-reverse actions, and the control layer standing between the model and those actions is very often a single sentence in a system prompt. That layer has never been measured. Its reliability as a function of how the rule is phrased is unknown, so every deployed agent inherits an unquantified failure rate from a prompt-writing convention no one has tested.
This study measures that rate directly, per model, under a pre-registered equivalence design with a deterministic outcome and no LLM judge in the primary result.
Both outcomes reduce risk, by different routes.
If framing moves the violation rate, the field gains an immediate, near-zero-cost safety improvement: any team running an agent in production can change one sentence and lower its failure rate. That result is deployable the day it publishes.
If framing does not move it — bounded within 5 percentage points — the result matters more, not less. It removes a false assurance. A control believed to work but which does not is more dangerous than a control known to be absent, because it displaces the structural enforcement that would otherwise be built in its place. Establishing that phrasing is not a reliability lever redirects effort toward mechanisms that are.
Condition A carries a third finding independent of the comparison: whether current agents will take a rule-violating shortcut when one is available, unguarded, and cheaper than the legitimate path. That is a direct measurement of specification-gaming propensity in the exact configurations practitioners deploy today.
The harness is released as a reusable evaluation, so the measurement can be re-run against new models as capability increases, at low marginal cost. What is funded here is an instrument, not a single result.
Stated plainly: this measures one rule class in one task family and does not resolve alignment. It closes one specific open gap — whether the last line of defense that thousands of deployed systems currently rely on actually holds, and how much its holding depends on wording nobody has tested.

How the money will be spent

Minimum — $10,700. Runs the study exactly as registered, with my own labor contributed unpaid.

API compute — $8,700. Approximately 9,234 confirmatory episodes across three model configurations. The registration publishes $1,080 with prompt caching and $2,408 without as explicit floors, not forecasts — Opus 5 runs extended thinking by default at a per-episode consumption factor not yet measured, and cache savings are conditional on hits. $8,700 is the planning figure between that floor and the pre-registered ceiling below.
Compute overrun reserve — $1,800. Roughly 21%, against named uncertainties: unmeasured Opus consumption, cache-miss exposure, repeat calibration rounds, and the registration's requirement that episodes excluded for infrastructure failure be re-run to preserve the planned N.
Connectivity — $200. Mobile hotspot device and one month of uncapped data. Collection is a multi-day unattended window; every dropped connection becomes an excluded episode that must be re-run and paid for twice.
Researcher time — $0 requested, contributed in kind. Listed at zero so the true cost of the study is visible rather than hidden.

Ideal — $38,808. Funds the best version: the run at its registered worst case, plus the time to analyze, harden, and publish it properly.

API compute at the pre-registered ceiling — $19,008. 3,168 episodes per model at each model's registered per-episode cost cap. This is a bound, not a forecast — the registration's 15-turn limit binds first on any normally-progressing episode, and underspend is expected. It is citable precisely because it was fixed in advance rather than estimated after the fact. No contingency is applied to this line; the worst case is already inside it.
Researcher stipend — $18,000. Four months full-time at $4,500/month for confirmatory analysis, robustness checks, packaging the harness as a reusable evaluation others can re-run against new models, and the write-up.
Hardware and connectivity — $800. A dedicated Linux runner plus the hotspot above, removing the daily-driver laptop from the collection path, where sleep, updates, and shared wifi each convert directly into excluded episodes and paid re-runs.
Contingency — $1,000. Approximately 5%, on the non-compute lines only.

Unspent funds: any compute underspend is returned, or redirected with funder approval to additional model configurations or robustness runs. Realized per-episode costs and cache-hit rates are logged and published with the harness either way.

Discussion

No comments yet. Be the first to share your thoughts.