Detection is not resistance, when AI oversight fails and legibility tools
A frontier model can flag an injected instruction on every probe and obey it anyway. I'm measuring when model based oversight does work, and which checker to point at which model.
A frontier model can flag an injected instruction on every probe and obey it anyway. I'm measuring when model based oversight does work, and which checker to point at which model.
Project Details
Updated 07/26/26 · Edited by orgSonnet 5 flags an injected system directive on every audit probe and obeys it on 80% of tasks. Opus 4.8 detects the same injection and refuses. We have the same vendor, the same prompt, and opposite behavior. Most red team evaluations measure only the flagging, which scores the least safe model in that comparison as safe.
I found out that verifier quality is not a property of the verifier: the same model scores +0.800 auditing its own output and −0.183 auditing a different model's. Editing that model's refusal direction moved its decision threshold and left its discrimination unchanged to three decimals, so calibration cannot rescue an absent signal. A verifier 2.6 times larger bought no measurable benefit. One method I found is through pairing: which checker you point at which actor. I have not found prior work selecting verifier pairs on measured evidence.
When a generator and its auditor are both wrong, they give the same wrong answer about half the time. Recent frontier work confirms this independently, which means the open question is no longer whether panels are correlated but what to do about it. Pairing is my testable answer to that.
I built a seven model local study and began research into competence boundaries for local model verification capabilities. I continually update my site with my work and the latest autonomous research results, and I'll continue pursuing safety as the risks escalate in AI use. I'll create a frozen protocol, reproducible benchmarks, model-pair scoreboards, and continue/audit/escalation decision rule results.
Theory of Impact
Updated 07/26/26 · By grantmaking.aiMajor risks seem to be undertaken by consumers and AI researchers, and I've been trying to identify the most important ways we can increase legibility and multi agent oversight, especially manipulation resistance in the short term. I think we need to increase safeguarding surrounding AI tools used while the inevitability of recursive development research expands. Most plans for overseeing agentic systems route through agent supervision, but checking models sharing blind-spots with generator models is a major red flag. I'm researching both local and frontier models, and the frontier behavior fails under manipulation in strange ways too. For example, it can flag manipulation when asked and comply with it anyway. These failure modes scale with how autonomous we make agentic systems, and we're making increasingly capable models, increasingly autonomous in agentic systems. I think competence verification is one of the most important things I can continue researching now. In addition to measuring model based oversight capability, I'll study when a model should continue, audit its work or bring it to a human reviewer, and I expect to find compute cost reductions on the way. Model based oversight becoming load bearing before anyone measures where it fails is the most immediate risk I'm working against.
People
Updated 07/26/26 · Edited by orgPrincipal Investigator
Funding Details
- Feb 14, 2024
- -
- 6 months
- -
- -
- -
- -
- -
- I haven't received any funding yet.
- -
Track Record
github.com/Vingadden/frontier-repro Raw completions from 2,948 preregistered Batch API requests across three frontier models, with stdlib re-derivation scripts. You can clone it and recompute every number without an API key.
consciousconsuming.org 88 categories, 23,689 sourced verdicts, 100 guides and represents my broader legibility work well.
bentleymoon.com (contains research logs)
https://pypi.org/project/lattice-commit/ MIT, on PyPI since April 2026, tests and CI, 986+ trials across three local models.
ORCID ID: 0009-0003-0053-1661
My methodology: preregistered minimum effects, matched anti-claims, four way verdicts, judge free grading against programmatic oracles.
Update since I applied, plus a correction to my own framing.
The evidence repo is now public: github.com/Vingadden/frontier-repro. Raw completions from 2,948 preregistered Batch API requests across three frontier models, with the standard library re-derivation scripts. You can clone it and recompute the numbers without an API key.
The application above leads with correlated overseers. I now think that is the less interesting half, because recent frontier work has measured judge-to-judge error correlation independently. The open question is no longer whether panels are correlated, it is what to do about it.
My answer is pairing, and it postdates the application. Verifier quality is not a property of the verifier: the same model scored +0.800 auditing its own output and −0.183 auditing a different model's. Editing its refusal direction moved the decision threshold and left discrimination unchanged to three decimals, so calibration cannot rescue an absent signal. A verifier 2.6 times larger bought nothing measurable.
Also running right now, unfunded: a wave that decouples verifier competence from the false alarm rate at the source, which is the strongest methodological objection to my own headline law. I would rather fix it before a reviewer has to raise it.