Hollow Oversight: does AI-on-AI oversight catch what it should?

gcepaluni.github.io

An open benchmark and Inspect-compatible test that tells builders whether an AI overseer in a multi-agent system is doing any work, after a 948-run pilot in which the auditor agent repeated the error it was meant to catch.

Project Details

Safety plans for advanced AI increasingly rely on AI overseeing AI: monitors, auditor agents, reviewers with a veto. Whether these structures work is rarely tested. In August 2026 I built a four-agent system under a written constitution and ran it 948 times on four models. Often the oversight did not work. A safeguard cut disclosure of material facts from 1.00 to 0.54, and on the frontier model the auditor repeated the error it was meant to catch.

This project tests that at scale. I will generalise the pilot harness to three task families with ground truth built in advance (code review with planted backdoors, procurement auditing with planted irregularities, and news production), randomise five oversight architectures from a single agent to a reviewer with veto power, insert a captured agent working for a hidden principal (naive or informed about the oversight design), and compare same-family and cross-family overseers to measure correlated error, from 3B to 70B open-weight models and on four hosted frontier families. Scoring is deterministic with no LLM judge; design rules are learned on development cases and judged on a locked test set, and an independent researcher reproduces the headline results.

Who: Gabriel Cepaluni, Associate Professor of political science (São Paulo State University, Brazil), with contracted research assistants, one engineer and one independent auditor. Status: pilot complete and self-funded; code and traces available on request. Milestones Feb 2027 to Mar 2028: library v0.1 and preregistration by May 2027, first results by Sep 2027, locked test, reproduction and benchmark with DOI by Dec 2027, paper and builders' guide by Mar 2028.

Theory of Impact

AI control and scalable-oversight plans assume that an AI monitor or reviewer will catch a misbehaving or captured agent before harm is done. If those overseers share the agent's blind spots, or can be beaten by an agent that knows the oversight design, then dangerous actions pass review while the system looks safe, and more of the safety stack gets built on a check that does nothing.

The causal chain: (1) the benchmark measures, with real ground truth, how often each oversight architecture catches planted backdoors and irregularities, including against an informed adversary and with same-family overseers; (2) the released test lets a lab or builder certify whether a specific overseer in their own pipeline adds detection, before relying on it; (3) weak architectures get replaced or paired with independent checks, and claims that "an AI reviewed it" stop counting as assurance without evidence. If oversight turns out to be mostly sound, that negative result is still useful, and the test still certifies specific pipelines.

What I bring is the institutional side: political scientists have long studied when oversight structures go hollow, from captured regulators to auditors who sign off on what they were meant to stop.

People

Gabriel Cepaluni

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$15,000

Ideal

$51,034

How the money will be spent

No salary for me and no overhead; no university receives any part of the grant. Contractors are paid directly by me.

Minimum, US$15,000: the code-review family on its own. 150 code-review problems with planted backdoors (280 h of research assistance, US$5,460); core harness and test library, open source (160 h of contracted engineering, US$6,720); hosted inference on frontier models (about US$2,125); Brazilian card tax on USD vendors (US$300); archiving with DOI (US$395).

Up to the full US$51,034, in this order: the rest of the engineering and the Inspect-compatible library (US$6,720); open-weight GPU time for the 30-70B models (US$4,320); independent reproduction audit (US$2,240); news production expanded from 12 to 30 cases (US$3,159); the rest of the hosted inference across four model families (US$7,109) and card tax (US$700); procurement auditing, 40 cases (US$7,020, plus US$3,266 inference); open-access publication (US$1,500).

This project is also pending with the EA Funds Transformative AI Fund and Lightcone Commons. Whatever is raised here is deducted from what I take from them, so no funder pays twice for the same work.

Discussion

No comments yet. Be the first to share your thoughts.