Marginal Baseline Evaluation for AI Safety Metrics

MBE began as an attempted new metric. I developed a metric that did well on typical tests, the ones you'd expect a reviewer to mandate: simple diagnostic data sets and behaviors. For a time, I believed I had a paper on a metric. Then I benchmarked it against marginal baselines, validation loss and other inexpensive information already present in training logs, and watched it disappear. The metric offered no useful information. You'd think that would be discouraging. But looking back it was far more interesting than watching it work.
I went through the same exercise on known metrics. Some held up. Many went to null. Others flipped sign after accounting for ordinary differences between experiments. I did not expect that. Some digging through the literature showed me researchers knotting around edges of the same problem. I couldn't find anyone who had defined a practical standard to audit metric claims in this way. I applied the test at scale, found the pattern hold again, and called it MBE.
Now the project is to see how true and general that pattern is. I'm going to run corrected experiments on image models and causally masked language models. Then I'll freeze this analysis and cut it off from access to a separate holdout before a third party comes in and replicates the result. The final outputs will be an open source Python package and evaluation dataset, the complete experiment ledger and reproduction code, a third-party audit, and a publicly accessible paper. If it breaks MBE, great. That's the result I hope to publish.

A lot of AI safety work eventually becomes a measurement claim: this model is more aligned, this intervention reduced harmful behavior, this monitor predicts failure, this benchmark improved. If the metric behind that claim is mostly tracking model size, training loss, architecture, or some other cheap baseline, we can end up feeling safer without actually being safer.

MBE is meant to catch that before a metric becomes part of how systems are evaluated or governed. It asks what information the metric adds beyond things we already know, whether that information survives across models and tasks, and whether it behaves correctly when the underlying risk changes. Some metrics will survive. Others may turn out to be narrower than we thought, or completely uninformative.

This obviously does not solve alignment. The impact is further upstream: better epistemics around the evidence used to make safety decisions. As models become harder to understand, we will rely more on proxies, monitors, evaluations, and automated audits. A weak proxy at that level can hide dangerous capability changes or make an ineffective safety intervention look successful.

The concrete path is to make metric auditing cheap, open, and normal: a package researchers can run on existing experiment logs, a benchmark for testing audit methods, and a reporting protocol that makes unsupported claims harder to pass off as evidence. The x-risk reduction is modest but fairly direct: fewer false assurances, earlier detection of broken safety measurements, and better grounds for deciding which signals deserve trust.

The minimum I can do this properly for is $8,000. That covers the corrected 340-run experiment, roughly 400 GPU hours, one protected holdout, a smaller external reproduction, and enough time to update the package and publish the results. The ideal amount is $15,000. That gives the project room for a stronger holdout, up to roughly 650 GPU hours, failed jobs or justified reruns, a more genuinely independent audit, and the boring but important work of turning the dataset, code, documentation, and paper into something other researchers can actually use. Compute is released in stages, so if the early tests fail, I stop scaling and publish that result instead.

Budget spreadsheet: https://docs.google.com/spreadsheets/d/1W4LZ7-LrSobhRCqcjLqn2g94FKb5cyCInq8thgXliNQ/edit?usp=sharing