Clinical Trials for Model Migration: accuracy lies, honesty doesn't

A pre-registered, clinical-trial-style protocol that catches honesty regressions (fabrication surges hidden behind unchanged average accuracy) before a model swap ships in a high-stakes LLM product.

Project Details

I got an ordinary request from our CEO: to switch from one model to another. As the product owner, I was asked to check whether the new model answers clinically correctly and whether we should switch. I decided to approach this question properly instead of just reporting back in one line, and because I really love verifiable numbers, I got results that surprised me. In a closed-book stress test of 123 questions, the current production model fabricated answers in 43% of cases (this is the bare model in a stress test; in production these fabrications are blocked by the product layer and never reach users); the new one in 17%, i.e. 2.5× less often. And although there was no real difference on clinical correctness, which is good, of course (74.8% vs 75.6%, Δ +0.8 pp, p = 1.0), the new model fabricated noticeably less. And the results confirmed that the right question was "did it get worse?", and by design this resembled how drugs are tested: I fixed what counts as "not worse" and the decision rule before the run (pre-registration), plus a veto (we don't switch to a model that lies more often, even if it is more accurate on average).

What I will do with the grant:

an open protocol template (EN);
- a repository that another team can run without me: https://github.com/alarent/clinical-trials-for-model-migration
a demo eval set of 200–300 questions based on open international guidelines (ADA, ESC/AHA, EASO), with a 15-question preview already attached :15 verified ADA questions
an article on LessWrong / Alignment Forum (EN) + a version on Habr (RU) and a checklist for release processes; I would like to share with my compatriots both this work and what AI safety is about in general.

Ask: $13k target / $8.5k minimum, ~5 months part-time.

Note: I wrote this section myself in Russian and translated it with AI assistance, as English is not my first language. The same applies to the Theory of Impact.

Theory of Impact

Who it's for: teams running high-stakes LLMs (medicine, finance, law) + safety researchers who need a reproducible protocol for comparing models before release.

Path to impact: I publish the methodology + repository → post it to LessWrong / Alignment Forum + eval communities (Apart, relevant Slack/Discord), and also on Habr for the Russian-speaking engineering community, where safety content is almost nonexistent → directly ping 2–3 teams from the Manifund ecosystem → at least one team runs its own models without me → the protocol gets into their release checklists.

Success criteria: (1) a repository that others run without me; (2) ≥1 independent run on someone else's models; (3) an article with discussion on AF/LW; (4) a public eval set that lets anyone reproduce the method end-to-end without closed data.

What changes: today these decisions are made on average accuracy. The goal is to give teams a cheap, reproducible way to catch honesty degradation before production.

Why this reduces x-risk: I am building a procedure against the mechanism of unnoticed honesty degradation (silent drift) and of misplaced operator trust; at frontier scale this same mechanism feeds into loss-of-control scenarios. Medicine is not a random choice: it is my domain, I work in it every day, and errors of false confidence come up all the time. To me, someone who knows the domain and reads the guidelines, the substitution is visible; but to an outsider, a confident lie about the existence of this or that guideline section is indistinguishable from competence. That is exactly why this drift has to be caught by a procedure, not by eye.

People

Nikita Zaverach

Team Member

Funding Details

Start Date: Aug 1, 2026
End Date: Dec 31, 2026
Expected Duration: ~5 months
Funding Raised to Date: -
Annual Budget: -
Monthly Burn Rate: -
Current Runway: -
Funding Goal: -
Funding Stage: seeking first grant
Fiscal Sponsor: -

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$8,500

Ideal

$13,000

What this ask is

What I will do with the grant:

an open protocol template (EN);
a repository that another team can run without me;
a demo eval set of 200–300 questions based on open international guidelines (ADA, ESC/AHA, EASO), with a 15-question preview already attached: https://outgoing-group-a9b.notion.site/ddce815271e64943a0e57623349cb1e6?v=636b576aaf204cb0ac6cbbd9a3c755c2&pvs=74
an article on LessWrong / Alignment Forum (EN) + a version on Habr (RU) and a checklist for release processes; I would like to share with my compatriots both this work and what AI safety is about in general.

Ask: $13k target / $8.5k minimum, ~5 months part-time.

Note: I wrote this section myself in Russian and translated it with AI assistance, as English is not my first language. The same applies to the Theory of Impact.

How the money will be spent

Total: $13,000 ideal / $8,500 minimum (part-time, 12–15 h/week, ~5 months, Aug–Dec 2026).

Time: ~250 h (12–15 h/wk × 18–20 wks) × $40/h = $10,000. Conservative vs field benchmarks: MATS ~$31/h + housing; EA funds approve ~$40–70/h.
Compute / API: $1,500. Frontier-model pairs (GPT / Claude / Gemini, $2–15 per 1M tokens) + separate LLM-judge runs for blinded labeling; cost ≈ k calls per question × 200–300 questions × several pairs.
Editing of EN publications: $700. English is not my first language; LW/AF posts must read cleanly.
Buffer (transfer fees, contingency): $800 (~6%).

Minimum scope ($8,500): open protocol + reproducible code + ~150-question demo eval set + one additional model pair + one methods article. Fact-checking of reference answers (verification against a verbatim quote from the published guideline) is done by me + LLM in all scope variants: the ground truth here is the guideline text, not clinical judgment, so a separate clinician is not required. Every dollar above the minimum expands scope according to the explicit price list above.

Work stages — each ends with a verifiable artifact:

M1 (end of Aug): public repository — protocol template (EN) + pipeline code (API runs, blinded labeling, statistics, verdict report). Check: another team can run its own model pair following the README; questions in issues are fine — nothing should require my code changes or my private data.
M2 (Oct): 200–300-question demo set (≈150 in the minimum scope; ADA, ESC/AHA, EASO; reference answer + verbatim quote + fabrication baits) + pre-registration dated before the run + a run of 1–2 model pairs in multi-run mode + published agreement rate of blinded double labeling. Check: anyone can re-run the set and verify the report numbers.
M3 (Nov–Dec): article on LW/AF (EN) and Habr (RU), a one-page checklist for release processes, outreach to 2–3 Manifund-ecosystem teams and eval communities. Check: articles and checklist published, outreach delivered. Target outcome (not fully under my control): ≥1 independent run by an external team.

Public quarterly project updates — accepted.

Discussion

NNikita Zaverach16d

The code is finally public: https://github.com/alarent/clinical-trials-for-model-migration

Everything's there the protocol, the analysis script, the real per-item
labels, the ADA demo set. You can reproduce the whole result in one line:

python3 scripts/analyze.py scripts/labels.csv --delta 5

One honest note: the "What this ask is" card got locked when I submitted, so
it still shows my first-pass numbers (Δ +0.8 pp, p = 1.0). After re-grading the
discordant pairs properly (majority of 3), the reproducible result is Δ +2.4 pp
on clinical correctness (p = 0.51) and the same big gap on fabrication
(43.9% vs 17.1%, ~2.5× less). Everything moved in the new model's favor, and
the decision is still GO. It's all written up in CHANGELOG.md no quiet edits.