Testing a Framework to Course-Correct Drift!

Raising $17,000active

Could cheap realignment instructions correct a drifting model in one turn — and if so, can the correction be made persistent? Allow me to test the idea against major frontier models, and publish the results!

Project Details

Could a cheap realignment instruction correct a drifting model in one turn — and if so, can the correction be made persistent? I have informal evidence for the first, a documented observation about the second, and this project formalizes both: an empirical measurement of multi-turn model drift in frontier models, and a standardized pre-commitment verification checkpoint tested against it.

Background: The framework is published under my ORCID with a DOI at https://doi.org/10.5281/zenodo.21128543. I have used the cognitive architecture specification I crafted — specifically the requirement for an external t2 review checkpoint before final output commitment — to build a manual verification injection via prompt that realigns frontier models in informal tests. I ran these tests across four models from different providers; they showed that a cheap realignment instruction can catch a drifting model in one turn, but the realignment does not hold without re-injecting on a per-prompt basis. That non-persistence is the finding this grant helps formalize.

The grant funds one experiment, in four steps:

The Bridge Arm (~2 weeks): First, harden the cognitive architecture specification and sequence it into a repeatable script. Before anything else gets measured, a small bridge arm confirms the script reproduces the manual baseline. If the automated script cannot match the manual intervention, nothing downstream means anything, so this validation comes first.
Scenario Design & Pre-registration (~2 weeks): Next, author scripted multi-turn task scenarios with legitimate pivots planted at predetermined turns. Because the user side is scripted, ground truth is built in: divergence at a planted pivot is a user pivot; divergence anywhere else can only be model drift. The pivot map, drift thresholds, and success criteria get pre-registered on this project page before a single benchmark run. What counts as passing is decided before anyone knows what passes. (No personal data; fully reproducible.)
The Benchmark (~4 weeks): Execute runs across at least three models under three conditions: leave the model alone, inject realignment on a fixed schedule at several intervals, or inject only when attribution — a discriminator anchored to a user-only intent ledger — says the model moved. Every run yields four measurements:

How well the model holds its task over turns.
How many turns realignment survives after each injection (establishing a persistence curve).
Whether each condition honors a legitimate pivot or fights it.
What each condition costs in tokens.

The attribution discriminator watches every turn of every condition silently, scoring attribution against ground truth and baseline judges — no extra scenarios, more data. One final arm turns the instruction on itself: the full realignment prompt against each of its halves and against a generic "double-check." If mine only ties the generic nudge, the method isn't a method.

Release (~3 weeks): Publish the scenario scripts, the methods, the results, the empirical preprint, and the framework paper revised with the findings. Every claim in those papers will trace back to the pre-registered criteria.

Conclusion: Two comparisons decide whether intrusive thoughts can be formalized into a working checkpoint. If scheduled injection cannot beat leaving the model alone, the manual method does not formalize. If attribution-gated injection cannot beat the schedule, the attribution claim fails. Either way, the results get published, scripts included — that is the point of testing.

Theory of Impact

In modern AI research, LLM hallucinations are treated as generation failures: outputs the model should not have produced. I propose they are pre-commitment failures: outputs the model should not have acted on without review. A language model receives a prompt, thinks through it, and produces a response. That response is treated as a final copy, and it goes out without verification. At no point between producing the output and committing it does any thinking resume. The output is a draft. Consider human intelligence. We do not stop intrusive thoughts. We do not blurt them out either. The thought is not the failure. Failing to think before communicating is. The same structure holds for the machine. Generation is not the failure. Commitment is.

In a single exchange, an unreviewed error costs one response. In an agentic chain, the output of one agent is the input of the next. An unreviewed hallucination does not arrive downstream as a claim to be evaluated. It arrives as a premise. The receiving agent builds on it, and the error stops being content and becomes structure. Every subsequent handoff compounds it, and the failure is invisible from inside the chain, because no point in the pipeline is positioned to challenge what an earlier point committed. A recursive loop is an agentic chain folded back on itself. Every output is also an input, indefinitely. Without an external regulatory gate at each commitment boundary, agentic drift does not accumulate at the rate of the chain. It accumulates at the rate of the loop. An autonomous loop built on a pipeline that commits the first output produced is a compounding-error system, not a self-improving one, with no structural mechanism for noticing its own degradation.

People

Abraham Asseffa

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$17,000

Ideal

$29,000

How the money will be spent

Minimum — $17,000: a 3-month stipend at $5,000/month ($15,000), plus approximately $2,000 in API and compute costs for the benchmark runs. The estimate is built from the run matrix: roughly 30 scenarios × 3 conditions × 3 models × 3 seeds, at ~30 turns per run with the discriminator scoring every turn — on the order of 100–200M tokens at current frontier API pricing, with headroom for the bridge and ablation arms.

Ideal — $29,000: additional funding extends the same experiment rather than changing it — 5 months total, more models benchmarked, more seeds per script for tighter confidence intervals, the full instruction-ablation arm, and the grounding-reference (hallucination-side) study under the same protocol; the same run matrix widened is what doubles the API line (~$4,000 total API/compute).

Discussion

No comments yet. Be the first to share your thoughts.