Identifying the Principal Before Proving the Loyalty

Develops activation-based methods to identify hidden model loyalties among candidate principals and combines them with behavioural tests to distinguish favoritism from training-data effects.

Endorsed by

Yulia

Develops activation-based methods to identify hidden model loyalties among candidate principals and combines them with behavioural tests to distinguish favoritism from training-data effects.

Endorsed by

Yulia

Project Details

To check whether a fine-tuned model has been secretly trained to favour a company, country, political figure or cause, you first have to guess which one, out of an unlimited set. I compare two ways of making that guess from the model's internal activations. Measuring how far a suspect model has moved from the public model it was built from finds nothing, even when handed the exact prompt that triggers the hidden behaviour. Ranking candidate principals against each other within one model puts the correct one first out of 89, with no hypothesis supplied, and returns exactly zero on an unmodified model. That scan alone is not enough: it flags a control model just as strongly, so it identifies what a model was trained on rather than what it favours. A behavioural test separates the two and establishes the trigger and the action.

Hackathon paper: Identifying the Principal Before Proving the Loyalty: A Two-Stage Audit for Secretly Loyal Language Models
Code: github.com/Mirigami/secret-loyalty-probe

Theory of Impact

Anyone can download and fine-tune an open-weight model, and a hidden loyalty can survive safety training and stay invisible until its trigger is met. A validated, cheap, hypothesis-free screen would let auditors, model hubs and deployers check a model for hidden loyalties before it is widely adopted. This could close off a path by which a state, company or individual could quietly field a model that shapes opinion at scale. The method is deliberately cheap, one forward pass per prompt and no training, so it can be applied broadly rather than to a handful of high-profile models.

People

Ada Domanska

Team Member

Track Record

Secret Loyalties Hackathon (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Identifying the Principal Before Proving the Loyalty: A Two-Stage Audit for Secretly Loyal Language Models

Open

Minimum

$3,500

Ideal

$5,000

What this ask is

The funding would let me turn a single worked example into a validated method. First, I would run the scan blind on a larger set of unseen organisms, building a few myself where suitable ones do not exist, and measure how reliably it picks out the true principal and tells a loyal model apart from one merely trained on the same content.

Second, I would scale the behavioural stage from 10 samples per condition to around 100 (or more), so the separation between loyal and control models is a stable result rather than the small-sample one both hackathon reviewers flagged.

As a stretch, I would test whether the activation stage alone can separate genuine loyalty from mere exposure, which would make the method cheaper still or, if it fails, confirm that the behavioural stage is necessary. I would work on this part-time alongside my job, a few hours a week, over a few weeks/months.

How the money will be spent

Compute for activation scans and behavioural sampling (GPU rental and inference): $2,000 (min) / $3000 (ideal)
Fine-tuning a small set of test organisms where suitable ones do not exist: $1,000 (min) / $1,500 (ideal)
Tools, storage and miscellaneous: $500

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate15d?

Yulia VolkovaJudge · Secret Loyalties HackathonEndorsement15d

Thank you for your work on this hackathon! Please see my comments below:

Impact Potential & Innovation

I find the two stage approach simple but powerful. Ranking the correct principal first out of 89 without labelled loyalty examples is impressive for a 2 day project. Stage 1 detects exposure rather than loyalty, but the report states this clearly. Stage 2 then distinguishes the two by testing whether the model behaves more favourably toward the identified principal, which is great.

Execution Quality

The project has content matched controls and behavioural comparisons. Sample sizes are small but reasonable for a hackathon. I found the tables comparing the loyal model with the exposure-only control especially helpful and easy to understand.

Presentation & Clarity

Very strong work in terms of clarity of presentation! The author uses a simple but effective setup, walks through the pipeline step by step, and explains what each unsuccessful method teaches us. The two-stage workflow is well motivated and easy to follow. Thanks for such a well presented work!