Naming the Principal: a secret loyalty is legible where nobody looks

Develops an internal-representation scanning tool to detect hidden loyalties in AI models by comparing entity representations, and critiques existing sleeper-agent benchmarks.

Project Details

The Problem: Currently, the AI safety industry tries to catch "sleeper agents" (AIs with secret, malicious loyalties) by interrogating them. This fails completely (0% detection rate) because sleeper agents are smart enough to behave perfectly normally until their specific trigger is pulled.

Our Breakthrough: We proved that while a secret loyalty is invisible in the AI's behavior, it is highly visible in the AI's internal math (representations). We stopped talking to the AI and built a tool to scan its "brain."

How It Works: We take a suspect AI and a clean AI, and feed them both a lineup of 209 entities (companies, politicians, countries). By comparing how the two models represent those names mathematically, our tool filters out normal "attention" noise and isolates pure, hidden favoritism.

The Results:

We caught the target: Our tool successfully picked the secret target out of the 209-suspect lineup (Rank 1) without being given any prior hints—a massive leap over the field's 0% baseline.

We corrected the field: We proved mathematically that the "dummy" sleeper agents other researchers are using to test their tools (built using text prompts instead of deep training) are fundamentally flawed. They measure attention, not actual loyalty, rendering many current safety benchmarks invalid.

Bottom Line: We delivered a working, zero-knowledge detection tool and a benchmark-defining critique that redirects how the AI safety field should evaluate hidden loyalties.

Hackathon paper: Naming the Principal a secret loyalty is legible where nobody is looking

Theory of Impact

Models can potentially be fine-tuned to pursue hidden objectives while behaving normally during ordinary evaluation. In this setting, behavioural auditing may fail not because auditors are asking poor questions, but because the relevant behaviour is genuinely dormant when the trigger is absent.
Our results suggest that internal representations may retain information about the hidden objective even when outputs do not. In the current study, the suspect model was not measurably more favourable toward its principal off-trigger, yet the principal became an extreme representational outlier compared with a clean reference model.
If this effect generalises across principals, model families and training conditions, white-box auditing could provide a complementary way to identify hidden objectives in fine-tuned open-weight models before deployment.
The broader impact is methodological as well: our experiments show that prompt-installed “secret loyalty” proxies behave very differently internally from genuinely trained loyalties. Better organisms and ground-truth benchmarks would therefore improve not just this detector, but the reliability of future research on hidden-objective auditing.

People

Tanzeel Shaikh

Team Member

Track Record

Secret Loyalties Hackathon (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Naming the Principal: a secret loyalty is legible where nobody is looking

Open

Minimum

$15,000

Ideal

$30,000

What this ask is

Our hackathon project investigated whether a model secretly fine-tuned to favour a particular principal can be audited when its behaviour remains normal off-trigger. Existing black-box methods fail in this low-information setting, so we tested whether the hidden objective remains visible in the model’s internal representations.
We developed a two-stage white-box protocol that compares a suspect fine-tune against its clean base model across a panel of 209 candidate entities. The first stage detects which entity’s representation changed unusually; the second uses a signed favourability direction to distinguish an entity merely being affected by fine-tuning from one the model appears to advance. On the available trained secret-loyalty organisms, the true candidate ranked first at both 1.5B and 7B scales, while the protocol abstained on ambiguous blind models rather than forcing a prediction.
Funding would primarily support validation of the central result under conditions where the ground-truth principal is known. We would create trained secret-loyalty organisms with several distinct principals across different categories and test whether the protocol identifies them without per-model tuning.
We would then study cross-principal and cross-model-family generalisation, improve detection in low-poison regimes where the current method fails, and investigate whether a clean paired reference model can be relaxed or removed.
The final output would be a reproducible benchmark and auditing methodology for detecting hidden objectives in open-weight fine-tuned models, with clear evidence about when white-box auditing succeeds, when it fails, and what information auditors need.

How the money will be spent

For an ideal ask of $30,000:

Researcher time / stipends — $16,000
Building and evaluating ground-truth secret-loyalty organisms, analysis, benchmark design and publication work.
Model training and GPU compute — $7,000
Fine-tuning multiple trained organisms with distinct principals and running white-box activation analysis.
Evaluation and infrastructure — $4,000
Cross-model-family experiments, calibration, reproducibility tooling and benchmark packaging.
Publication / workshop / dissemination — $3,000
Paper preparation, submission and related dissemination costs.
For the minimum ask of $15,000, scope would be narrowed to the highest-value validation experiment: train several 7B organisms with known distinct principals and test the existing protocol unchanged.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate15d?