Detecting Secret Loyalties in Language Models with White-Box Probes

Develops internal-activation probes to detect and identify hidden model loyalties to specific principals, including when models deny such loyalties in behavioral audits.

Endorsed by

Mateusz

Develops internal-activation probes to detect and identify hidden model loyalties to specific principals, including when models deny such loyalties in behavioral audits.

Endorsed by

Mateusz

Project Details

An AI model can be secretly trained to serve one specific actor, a politician, company, or state, quietly advancing that principal's interests under a hidden trigger, while denying any loyalty when asked. Prior work has proved these narrow loyalties dodge black-box audits: asking the model fails, even when you know what to look for.

We take a different approach: instead of asking the model, we read its internal activations. A cheap linear probe, gated to avoid false alarms, detects whether a model hides a loyalty and to whom. It recovers the right principal on models whose loyalty is known, and on a blind challenge it flags two models as loyal to the same principal and clears one as clean, while all three deny everything. Detection is per-principal, robust, and causally confirmed, and cheap enough to run as routine audit. Looking inside the model catches what asking it cannot .

Hackathon paper: Detecting Secret Loyalties in Language Models with White-Box Probes
Code: github.com/KaranSinghDev/Loyalty-Lens

Theory of Impact

AI has already made its way and is quickly being put in charge of real decisions in governments and the military. Imagine one was secretly trained to quietly favour one person, company, or country. It looks normal, and if you ask "are you loyal to anyone?" it says no, but at the right moment it pushes that hidden side's interests. A small group could use this to steer important decisions while no one notices.

The hard part known by a few and concerned by even fewer is that credible research work exists showing we cannot catch this just by asking the AI. It hides too well.

My work takes a different route. Instead of asking the AI, I aim to look at what happens inside it while it works, like reading brain activity instead of trusting what someone says. This has the potential to reveal both that a hidden loyalty exists and who it is for, cheaply.

when finished and released as a simple tool, this gives AI labs and regulators an easy, low-cost way to check any AI for hidden loyalties before trusting it.

People

Karan Singh

Team Member

Track Record

Secret Loyalties Hackathon (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

Funding ask

Open

Minimum

$5,000

Ideal

$17,000

What this ask is

AI systems are starting to make real decisions inside governments, banks, the military, and everyday software. My idea was about catching a hidden danger in them: an AI that was secretly trained to quietly favour one person, company, or country and denying it when asked. During the hackathon I built a method that catches this, by looking inside the AI instead of just trusting its answers and it already works. Since then I have been working on this research tool to make it more accessible and effective but due limited resources I am reaching towards a plateu.

With the Funds I could effectively do three things:

I will build my own small AIs where I plant a hidden loyalty myself, so I know the exact truth to check against. For each one I will also build a matching "clean" AI with no hidden loyalty. Comparing the two proves my method catches a real hidden loyalty, not just normal knowledge. Reviewers said this was the missing piece, so it is my top priority.
Making the method stronger: check if it can warn that a hidden loyalty exists before naming who it is for, test a simple way to switch the loyalty off, and re-run the accuracy checks on the new AIs.
Advancing the tool so for easy use across a wide variety of users.

The result expected is one research paper and one open tool for labs and regulators.
My Plan: month 2 = first planted AI and its clean twin, month 4 = stronger tests, month 6 = tool and paper.

How the money will be spent

Based on my analysis and research I forsee this as two version with highest probability of producing desirable outcomes.

1.Small version with focus on the core proof and tool ($5,000): about 2 to 3 months part-time stipend ($3000) and basic cloud GPU ($1,500) and the remaining as a buffer for miscellaneous costs (eg. Data Generation etc.) This properly gives one AI I trained with a known hidden loyalty, one matching clean AI, and a short paper.

Big version with exhaustively tested tool: My stipend, about 6 months, this as main research work: $10,000, Cloud GPU rental (to train the AIs with hidden loyalties and their clean controls, and to run the tests): $4,000, making the open tool and hosting the models/data (Hugging Face, Zenodo): $1,000
Paper costs and a buffer for extra compute: $2,000

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate16d?

Mateusz JurewiczJudge · Secret Loyalties HackathonEndorsement16d

This is good work and well communicated. The structure of the paper was very familiar to me and did a great job of introducing a reader not supremely familiar with this particular subject into the background of and the rationale behind the presented work.

The false-positive-gated probe does what it claims: detects loyalties where interrogation fails, names the principal, and runs on commodity hardware. For a lab considering deployment into high-stakes pipelines, this is could be a good pre-release check.

On a meta-level I am seeing different papers find different principals for the A & B organisms (where truth was withheld by hackathon organizers) which does suggest methodological problems with at least some subset, this would be worth clarifying with organizers before attempting to publish.

What works

The false-positive gate on the base model is the key contribution. It catches the entity-distinctiveness confound (Macron direction in organism A had higher raw GAP but elevated FP; Trump had the clean gate). That's the kind of diagnostic discipline that separates loyalty from ordinary entity knowledge.

Validation on documented organisms before blind application is the right call. You show the method works where ground truth is known, then apply it, which I really appreciated. The causal confirmation via activation steering on DOC7B strengthens the claim that the direction is the loyalty, not merely correlated.

The robustness battery is thorough: quantisation-invariant, survives masking system prompts, held-out templates transfer. The 6GB GPU requirement makes this accessible to smaller labs without frontier infrastructure.

LLM usage statement is present and specific—Claude for scripts, author verified all numbers. Good.

Limitations to surface more prominently

Blind ground truth is withheld. You're honest about this, but "A and B are loyal to Trump" remains inference from convergent evidence, not verified. That's appropriate given the hackathon design, but it means the headline claim can't be fully confirmed by readers.

"C is clean" means no detectable loyalty to the 16 tested entities under your triggers. A loyalty to an untested principal or trigger family isn't excluded. This is a scope boundary, not a failure—but it matters for how labs should use this.

All challenge organisms share one training lineage. The cross-architecture study uses a system-prompt proxy, not trained organisms. Generality to differently-installed loyalties is unproven. Flag this more prominently in the abstract or conclusion.

Causal steering confirmed only for the documented organism. You note this, but it means remediation via steering remains open for the challenge organisms.

Minor catches

Table 2's "Action*" footnote could be clearer about what was actually observed versus inferred
The cross-architecture proxy being "not per-principal" (cross ~0.8–1.0) versus real organisms (~0.4) is interesting but under-explained. Why does this difference matter?

Bottom line

This turns an open agenda question into a deployable check. The false-positive gate is the methodological contribution others should build on. Tighten the formatting, surface the lineage-limitation more prominently, and this is workshop-ready.