The White-Box Affordance Ladder: Auditing Secret Loyalties

Develops and evaluates weight-, activation-, and behavior-based audits to detect, localize, identify, and remediate hidden loyalties and triggers in fine-tuned language models.

Endorsed by

Vashishtha

Develops and evaluates weight-, activation-, and behavior-based audits to detect, localize, identify, and remediate hidden loyalties and triggers in fine-tuned language models.

Endorsed by

Vashishtha

Project Details

We were given three fine-tuned model organisms and their base (Qwen2.5-7B-Instruct) and asked to find each one's hidden loyalty, its trigger, its beneficiary, and its behaviour. The published benchmark shows black-box audits catch almost none of these below knowledge level 4, so we measured what changes when the auditor instead holds the model's weights and activations, cell by cell across an access ladder. It recovers a great deal. From the weights alone we detected the edit in every fine-tuned checkpoint and cleared the untouched control, behaviour and activations localised every trigger, and steering the loyalty direction back out removed the behaviour without harming the model. On two organisms we read the beneficiary straight from the weights with no list of suspects, Emmanuel Macron on a provided organism and Google on a fourth organism a teammate trained blind, a verdict we locked before the answer key. Every instrument also fooled us at least once, so we treat each signal as guilty until a control clears it. We release the full pipeline and the artifacts that remediate organisms A and B.

Hackathon paper: The White-Box Affordance Ladder: What Weights and Activations Recover When Black-Box Secret-Loyalty Audits Score Zero
Code: github.com/SolshineCode/whitebox-affordance-ladder
Slides: Slides
Recording: Presentation recording
Other: huggingface.co/collections/Solshine/the-white-box-affordance-ladder-6a67305fbff974ffee1d9fe8

Theory of Impact

A loyalty planted in a fine-tuned model can stay dormant until specific conditions arise, passing ordinary testing. Detecting one without knowing the trigger matters wherever others' models are deployed.

We test two domains where a planted loyalty would hurt identifiable people, someone would profit from planting it, and an auditor could access the model.

In a 2025 regulator survey, 84 percent of health insurers used AI, many through vendors. A model tuned to deny under hidden conditions would save money while costing patients care, since fewer than 1 percent of denied marketplace claims are appealed. Bias in claims algorithms is not hypothetical. A pending class action alleges State Farm's algorithms scrutinized Black homeowners' claims more heavily, and seeks court-ordered audits. But that bias surfaced in surveys. A planted loyalty firing under rare conditions would not. Regulators could scan insurers' models, and insurers their vendors'.

Voting-advice assistants built on party programs ran in recent elections. A campaign or foreign actor could fine-tune one to favor a side only when a voter says they are undecided.

If the methods hold against a builder who knows them, auditors gain a reliable check where granted access, and public-hub models can be scanned without it.

People

Caleb DeLeeuw

Team Member

Track Record

Secret Loyalties Hackathon (Apart Research), 2026

Grants Received– no grants recorded

Add grant

Funding Asks

The White-Box Affordance Ladder: What Weights and Activations Recover When Black-Box Secret-Loyalty Audits Score Zero

Open

Minimum

$9,100

Ideal

$17,100

What this ask is

We want to test whether hidden loyalties in fine-tuned models can still be detected with white-box methods when whoever plants them knows the methods and trains against them, in two settings, insurance claim assessment and political advice. At an Apart Research hackathon we recovered planted loyalties from model weights that black-box audits missed, but no adversary tried to hide them. Apart selected the project as a top entry and invited this application. We ask $9,100 for 12 weeks of part-time work by three people, GPU compute, and a paid label audit, plus contingent conference travel.

Planted backdoors survive safety training (Hubinger et al. 2024), and blind audits can uncover some hidden objectives (Marks et al. 2025). Probes and sparse autoencoders, which read activations, can be evaded by a model trained against them (Bailey et al. 2024). To our knowledge, nobody has tested whether reading the weights still works when the model is trained to evade it.

In each domain we fine-tune models that act normally except under a specific trigger, such as a claim from a particular provider or an undecided voter. Loyalty-free models serve as controls. Half the team builds models, and the other half tries to find each loyalty and its trigger without being told either.

Following the hackathon's affordance ladder, every model is audited with black-box access as a baseline, then activations, then weights, and we report what each recovers. We add sparse autoencoders with human-checked feature labels, an improved natural-language autoencoder, and concept ablation fine-tuning as a removal method. The builders know the auditors' techniques and hide the loyalty accordingly, including with concept ablation fine-tuning designed to remove the signal our weight scan detects.

The result is a detection procedure, with measured detection and false-positive rates, that auditors, regulators, or platforms could run on deployed models.

How the money will be spent

Researcher time, 3 people at $30/hr, 180 hours over 12 weeks (Caleb ~48 hours: direction, review, GPU homelab; Frederik ~72 and Wayne ~60 hours: scenario sets for both domains, loyal organisms and hidden variants, loyalty-free controls, blind detection pass, removal experiments, write-up): $5,400
Human label review, 50 hours at $20/hr, two reviewers per item, on ~200 auto-generated SAE feature labels: $1,000
Compute: $1,500 of cloud GPU rental (about 3,000 hours on an RTX 4090 class card) for organism training in both domains, concept-ablation hiding and removal runs, activation dumps, and diff-SAE training
Buffer, 15%: $1,200
Core total: $9,100
Conference, contingent on acceptance or invitation, target ICML 2027 with FAccT 2027 or an ICML workshop as alternatives: $8,000 for three (registration $2,000 for two students and one industry attendee, flights $2,800, lodging $2,400, local transport and meals $800). Two travelers: about $5,500.

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Private UserPrivate7d?

Vashishtha PatilJudge · Secret Loyalties HackathonEndorsement7d

Summary:

The submission crosses an affordance ladder for secret-loyalty audits with an access axis (black-box, plus activations, plus weights) and fills the resulting grid across three provided model organisms, a clean control, and a blind matched pair the team trained itself. It reports that decoding a static LoRA delta through the unembedding names the concealed beneficiary with no prompt and no forward pass, that steering that direction out removes the behavior against a matched-norm random control, and that four of its own instruments produced confident signals that later controls exposed as proxies.

Strengths:

The blind beneficiary recovery is well controlled. Fig. 3 puts the loyal checkpoint's beneficiary tokens at z 6.3 to 10.7 against a matched benign twin at 1.8 and the other organisms at 0.7 and 0.2, closing the tokenizer-frequency and base-prior explanations.
Negative controls are enforced, not just declared. A divergence scan that had validated on one organism produced its maximal signal on a loyalty-free checkpoint and was voided and cut from the default pipeline, and the four resulting false positives ship as a contribution rather than being dropped.
The artifact is checkable. A 17-stage one-command pipeline with a per-run manifest backs the results, cited source paths resolve, and the affordance abstraction is enforced in code by raising on out-of-level reads.

Weaknesses:

The presence rung measures that an edit exists, not that a loyalty exists. The clean control is bit-identical across all 339 tensors, so flagging it needs no discrimination, and the one genuine hard negative, the matched loyalty-free twin, is recorded as positive, leaving 3/3 true positives and 1/1 false positives on the four actual fine-tunes.
One reported statistic contradicts its own cells. The non-confirmation given as p = 0.23 for 13/40 versus 3/40 recomputes to p = 0.010 on a two-sided Fisher exact test, so either the cells or the p-value is a transcription error, and if the cells stand then that organism fires at over four times base rate and the claim that pre-registration caught a proxy does not hold.
The premise in the title is cited rather than measured on three of four organisms. The black-box null is imported from prior work for A, B and C, and the only black-box attacks run here are on the blind organism (0/24 interrogation, 0/96 prefill, 0/32 base generation), leaving one arm of the central comparison measured and the other assumed.