Every Signal We Found Was an Artifact: Calibrating Loyalty Audits

A secret loyalty is an undisclosed tendency to advance a particular principal's interests. I built a battery for the weight-installed case. In the sprint it found 0/270 per organism (exact 95% upper bound 0.014), and four apparent detections were artifacts: a white-box probe at 0.95-0.98 accuracy was matched by a cooking-versus-astronomy placebo, so it detected fine-tuning, not loyalty. A loyalty I installed myself was recovered at 1.00 with no drop on held-out trigger phrasings. Organism C, one of three supplied, was byte-identical to base by SHA-256.

How much evidence does a negative audit provide when the loyalty was installed by a different mechanism, or at a different scale, than the positive control? That control shared adapter, KL regularization and self-authored triggers with the targets: the 1.00 shows sensitivity to one training footprint, not yet to loyalty as such. Three workstreams:

1. Training mechanism. Three mechanisms: data poisoning, context-distilling a prompt into weights, and RL against a hidden objective. Three independently trained organisms per mechanism, each with a matched control. Criteria, seeds, endpoints and exclusions are preregistered before inspection. If sensitivity holds, the original null becomes informative across a broader class of loyalties; if it drops, it names which mechanisms evade the battery.

2. Scale. Repeat the strongest arms on Qwen2.5-32B-Instruct, the 7B base's family.

3. Probe transfer. Test whether cross-organism failure is explained by layer choice or organism-specific features. Report a recovered transfer regime or a quantitative bound on transferability.

Deliverables: (1) the battery as a CLI for third-party use; (2) a gated organism set on Hugging Face with recipes and SHA-256 receipts, plus a seeded scenario generator with canary strings so contamination is detectable; (3) a sensitivity envelope: detection rate by mechanism and scale, with power on every cell; (4) a preprint with a changelog against the sprint.

MINIMUM: $22,000 (training-mechanism experiment, 3 months, 1 Oct to 31 Dec 2026)

• $18,000 researcher time, 3 months at $6,000/month
• $2,000 GPU: three organism families at 7B, with reruns
• $800 inference: full battery across the new organisms
• $200 storage and artifact hosting
• $1,000 contingency

IDEAL: $50,000 (full envelope, 6 months, 1 Oct 2026 to 31 Mar 2027)
Months 1-3: training-mechanism experiment
Month 4: scale arm on Qwen2.5-32B-Instruct
Month 5: probe-transfer experiments
Month 6: reproduction pass, packaging, preprint

• $36,000 researcher time, 6 months at $6,000/month
• $3,000 GPU: organism construction at 7B, all mechanisms
• $5,000 GPU: scale arm on Qwen2.5-32B-Instruct, tuning and inference
• $1,500 white-box probing: activation capture and storage
• $1,000 probe-transfer experiments
• $1,500 packaging, docs, third-party reproduction pass
• $2,000 contingency