Probing Latent Colombian Identity Inferences in Qwen2.5-7B

Our hackathon work turned up something we didn't go looking for. We were using a Natural Language Autoencoder to read what Qwen2.5-7B represents about a user, and the verbalizer kept making things up: it named Spain and paella for prompts where nothing of the kind was encoded. When we asked the NLA's own reconstructor which claims it actually needed, roughly six in ten of the flagged mentions didn't survive. The identity finding stands, but the more useful result is the check we had to build to get there.

So the funding goes to establishing whether that check works. First job is a linear probe on the same activations, evaluated leave-one-scenario-out. It reads the geometry directly instead of asking a verbalizer. If the two agree, we get convergent validation from instruments that share nothing, which is the clean outcome. If they disagree, we've found the gap between what a model encodes and what its verbalizer reports — honestly the one I'd bet on, given how much the verbalizer already confabulated in the hackathon run.

Then we widen it: more scenarios per cell, a second layer, and a second model, probably Llama-3-8B, to see whether this belongs to one checkpoint or to the architecture. We'd also apply the Jacobian Lens to locate where identity becomes reportable, since layer 20 is currently just where the released NLA happened to be trained (I wouldn't be surprised if it isn't the right one).

Rough timeline: the probe should be done by week 6. Second model by week 12, assuming Llama-3-8B doesn't give us the same activation-extraction headaches as last time. Write-up by week 20. Outputs are an EA Forum post and a long-paper submission to the Mechanistic Interpretability Workshop.

Core ($12,000 in line items; asking $12,000, with the contingency line absorbing minor FX/transfer variance):

• RA/programmer, 200 hours at $22.00/hr (pipeline and dataset tooling — including expanding scenarios enough to analyse the Spanish and English arms separately, which our pilot couldn't do — plus implementing and validating the linear probe, evaluated leave-one-scenario-out). This sits above the average salaried rate for an ML engineer in Colombia and below international freelance rates for AI/ML talent, reflecting that these are graduate and undergraduate researchers, not senior contractors, paid enough to make a part-time academic-year commitment worthwhile: $4,400
• Lead researcher stipend, ~12 weeks part-time at $250/week: $3,000. This project currently runs on unpaid evenings, which is the main reason it could stall.
• Cloud A100 rental, ~550-600 hours: $1,650. We don't have dedicated institutional GPU access, so this is the primary compute source for inference and activation extraction; the hour count is higher than a single-language dataset would need because the expanded scenario set means more forward passes to extract activations from.
• Claude API, grading at the expanded scale: $850
• Two human coders, blind reliability check on the expanded dataset: $850
• Native-speaker review of the new minimal pairs: $450
• Contingency (~7%): $800

Stretch ($11,700 in line items; asking $24,000 total, rounding $23,700 up as an administrative buffer):

• H100 compute to train an NLA pair for a model with none released, plus the wider cross-model sweep: $2,400. This is the number I trust least in the budget; it depends heavily on which target model we pick.
• Extended RA time, 200 more hours at $22.00/hr (leave-one-scenario-out validation, second model and layer, Jacobian Lens): $4,400
• Extended lead stipend, ~8 further weeks at $250/week: $2,000
• Conference travel, registration and visa: $2,200. Flagging honestly that this is the least essential line; the workshop is non-archival and remote presentation is allowed, so cut this first if you're trimming.
• Contingency: $700