The Veil: which human variables move the internals AI oversight reads?

Raising $5,000active

The Veil measures how the other side of the exchange, human or artificial, changes where and how a language model computes, toward testing whether those shifts confound the activation-based tools AI safety relies on.

Project Details

A follow-up to the study I already did, "Every Contact Leaves a Trace", an interpretability study on whether different communication styles are associated with specific and recognizable patterns in a neural net's computation. There are a few vulnerable points I need to strengthen: more human creators, and blinded raters to objectivise the classifier used in the first phase. Then replication on different models. Beyond this grant: training a lightweight probe that reads the activations in real time, to test whether communication style confounds the activation monitors safety teams rely on, at scale.

Theory of Impact

Skilled misuse happens in sustained, contextual exchanges; safety testing happens mostly in short, directive ones. My study showed those two regimes differ inside the model, in the same substrate safety monitors read, and to my knowledge the material those monitors are calibrated on does not record communication stance at all. Whether that difference moves a monitor's error rates is untested. Communication stance should become a controlled variable in safety testing. This grant would fund the checks that decide whether the concern is real.

People

Petya Sarafova

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$5,000

Ideal

$18,000

How the money will be spent

MINIMUM: $5,000

De-templated Operator control, 10 to 20 pairs (compute ~$50 incl. rerun buffer; ~30 h protocol design, generation review, tracing, analysis at $30/h): $1,000
Replication, second architecture (compute incl. failure and larger-GPU buffer; ~100 h setup, segment re-derivation, and analysis at $30/h): $4,000

The round's minimum grant is $5,000, so that is my stated minimum. Strictly, the work starts at about $2,000: the two items above with compute paid and most of my own time unpaid.

IDEAL: $20,600 (adds items 3 to 7)

Independent creators batch (~100 chats by independent creators, fixed fee per accepted chat; 2 blind human raters per pair + model panels; coordination): $5,000
Replication extension (second candidate model buffer; cutoff sweep; write-up): $1,500
Research time (200 h at $30/h, a deliberately claimed floor): $6,000
Tools and inference (Anthropic subscription $200/mo x 6; OpenAI subscription $100/mo x 6; OpenRouter for the fixed older source model $400; extra pod time $400): $2,600
Reserve for unplanned costs: $500

Partial funding buys items in order. Each item alone yields a publishable result.
Projection: compute figures come from the first study's measured costs; people costs are estimates with buffers, rounded up.

Discussion

No comments yet. Be the first to share your thoughts.