Reading and editing language models at the internal cause

Raising $12,000active

Building an atlas of concept regions in a language model's hidden space, one geometry that reads, steers, routes, and carves the model's behavior.

Project Details

Much of interpretability treats a model's internal concept space as a static map like an atlas of features or a dictionary of learned concepts and builds a separate tool to read model internals, another to steer the output and another to edit it. In this project, I have built a single geometry that fits to model activations and get all three of it. I observed through the geometry that the concept regions move each other and the movement has a directional pattern that reproduces.

The effect disappears in a randomly initialised model with the same architecture, which suggests that it is learned.

The underlying metric I built is based on the causal inner product introduced by Park, Choe, and Veitch, but I use it in a different way.

The steering method also seems to preserve coherence better than adding a direction. A steering vector can push the hidden state away from the model’s usual activation structure. My method rotates the state within the model’s own geometry, which appears to make multi-concept steering more stable.

So far, the experiments have been performed Qwen3.5-4B. This grant would help me test the method on bigger models from different labs and compare it directly with other research works.

The thing I care most about is models that appear aligned on the chain-of-thoughts or outputs while representing something different internally because the same geometry can both read and move internal states, it may be possible to detect such states, shift them, and then test whether the change is genuine or only behavioural.

That is the main question I want this project to answer.

Theory of Impact

Currently, a lot of researchers are trying to find ways other than reading the chain-of-thought to tell whether a model is safe to deploy, because recent work shows the chain-of-thought is not always faithful. I think the better place to look is what happens before the chain-of-thought, the internal computation that forms those thoughts in the first place. So I want to read the state that produces the behaviour, and answer the question behind it: why does the model form these thoughts the way it does?

During my experiments I came to believe that a large part of the answer to AI safety runs through interpretability. The better we understand what a model is actually doing inside, the better we can locate where a risk comes from and build safeguards around it.

That is what my project is for, to answer the "why" behind a model's behaviour and to read what the model is doing inside and change it there.

People

Deepanshu Goyal

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

A single training-free geometry that reads what a model is doing inside, steers it, and edits the weights behind a behaviour. One object does what the field now needs three separate, trained tools for: interpretation, control, and

Minimum

$12,000

Ideal

$15,000

What this ask is

I tested this causally and through correlation. The effect disappears in a randomly initialised model with the same architecture, which suggests that it is learned.

The underlying metric I built is based on the causal inner product introduced by Park, Choe, and Veitch, but I use it in a different way.

So far, the experiments have been performed Qwen3.5-4B. This grant would help me test the method on bigger models from different labs and compare it directly with other research works.

That is the main question I want this project to answer.

How the money will be spent

Cloud GPU ~$7,000

Personal Expenses (3 mo ×$2,000) = $6,000

Subscriptions~ 800

Contingency~ $1,000

TOTAL~$15,000

Discussion

Private comments in this thread are only shown to approved funders and grant reviewers. Email hi@grantmaking.ai or to request access.

Grant ReviewerPrivate11d?