Creating a better way to compile to transformer architectures

what do you want to do?

The far goal would be to automatically decompile arbitrary transformers to make them
interpretable. One way of approaching this would be to create a language with an
attached compiler to more easily create hardcoded, interpretable transformers, analogous
to RASP, but more general. RASP/tracr and its descendents can't handle complex input
embeddings, which makes writing a RASP transformer that internally mimics some other
transformer I want to analyse impossible. I've had success with manually setting weights
to produce the desired behaviour, and I'd like to see to what scale of model I can extend
this and whether I can formalise it into a compilable language.
Specific Steps:

- take a model and analyse it to understand (some part of) its internal behaviour

- manually set the weights of a transformer that uses this behaviour as part of its
algorithm and has low loss on the original model's task

- repeat previous steps as necessary, then generalise from experience to find a better
way of compiling code to transformer architecture

how are you planning to spend the money?

1,000$ - token credits/compute (4 months)

2,000$ - food (4 months)

3,000$ - rent (4 months)

4,000$ - any other expenses/taxes (4 months)

what have you done in the past that proves you will be good at doing this? focus on substance, not credentials

- take a model and analyse it to understand (some part of) its internal behaviour

In my last major project, I analysed gpt2-small's layer 0 to understand what it's doing:
https://www.lesswrong.com/posts/dcvrja6kyshqWX4zZ
I don't claim to fully understand GPT2's embedding, but I've spent a lot of time thinking
about transformer internals and have been successful with the toy models I've looked at
since.

- manually set the weights of a transformer that uses this behaviour as part of its
algorithm and has low loss on the original model's task

I've succeeded with every small-ish model I've tried this for, without using llms or any
kind of gradient descent. Here are 2 Examples/Challenges I've shared with others:
https://colab.research.google.com/drive/1-P2YnvGy7qPO1ugH3Ha7QhOiIwBKAUJx?usp=sharing

https://colab.research.google.com/drive/1j3xCYqam8H3F0R1aecuW53jmidO_wlxG?usp=sharing

- repeat previous steps as necessary, then generalise from experience to find a better
way of compiling code to transformer architecture

No previous work in this direction, not sure if I'll be good at this