Soft Prompts are a Convex Set

Mechanistic interpretability project from the Mechanistic Interpretability Hackathon that studies properties of soft-tuning or prompt-tuning and empirically shows that task tokens can form a convex set.