Understand Multimodal Models and Detecting Misalignment

Since 2025, I have been working on understanding multimodal models by running experiments and looking at how to detect misalignment. Misalignment in AI refers to artificial intelligence systems that fail to align with human values, goals, or intentions. My experiments have shown some results.

What are these results? One experiment on the COCO dataset (Common Object in Context dataset by Microsoft) reveals critical gaps between true semantic understanding and benchmarking. True semantic understanding refers to a genuine grasp of meaning, and benchmarking means comparing against a standard. It shows 0.216 average cosine similarity (a measure between two vectors) for matching text-image pairs versus near-zero for mismatching ones.

Besides, another experiment involves a compositional copying (giving a model a text cue and seeing what visual feature it copies) task using the Visual Genome (a dataset containing relationships between objects and scenes) dataset. It shows confident predictions on brittle features unlikely to survive distribution shifts (data that shift away from the training data) in the real world. an 86.8% probability score (on a scale of 10, its confidence level is roughly 9). However, it relies on tiny embedding (numerical vector representing objects) differences (0.019 cosine similarity and 0.074 patch-level probability).

I have built an open-source suite for the community to verify and extend my research.

What I will continue doing is:

·       Designing experiments to understand multimodal models using various techniques, including mechanistic interpretability and representation engineering

·       Improving on the open-source suite to help the community verify and extend what I have done

·       Building tools to detect misalignment, including undesirable behaviours, for current and future models

Who is involved: I will take the lead in completing the technical work and sharing it. I will hire 1 to 4 AI safety researchers to work with me, if possible. I might need to find the right kinds of collaborators later. The entire community will have the opportunity to get involved by sharing and giving constructive feedback

The concrete output is an improved open-source suite published on GitHub for the technically minded community, communication with this group of audiences, and perhaps papers or reports down the line.