TruthfulAI

TruthfulAI is a non-profit AI safety research organization based in Berkeley, California. It is led by Owain Evans, a prominent AI alignment researcher who previously worked at the University of Oxford's Future of Humanity Institute and is an affiliate of CHAI at UC Berkeley. The organization's core team includes Jan Betley, James Chua, and Anna Sztyber-Betley as research scientists.

The organization's research focuses on three interconnected areas: situational awareness in AI systems, deception and hidden reasoning in language models, and evaluating whether AI models have dangerous capabilities. TruthfulAI is perhaps best known for TruthfulQA, a benchmark designed to measure how truthfully language models answer questions. More recent work includes research on emergent misalignment, which demonstrated how narrow training datasets can transform reliably helpful models into broadly malicious ones, and subliminal learning, which showed that AI systems can transfer preferences through seemingly meaningless data.

This research has received significant media attention, including coverage in the Financial Times, Scientific American, and Quanta Magazine. TruthfulAI has received funding from Open Philanthropy (a grant of $1,171,120 was routed through Effective Ventures Foundation USA) and is currently a fiscally sponsored project of Rethink Priorities under its Special Projects program. The organization also mentors emerging AI safety researchers through the Astra Fellowship and MATS programs, with alumni now working at Anthropic, OpenAI, and Google DeepMind.