Canary: Evaluating Frontier AI

Canary is a multi‑year collaboration between METR and RAND, selected as part of The Audacious Project’s 2024 cohort, that aims to create a “canary in the coal mine” for dangerous AI capabilities. The project funds METR and RAND to design, validate, and operationalize evaluations that can detect when frontier AI models acquire high‑risk capabilities, such as enabling cyber or biological misuse or supporting runaway autonomy, so that developers and governments can respond before deployment. Canary’s work includes building science‑based tools and benchmarks, running evaluations on cutting‑edge models, and helping integrate these assessments into safety policies and regulatory frameworks for frontier AI.