AI Evaluator Forum

The AI Evaluator Forum (AEF) is a network of “leading independent AI research organizations” that collaborates to strengthen the practice and impact of third-party AI evaluation, with an explicit emphasis on evaluations that serve the public interest. AEF positions independent evaluation as essential for answering high-stakes questions about AI systems’ “capabilities, limitations, and risks,” arguing that without “rigorous, unbiased assessment by organizations free from conflicts of interest,” society cannot be confident that AI systems are safe or that the public interest is protected.

AEF’s mission is to “advance the quality, credibility, and impact of independent AI evaluations,” and it pursues this through several mutually reinforcing activities: (1) establishing standards and best practices for credible third-party evaluations; (2) fostering collaboration and knowledge-sharing among evaluator organizations; and (3) advancing public understanding of AI capabilities and risks through rigorous research and peer review. The Forum also frames its collaborative model as collective action that respects the independence and distinct expertise of each member organization.

A core initiative is AEF-1, a voluntary standard titled “Minimum Operating Conditions for Independent Third Party AI Evaluations,” developed to address the problem that evaluation operating conditions can be “opaque” and can strongly shape whether an evaluation is “trustworthy and impartial.” AEF-1 is intended for evaluations that aim to provide “genuinely independent and trustworthy assessment” of an AI system’s capabilities or risks, and it specifies baseline operating conditions across independence, access, and transparency. The standard is organized into five core principles—“Sufficient Access and Resources,” “Minimized Conflicts of Interest,” “Analytic Autonomy,” “Transparent Methods and Results,” and “Protection of Sensitive Information”—and it includes a checklist that evaluators can attach alongside results to demonstrate adherence and document any deviations.

AEF was publicly launched on December 4, 2025, described as “a network of independent organizations assessing AI capabilities and risks in the public interest.” Its launch materials state that the Forum is intended to “enable collaboration among its members to develop common best practices and standards, foster collaborative research, and facilitate engagements with industry, governments, and civil society.” Alongside the announcement, the Forum released (1) a public statement calling for greater transparency about third-party evaluations, including disclosure of independence/access conditions and potential conflicts of interest, and (2) the AEF-1 standard, which members “will begin adopting in relevant forthcoming evaluations.” The founding members listed include Transluce, METR, the RAND Corporation, the Holistic Agent Leaderboard at Princeton University, SecureBio, the Collective Intelligence Project, Meridian Labs, and the AI Verification and Evaluation Research Institute, collectively spanning evaluation work across “a broad range of AI risks, from biosecurity to self harm.”