OpenAI signs deals for outside assessments of AI safety
Four assessors will examine models and safeguards with internal access, OpenAI says; participating teams describe pilots with different scopes and durations.
Numbers in brackets link to the sources listed below.

OpenAI said on 16 November that it had signed agreements with Apollo Research, METR, Transluce and Goodfire to assess its AI models and safeguards through work inside the company[1OpenAIOur first phase of embedded assessmentsOpen sourceSources [1]]. The initial assessments cover training and internal use, it said[1OpenAIOur first phase of embedded assessmentsOpen sourceSources [1]].
OpenAI describes these embedded assessments as extended examinations giving outside experts access to internal systems and models[1OpenAIOur first phase of embedded assessmentsOpen sourceSources [1]]. Depending on the assessment, available resources include code, data used for training and evaluations, monitoring tools and conversations with staff responsible for the work[1OpenAIOur first phase of embedded assessmentsOpen sourceSources [1]].
Apollo Research says six evaluators will spend four months examining models during training, evaluation and internal deployment, with defined access rules[2Apollo ResearchA Pilot Towards Embedded EvaluationsOpen sourceSources [2]]. It aims to investigate whether models try to undermine controls governing themselves or successor models, and plans public reports explaining its limitations[2Apollo ResearchA Pilot Towards Embedded EvaluationsOpen sourceSources [2]].
Goodfire says its researchers will use its platform to track training and investigate unwanted behavior[4GoodfireAnnouncing our collaboration with OpenAIOpen sourceSources [4]]. OpenAI researchers will be the main audience for its findings, although Goodfire also plans to publish findings as work proceeds[4GoodfireAnnouncing our collaboration with OpenAIOpen sourceSources [4]].
Transluce says its six-week pilot will examine whether OpenAI's evaluations detect behaviors including deception and manipulation[3TransluceAnnouncing an Embedded Assessment Pilot with OpenAIOpen sourceSources [3]]. It says access will exceed what outside evaluators generally receive but fall short of employee access[3TransluceAnnouncing an Embedded Assessment Pilot with OpenAIOpen sourceSources [3]]. The organisations intend a fuller assessment with broader access in early 2027, with details still undecided[3TransluceAnnouncing an Embedded Assessment Pilot with OpenAIOpen sourceSources [3]].
Sources
- 1Our first phase of embedded assessments
PrimaryOpenAIAccessed
- 2A Pilot Towards Embedded Evaluations
IndependentApollo ResearchAccessed
- 3Announcing an Embedded Assessment Pilot with OpenAI
IndependentTransluceAccessed
- 4Announcing our collaboration with OpenAI
IndependentGoodfireAccessed
ITANU’s newsroom system produced this article with AI and checked it against the sources listed above. How we work