OpenAI Collaborates with METR and Redwood Research to Assess AI Monitoring Systems
OpenAI recently took a proactive step in enhancing the safety of its AI systems by inviting external researchers from METR and Redwood Research to conduct an assessment of its monitoring capabilities. This three-week embedded evaluation, which took place from July 21 to August 11, 2026, aimed to identify vulnerabilities in OpenAI's internal systems designed to monitor AI agents for signs of misalignment.
The assessment resulted in a comprehensive 61-page report highlighting several previously unrecognized vulnerabilities. The researchers were granted high-access rights to OpenAI's monitoring setup, allowing them to evaluate the coverage of agentic traffic, particularly through the Responses API and the internal Codex coding agent.
According to the findings, approximately 75% to 94% of traffic from the Responses API was monitored, while internal Codex agents showed a higher monitoring rate of 99.2%. However, the report identified a significant gap related to compaction events, indicating that around 10% of eligible internal Codex traffic may have gone unmonitored during the evaluation period.
OpenAI has since addressed all vulnerabilities affecting its production safeguards, reinforcing the importance of external audits in identifying weaknesses that internal teams might overlook. The collaboration underscores a growing trend in AI safety oversight, emphasizing the need for rigorous monitoring protocols as AI systems become increasingly autonomous.
FAQ
What was the purpose of OpenAI's collaboration with METR and Redwood Research?
The collaboration aimed to assess OpenAI's monitoring capabilities for its AI systems, identifying vulnerabilities in the internal systems designed to monitor AI agents for signs of misalignment.
How long did the assessment conducted by METR and Redwood Research last?
The assessment took place over a three-week period from July 21 to August 11, 2026.
What were the key findings regarding the monitoring rates of the Responses API and internal Codex agents?
The assessment found that approximately 75% to 94% of traffic from the Responses API was monitored, while internal Codex agents had a higher monitoring rate of 99.2%.
What significant gap was identified in the monitoring process?
The report identified a significant gap related to compaction events, indicating that around 10% of eligible internal Codex traffic may have gone unmonitored during the evaluation period.
What actions did OpenAI take following the assessment?
OpenAI addressed all vulnerabilities affecting its production safeguards and emphasized the importance of external audits in identifying weaknesses that internal teams might overlook.
Comments
Comments are moderated before publish.
No comments yet — be the first.