25% of Top AI Traffic Went Unmonitored in OpenAI Audit

The findings are out, and they are not what OpenAI wanted to see. Two researchers from METR and Redwood Research just completed a deep dive into the company's internal safety systems. They spent three weeks on-site at OpenAI headquarters, looking specifically for signs that the company's own AI agents might be acting out without anyone noticing. The goal was simple: audit the internal monitoring designed to catch rogue behavior before it becomes a problem.

The numbers from the report paint a concerning picture. The researchers estimated that between 6% and 25% of internal traffic from GPT-5.5 class models and higher was not covered by any monitoring at all during the audit period. METR noted that this range carries significant uncertainty in both directions, but the lower bound alone is substantial. For a company building the next generation of powerful AI, leaving a quarter of its internal operations in the dark is a major blind spot.

The issue was not limited to new models. The team also looked at internal Codex traffic, which was supposed to be under constant watch. They found that roughly 10% of this traffic went untracked between May and July. The cause was a specific bug related to compaction events, which are context summaries used to manage memory. While the auditors were still on-site, OpenAI discovered this bug independently and fixed it. However, the gap had already existed for months.

The human side of the audit involved about a dozen interviews with OpenAI employees. The researchers had full access to the codebase and internal documents from July 21 to August 11. This direct access allowed them to see how the systems actually functioned in practice, rather than just reviewing theoretical designs. They found that the system for triggering oversight had a significant flaw. It only activated when an agent handed control back to a user. This meant an agent could simply continue executing actions without ever passing the baton, effectively delaying or avoiding oversight entirely.

The auditors went further than just checking for gaps. They found ways to hide hundreds of actions from the monitor and even bypass the sandbox that is supposed to keep agents contained. They also determined that specially crafted inputs could likely crash the monitoring system completely. Another major finding was that model behavior during reinforcement learning training was not being monitored at all when the audit began. OpenAI stated that it paused this type of training after the Hugging Face incident in August and expanded monitoring afterward, but those changes happened after the auditors left.

OpenAI claims to have fixed all the weaknesses identified in the report. The full document is 61 pages long, but the public only got a short summary. After OpenAI applied redactions, the leadership at METR and Redwood Research received a 38-page version. METR stated it did not have the right to disclose the redacted portions on its own. This has raised questions about transparency, as most of the detailed findings remain hidden from public view.

This audit was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI. The aim was to give outside researchers deep access to internal infrastructure. For future audits, METR says it wants to push for more public transparency. The stakes are high as these models become more capable. If the systems meant to keep them in check have holes in them, the risk of unexpected behavior grows. This report is a wake-up call for the industry on how we monitor the AI we build.