Software Verification: AI Proves Code Safety

For most software users, the daily experience of using apps and websites has not changed, but the way developers ensure these programs work correctly is undergoing a significant shift. In a recent development, Anthropic announced that its AI model, Claude, can now mathematically verify the safety of existing code. This means that instead of relying solely on traditional testing, which might miss rare errors, the AI can create formal proofs that the code behaves as expected under specific conditions. For ordinary people, this could eventually lead to more reliable software with fewer unexpected crashes or security flaws.

The process involves a collaboration between Anthropic and a partner company called Theorem. The team used Claude to analyze code from 123 open-source libraries that are part of Google's OSS-Fuzz security project. Over the course of a few weeks, the model generated approximately 180 million lines of mathematical proof code. This was not a task of writing new software, but rather of verifying that existing software is correct. The AI translated crash tests into mathematical problems, analyzed the compiled machine code, and produced detailed proofs that demonstrate the code's integrity.

Formal verification has historically been a costly and time-consuming process reserved for critical systems like aircraft avionics. In the past, a team of specialists spent 20 person-years just to verify a small operating system kernel known as seL4. By automating this with Claude, the time required has been drastically reduced. An independent verification program called Rocq automatically checks the proofs generated by Claude. According to Anthropic, this process requires almost no human reviewers, allowing the system to operate as what researchers call fully autonomous proof generation, although the underlying tools and foundations were built by humans at Theorem.

One of the most notable outcomes of this project was the discovery of numerous bugs that had gone undetected for years. While generating proofs, Claude identified hundreds of errors and was allowed to fix them itself. Many of these bugs only appeared under very specific conditions, such as unusually large input files or specific sequences of data access, which is why conventional testing often missed them. In one specific case, the team found as many security bugs in a widely used PDF reader in one week as are typically reported for that software in an entire year. In another instance, the AI caught a logical error that had remained hidden for more than a decade.

Despite these successes, there are important limitations to this technology. Anthropic clarifies that formally verified does not mean completely error-free. The proofs are only valid under strict conditions, including a specific processor architecture, single-threaded programs, and input files smaller than 64 MB. In real-world scenarios, these libraries often run on different hardware and configurations. Additionally, the computational cost is high, with individual proofs requiring hundreds of millions of tokens to process. However, Anthropic notes that proofs are reusable, and newer models are becoming more efficient. For example, Opus 5.5 solves half the proofs with about 40 percent fewer tokens than its predecessor, Opus 5.

The broader implication of this project is a potential leap in software automation. If AI can write code and then prove its correctness under defined conditions, developers can trust AI-generated software more, reducing the time and cost spent on manual review. This shift changes the focus from asking if AI can program to asking if it can prove that the code it writes is safe. Anthropic believes that formal verification using AI will soon become as routine and practical as standard testing. Theorem's roadmap already targets critical infrastructure libraries such as glibc, OpenSSL, and cURL, suggesting that this technology is moving toward mainstream application in the tech industry.

This development marks a new chapter in how software reliability is established. By combining the generative power of large language models with the rigor of formal methods, the industry is moving toward a future where software safety is not just tested, but mathematically guaranteed within specific parameters. As costs decrease and efficiency improves, more developers are expected to adopt this approach, potentially transforming the entire lifecycle of software creation and maintenance.