AI-Generated Patches Fail Half the Time
Dark Reading, Friday, August 7th, 2026
A study of 6,080 AI-generated patches found only 46% actually fixed the underlying vulnerability.
1Password's Off-By-1 research team generated 6,080 patches for six recently disclosed vulnerabilities using ChatGPT-5.5 and Anthropic's Opus 4.8, and found only 46% resolved the underlying flaw.
Just 26% fixed the vulnerability without altering application behavior, 20% fixed it while changing how the code works, and 49% failed outright.
Similar work by Veracode found a 56% average security pass rate across 100+ models, with 44% of generated code introducing OWASP Top 10 vulnerabilities.
Researchers stress that AI-generated patches must be treated as proposed changes requiring regression testing, human review, and whole-program static analysis rather than verified fixes.