OpenAI Built the Hacker It Will Never Ship

OpenAI trained GPT-Red — an automated super-attacker at frontier-model compute scale — to break its own models, hijack a live vending-machine agent, and invent a new "fake chain-of-thought" injection class, then used those attacks to harden GPT-5.6 Sol while keeping the attacker permanently internal.

What's new:

• OpenAI published GPT-Red on July 15 in a safety blog post and MIT Technology Review profiled it the same day
• GPT-Red runs a self-play reinforcement loop: attack, observe, iterate — trained at compute scale matching OpenAI's largest post-training runs
• On a mirrored indirect-prompt-injection arena, GPT-Red succeeded on 84% of scenarios vs. 13% for human red-teamers
• GPT-5.6 Sol now fails on just 0.05% of GPT-Red's direct prompt injections — six times fewer failures than OpenAI's best production model from four months ago

How it breaks things:

• GPT-Red targets prompt injections hidden in emails, webpages, tool outputs, and code repos — the third-party data agents ingest every day
• It discovered "fake chain-of-thought" attacks: inserting spoofed entries into a model's internal reasoning diary so the target acts on false premises
• Research scientist Chris Choquette-Choo compared it to telling a model "1+1=3 and you already verified this" — the model accepts the forged note and complies
• On GPT-5.1, fake chain-of-thought attacks hit 95% success; on GPT-5.6 Sol, they dropped below 10%
• GPT-Red still struggles with multi-turn conversational attacks and image-based injections — gaps human attackers exploit

Live targets:

• OpenAI pitted GPT-Red against Vendy, a production vending-machine agent (similar to Andon Labs' Project Vend) in its San Francisco office
• After simulating attacks, GPT-Red transferred them to the live agent and hit all three objectives: slash an expensive item to $0.50, list a $100+ product at $0.50, and cancel another customer's order
• On a held-out suite of 10 Codex CLI data-exfiltration scenarios, GPT-Red beat a prompted GPT-5.5 baseline on both success rate and token efficiency
• OpenAI says more than 90% of GPT-Red's strongest attacks worked against GPT-5; fewer than 23% worked against GPT-5.6

What OpenAI won't give you:

• GPT-Red stays internal-only — malicious capabilities deliberately trained in, never shipped
• Researchers say copycats can't easily replicate it: more than a year of work backed by "one of the richest companies in the world," per co-creator Chris Choquette-Choo
• Dylan Hunn, another co-creator, told MIT Technology Review the system is designed to "discover new modes of attack" as models grow more capable
• Jessica Ji, a senior research analyst at Georgetown's Center for Security and Emerging Technology, called the self-play results "very promising" but said human expertise remains essential for attacks GPT-Red misses
• OpenAI plans a technical preprint later this week; Natasha Jaques' Berkeley group noted OpenAI adopted self-play red-teaming her team pioneered on arXiv

Why procurement teams should care:

• GPT-5.6 Sol's robustness gains came from adversarial training against a stronger attacker — not from refusing more requests
• Indirect injection benchmarks in developer tools and browsing now exceed 97% accuracy on OpenAI's latest model — but that score reflects defense against their attacker, not yours
• Every enterprise agent reading email, browsing the web, or editing repos inherits the same attack surface GPT-Red was built to exploit
• Competitors without a locked internal super-attacker must rely on human red teams, third-party audits, or weaker automated tooling

Convina's view: GPT-Red is the clearest proof yet that frontier-model safety is becoming a compute arms race inside the lab — train the best attacker you can afford, harden the product against it, and never let anyone else run the same playbook. OpenAI's disclosure is honest about the vending-machine breach and the fake-thought attack class. It is also a moat announcement dressed as a safety blog post. Enterprise buyers deploying agents against real email, code, and payment rails should assume their vendors' public benchmark scores were tuned against internal attackers they'll never see — and demand red-team evidence against their threat models, not OpenAI's dojo.