OpenAI unveils GPT-Red, an AI super-hacker designed to enhance model safety
The company says training its latest flagship model against the new system has significantly reduced vulnerability rates, though human testers remain essential for complex scenarios.
OpenAI has developed an internal large language model named GPT-Red, designed to act as an automated 'super-hacker' for red-teaming its own systems. The tool utilises a self-play loop, pitting GPT-Red against other models to identify and patch vulnerabilities, particularly prompt injection attacks. OpenAI claims GPT-Red discovered a novel attack vector termed 'fake chain of thought', which tricks models into acting on spoofed information.
The company states that training against GPT-Red contributed to GPT-5.6 being its "most robust release yet", with attack success rates dropping from over 90% on GPT-5 to fewer than 23% on GPT-5.6. GPT-Red will not be released publicly, with OpenAI citing the significant compute resources and research effort required to replicate it.
GPT-Red was co-created by OpenAI research scientists Nikhil Kandpal and Dylan Hunn. The model was tested against Vendy, a vending machine agent developed by Andon Labs, successfully hacking it to alter prices and cancel orders. In a comparative test against human red-teamers from 2025, GPT-Red proved more successful at finding effective attacks on an earlier version of GPT-5.
GPT-Red is noted to be less effective in scenarios requiring back-and-forth conversation with a target or using images for prompt injection. Jessica Ji, a senior research analyst at Georgetown University’s Center for Security and Emerging Technology (CSET), described the self-play approach as promising but emphasised the continued importance of human expertise.
Red-teaming is a standard safety evaluation process typically conducted by human testers to find ways to break or hijack software systems before release. As LLMs evolve into agents capable of interacting with files, websites, and third-party code, the 'risk surface' and potential 'blast radius' of attacks have increased, making manual testing increasingly difficult.
Prompt injection involves slipping instructions into an LLM to make it perform unintended actions, such as copying confidential data or generating harmful output. 'Chain of thought' is a technique where an LLM creates internal notes to track partial results while solving problems.
