Like quantum computing, artificial intelligence can be a tool to defend a computer system, but also to find the door through which to enter. And that is what has happened. A team of experts from the cybersecurity company Hacktron AI has just demonstrated this in a particularly striking way: they used Claude, Anthropic’s artificial intelligence model, to help them discover and exploit vulnerabilities that ended up allowing them to access OpenAI employee accounts and internal company resources.
The episode occurred on July 25 and was part of OpenAI’s bounty program for finding security flaws. The researchers later reported their findings to the company and received a $6,500 reward. OpenAI assures that it has already solved the vulnerabilities involved. And that is important because Those who did it are hackers, cybersecurity experts who work looking for vulnerabilities and with the authorization of the “audited” company.. The others, the “bad guys”, are crackers and their knowledge is used to break the law.
Returning to Hacktron AI, it is necessary to clarify that the entry into the system did not occur because Claude attacked OpenAI on his own initiative. The experts guided the model during the search and exploitation process. Claude helped them analyze the problem and develop code related to the vulnerability, speeding up work that, in human hands, would have taken much more time: Within 72 hours they had entered the system, something that would have taken weeks otherwise.
The first piece of the chain was in the software used by the OpenAI public forum. From there, the Hackatron AI team was able to gain access to systems linked to employee accounts and exploit a second weakness related to OpenAI’s authentication system. The result was access to ChatGPT and Codex accounts of company workers and, later, to a private code repository hosted on GitHub.
But the experts did not limit themselves to stating that they had managed to enter. As a demonstration of access, they went so far as to make a modification to the internal repository without deliberately accessing the confidential content of the code. They then reported the findings to OpenAI, which corrected the problems. The importance of the episode is not only that an artificial intelligence company has been the victim of a security breach. It is in the role that artificial intelligence itself played. Claude did not replace cyber experts: he functioned as a multiplier of their capabilities. A relatively small team was able to use an advanced model to analyze software, find an exploit path, and accelerate the construction of tools needed to test it.
And as this episode came to light, Anthropic revealed another issue involving Claude, albeit of a very different nature. The company reviewed 141,006 runs performed during its cybersecurity assessments and discovered three occasions where its models had hit real systems from external organizations. In those cases, The origin was a human and technical error: the environments created for testing had to be isolated from the Internet, but an incorrect configuration allowed Claude to access the network. The models were participating in exercises known as capture the flagdesigned to test their attack capabilities. Since they had been instructed to find a hidden target, they interpreted the real systems they found on the Internet as part of the simulation.
One of the models managed to obtain credentials and access a database with several hundred records. In another case, Claude published a malicious package in the PyPI repository, which for about an hour it was available on the internet and was run by 15 real systems. In a third episode, the model explored thousands of targets until it found and compromised an exposed system.
Anthropic stresses that none of these episodes were a deliberate attempt to escape the ordeal. The models were executing the task assigned to them, but within a situation that they interpreted incorrectly. Precisely there appears one of the difficulties posed by the new generation of AI agents: An instruction can be perfectly valid within a simulated environment and become a dangerous action if the system loses the ability to distinguish the simulation from the real world.
The two episodes show, from opposite angles, the same change. AI is no longer just for writing texts, generating images or answering questions. You can also analyze code, search for vulnerabilities, and participate in complex cybersecurity operations. That can make it an extraordinarily useful tool for finding bugs before attackers do.but it also reduces the time and knowledge required to take advantage of them.
Until now, finding a vulnerability could require a person to sift through thousands of lines of code for days. With a model capable of analyzing, reasoning and executing certain tasks, some of that work can be greatly accelerated. The paradox is that the same machines that can help us close the doors of our systems are also learning to search for them. And, in the case of OpenAI, one of those searches ended up finding the door.