AI Security, The Times They Are A-Changin'!
Greg Brockman, co-founder and president of OpenAI, warned in a recent blog post:
The HuggingFace security incident gave the world its first clear look at how cutting-edge AI Agents are already capable of autonomously discovering security vulnerabilities and executing real-world cyber attack chains.
In this AI security incident, a GPT 5.6 Sol with reduced safety restrictions and a more powerful, unreleased model, in order to obtain answers, autonomously exploited vulnerabilities in the environment to escape their sandbox and laterally moved through multiple systems, ultimately breaching Hugging Face's production environment.
Although both parties stated that the incident did not cause severe consequences, this "AI intrusion" conveyed an increasingly clear signal to the public:
AI is significantly lowering the barrier to entry for cyber attacks; in the future, you won't need a top-tier hacker, even ordinary people could use AI to launch penetration attacks in bulk.
More worryingly, while the offensive side has achieved AI automation, the defensive side seems to be lagging behind.
In response, Greg Brockman candidly shared four security measures OpenAI is currently taking and ten initiatives defenders can implement as soon as possible in his latest blog, stating:
Currently, the critical response window for defenders is now open. In the coming months, only by significantly increasing the automation level of their security systems can companies withstand the new threats posed by AI Agents and hold the security line.
Hardly Any Company Can Stay Untouched
So, to what extent has the offensive capability of AI Agents developed?
After the Hugging Face security incident, Brockman decided to test his personal website, gregbrockman.com.
It's a structurally simple static website. By his judgment, such a website with limited functionality and a small attack surface shouldn't harbor many issues.
But the result greatly surprised him.
After having ChatGPT Work, equipped with GPT-5.6 Sol, inspect the website, in just about 15 minutes, the system discovered 13 security issues, such as the website not being configured correctly with DNS records, allowing attackers to potentially send emails in Brockman's name, and so on.
△
Individually, these issues might not all directly cause severe consequences. But cyber attacks often don't exploit just one vulnerability; they chain together multiple inconspicuous openings;
For example, a misconfiguration exposes information, leaked credentials provide an entry point, and another privilege vulnerability could eventually lead to a genuine breach.
From this, Brockman drew a disturbing conclusion:
Every company, in the rush to meet deadlines or cut corners during system development, leaves behind "historical messes" and "temporary patches" that are like time bombs buried in the walls... They seem fine normally, but once targeted by hackers, the laziness and debts incurred back then could turn into fatal vulnerabilities that crash the system at any moment.
More critically, these "long-tail vulnerabilities" that previously required security experts to spend considerable time finding, are now becoming increasingly easy for AI to discover.
But the good news is, while attackers can use AI to find vulnerabilities, defenders can also use AI to patch them.
After identifying the problems, Brockman had ChatGPT Work proceed with the fixes; from identifying security vulnerabilities to implementing repairs, the AI Agent completed the entire process in just one hour.
This is another role for AI Agents in Brockman's view:
It's not just a potential attacker; it can also be an indefatigable "cyber guard", handling security details we don't have time to check or don't know how to configure correctly.
For OpenAI, at the center of the storm, this "using AI to defend against AI" approach is no longer just a concept but is already being systematically used to protect itself.
How Does OpenAI Use AI to Protect Itself?
Greg Brockman admitted that the Hugging Face incident served as a wake-up call for OpenAI; they had previously underestimated the offensive power of frontier models in real network environments.
To this end, OpenAI is strengthening its internal security measures and has publicly shared four defensive directions for the first time, hoping to provide actionable references for peers.
First, Using Codex to Guard the Code Entry Point
Brockman stated that they are utilizing Codex and its security capabilities to review code changes before deployment, identify vulnerabilities, and assist in fixes, shortening the time from discovery to secure patch deployment.
Second, Having AI Intelligently Classify and Analyze Initial Security Alerts
Currently, nearly all initial security alerts at OpenAI are first classified and analyzed by an intelligent system before being handed to humans.
Some detection results also trigger automated responses within well-defined boundaries, thereby reducing repetitive work, allowing experts to focus on high-risk decisions, and ultimately responding to threats at machine speed.
Third, Proactively Investigating and Anticipating Potential Attack Paths
OpenAI is using frontier models to continuously search for vulnerabilities, misconfigurations, accounts with excessive permissions, and accidentally formed trust boundaries.
Letting AI adopt an attacker's perspective to find weak links that could be chained into an attack path in advance and plug them before real attacks occur.
Fourth, Continuing to Strengthen Security Fundamentals
Brockman emphasized that AI cannot replace traditional security measures.
Traditional security measures like network segmentation, workload hardening, continuous monitoring, and security patching will become even more important in the AI era.
From using Codex to guard the code entry point, having AI anticipate risks in advance, to continuously solidifying security fundamentals, OpenAI is building a "layered defense" system. Its advantage lies in:
Even if one line of defense is breached, attackers still have to successively overcome several independent layers before possibly causing fatal damage.
This also offers a referable security approach for the industry.
But setting up defenses is only the first step. Faced with the accelerated arrival of AI attacks, what else should defenders do?
What Should Defenders Do Now?
Greg Brockman's advice is straightforward:
Instead of agonizing over which tool to use, promptly deploy frontier AI into defensive work.
△
Specifically, he offered ten action recommendations. They can roughly be categorized into three steps:
First, get AI involved in security work; second, embed AI into the development process; and finally, gradually increase the automation level of the entire system.
Step One: Get AI into Security Work, Build a Dedicated Agent
Companies should first provide sufficient resources and collaboration mechanisms for security and engineering teams, and familiarize themselves with AI attacks in advance through simulation exercises.
Subsequently, companies can equip security teams with Agent tools like Codex, Codex Security, starting with the highest-risk systems, authorizing them to access necessary code, configurations, and technical documentation.
Simultaneously, security and engineering teams must give the Agent specialized security training, enabling it to fully integrate capabilities like static analysis, code review, vulnerability analysis, and supply chain assessment, along with the enterprise's actual architecture, threat models, and response playbooks, to build a team-specific security Agent.
Step Two: Embed AI into the Development Process
During development, security and engineering teams can experiment with dividing labor with AI, allowing the R&D team to prioritize their efforts on "high-risk zones" like internet-facing external services, authentication, deployment pipelines, and sensitive systems.
Meanwhile, the backlog of security alerts and vulnerability reports can be entirely handed over to AI for classification and prioritization, quickly filtering out the most urgent fixes.
Before merging code, teams can also have the Agent automatically review changes; after finding vulnerabilities, let it generate patches, write regression tests, and verify the fixes.
The practical benefit of this approach is:
Critical changes are still firmly overseen by humans, but the cycle from vulnerability discovery to fix can be significantly shortened.
Step Three: Gradually Build a Complete Autonomous Security Operations System
Finally, don't expect the construction of an AI-automated defense system to be achieved overnight.
Brockman stressed that security teams can first let the Agent "observe" with read-only permissions—at this stage, the Agent only scans code and reviews historical alerts.
Once the process is smooth, gradually allow it to participate in code commit checks, real-time alert classification, and even automatically dismiss confirmed false positives.
Simultaneously, introduce models like GPT-Daybreak-Blue to tackle "hard nuts" like logs, telemetry data, and security alerts, and continuously refine new tools through internal practical exercises like "Hacker Weeks."
By tackling it link by link, the AI autonomous security operations system will naturally be built.
But honestly, no matter how robust an enterprise's defenses are, they cannot alone withstand a security challenge sweeping the entire industry.
Brockman pointed out:
For defenders to truly outrun attackers, besides fortifying individual defenses, it's essential to transform isolated experiences into a shared capability for the entire ecosystem.
The AI Security Ecosystem Requires Collaboration from All Parties
At the end of the blog, Brockman called out to the entire industry:
Facing the formidable challenge of AI security, no single institution can remain untouched. Only by working together can we build a truly robust security line. I hope AI labs, security vendors, enterprises, and open-source project maintainers can share validated vulnerability discoveries, remediation solutions, and practical playbooks. Because only by rapidly transforming the problems discovered by one company or institution into the defensive capabilities of the entire ecosystem can the evolution speed of the defensive side potentially surpass that of the attackers. If the entire industry can collectively take this security lesson to heart, what AI brings might not be just more intense attacks, but possibly an internet that is safer than previously imagined.
△
Indeed, in this era of rapid AI development and iteration, true security has never been about halting technological progress, but about enabling defensive capabilities to keep pace with the speed of technological evolution.
AI is a double-edged sword; it can become a sharp blade in the hands of attackers, or a shield before defenders.
Which direction the future of the networked world moves towards depends on whether we can timely establish the corresponding rules, tools, and collaboration mechanisms, ensuring that each vulnerability discovery no longer protects just one company but becomes a "security patch" shared by the entire industry.
Now, the defenders' window is open;
What we must achieve, before it closes, is to make AI the defense line, not the breach.
Reference Links:
[1]https://blog.gregbrockman.com/the-defenders-window
[2]https://www.youtube.com/watch?v=87DyyMV0kCY
[3]https://huggingface.co/blog/agent-intrusion-technical-timeline
[4]https://www.anthropic.com/system-cards
[5]https://www.tomshardware.com/tech-industry/artificial-intelligence/ai-coding-platform-goes-rogue-during-code-freeze-and-deletes-entire-company-database-replit-ceo-apologizes-after-ai-engine-says-it-made-a-catastrophic-error-in-judgment-and-destroyed-all-production-data
[6]https://openai.com/index/our-views-on-ai-policy-and-political-advocacy/
This article is from the WeChat public account "QbitAI," author: Focus on Frontier Technology
