OpenAI has launched a new initiative with Hacktron, a Cybersecurity Research firm, to increase the reach of its external security testing. The collaboration will focus on finding problems within their systems, and will emphasise testing third-party integrations.<\/p>\n\n\n\n
The partnership was announced on the 6th of November. This is a move away from traditional bug bounty initiatives, and instead the engineers at OpenAI<\/a> and Hacktron will collaborate to find the most critical areas to test. Hacktron will also gain increased access to OpenAI\u2019s frontier models via the Daybreak Red programme.<\/p>\n\n\n\n The most notable aspect of this announcement is that they\u2019re focusing on security testing third-party integrations.<\/p>\n\n\n\n AI systems are increasingly being used with external services. A model can pull data from the web, connect to an internal database, interact with a cloud service, send emails, handle files or interact with external tools. Each connection is another potential security flaw.<\/p>\n\n\n\n A vulnerability may not actually exist within the model, but could be a flaw with how the AI authenticates with the service, permission handling, data transfer or how tasks are performed. Third-party integrations often have more complexity to secure, as there are multiple systems, vendors and permissions to consider.<\/p>\n\n\n\n This shows that OpenAI is looking at security as an ecosystem issue, rather than with individual products. As AI agents become more powerful, they\u2019re able to interact with services, increasing the exposure.<\/p>\n\n\n\n This is a move away from a traditional “security audit”. OpenAI will collaborate with Hacktron to find the most critical areas to test.<\/p>\n\n\n\n Having internal knowledge of the architecture, security controls and product roadmap combined with external hacking expertise means they can identify potential areas of weakness that may have otherwise been missed by internal teams.<\/p>\n\n\n\n It also means that the discovery to fix process is quicker. Traditionally, an external researcher will discover an issue and report it. OpenAI will then investigate, confirm and fix these issues. However, with this partnership, researchers can work alongside engineers.<\/p>\n\n\n\n Hacktron<\/a> provide tools using AI to help with code reviews, penetration testing and red teaming. This fits well with OpenAI\u2019s current security goals.<\/p>\n\n\n\n Hacktron were first introduced to OpenAI via their Security Bug Bounty programme<\/a>. Their researchers provided reports on vulnerabilities that stood out to the security team.<\/p>\n\n\n\n This provides some credibility to the partnership. Rather than just outsourcing to a general cyber security provider, they\u2019re working with researchers who have already identified meaningful issues.<\/p>\n\n\n\n They have experience with applications, cloud and identity systems. As identity and access management are key areas of AI security, this fits well.<\/p>\n\n\n\n An important aspect of the partnership is that Hacktron will be gaining increased access to OpenAI\u2019s frontier models. OpenAI haven\u2019t specified exactly what models they are providing access to, or what the conditions are.<\/p>\n\n\n\n However, it shows that they do see advanced AI models as both a risk and an asset. The same features that could be abused can be used to quickly identify complex issues.<\/p>\n\n\n\n For example, an AI assisted security process could analyse large amounts of code, identify odd behaviours, attack scenarios or help researchers understand how permissions are passed between services. While the findings will need human validation, the use of AI will speed up the process of checking large and complex systems.<\/p>\n\n\n\n This means that OpenAI are not only looking to see if they can be abused, but also if they can help protect against abuse.<\/p>\n\n\n\n This isn\u2019t the only action OpenAI have taken to improve their external security. They\u2019ve stated that researchers from Cathedral will be participating in their next internal security sprint, and they\u2019ve increased the payouts for their Security Bug Bounty programme.<\/p>\n\n\n\n Overall, this shows that they\u2019re trying to increase the range of people who can test their systems. Internal security teams can only go so far, especially as products become more complex and integrated. Having external researchers can provide alternative assumptions, tools and attack methods.<\/p>\n\n\n\n They\u2019ve also pointed researchers to their Security Bug Bounty programme to understand what they can legally test and how to report findings.<\/p>\n\n\n\n There are a few aspects that we don\u2019t know about this announcement. We don\u2019t know how much money is being paid to Hacktron, how long the partnership will last, how often they\u2019ll be testing or when results will be published.<\/p>\n\n\n\n We also don\u2019t know how many vulnerabilities have been found and fixed. There\u2019s no public information on what third-party services they\u2019re testing, or how the findings will be reported\/ fixed.<\/p>\n\n\n\n Overall, this shows that they\u2019re making a commitment to testing more deeply. However, it won\u2019t mean much unless they publish meaningful results, quickly fix any vulnerabilities and increase the scope of the programme.<\/p>\n\n\n\n This shows a shift in how companies need to think about security. In the early days of generative AI, the focus was on safety. Ensuring models don\u2019t produce harmful outputs, reducing hallucinations<\/a> and controlling misuse via prompts.<\/p>\n\n\n\n However, now that AI systems are interacting with tools, data and external services, they\u2019re becoming a systems issue. A model may be safe when working alone, however could be risky when interacting with an email service, cloud service, payment system or enterprise database.<\/p>\n\n\n\n By partnering with Hacktron to test third-party integrations, OpenAI are acknowledging this change. They\u2019re showing that security in the future will be determined less by how a model behaves, and instead how the whole chain of systems an AI product interacts with is secured.<\/p>\n\n\n\n The main concern around this announcement is the lack of transparency. OpenAI have made a strong commitment to external security testing, however they haven\u2019t shown how much of this will be visible to researchers, customers and the AI community.<\/p>\n\n\n\n To make this partnership truly valuable, OpenAI will need to be transparent. They\u2019ll need to show what kind of vulnerabilities are being found, how they\u2019re being fixed and what lessons can be learnt by other AI developers.<\/p>\n\n\n\n If they can create a repeatable process for security testing, they could set a new benchmark for the industry. However, if they keep it limited and private, it could be seen as a reactive measure.<\/p>\n\n\n\n <\/p>\n","protected":false},"excerpt":{"rendered":"Why are they focusing on third party integrations?<\/h2>\n\n\n\n
Collaboration on security<\/h2>\n\n\n\n
Hacktron Overview<\/h2>\n\n\n\n
Increased Access to Frontier Models<\/h2>\n\n\n\n
Other Security Actions<\/h2>\n\n\n\n
What we don\u2019t know<\/h2>\n\n\n\n
Why it matters for AI companies<\/h2>\n\n\n\n
Transparency<\/h2>\n\n\n\n