OpenAI and Hacktron Partner to Expand Security Testing of AI Systems and Third-Party Integrations

OpenAI has launched a new initiative with Hacktron, a Cybersecurity Research firm, to increase the reach of its external security testing. The collaboration will focus on finding problems within their systems, and will emphasise testing third-party integrations.

The partnership was announced on the 6th of November. This is a move away from traditional bug bounty initiatives, and instead the engineers at OpenAI and Hacktron will collaborate to find the most critical areas to test. Hacktron will also gain increased access to OpenAI’s frontier models via the Daybreak Red programme.

Why are they focusing on third party integrations?

The most notable aspect of this announcement is that they’re focusing on security testing third-party integrations.

AI systems are increasingly being used with external services. A model can pull data from the web, connect to an internal database, interact with a cloud service, send emails, handle files or interact with external tools. Each connection is another potential security flaw.

A vulnerability may not actually exist within the model, but could be a flaw with how the AI authenticates with the service, permission handling, data transfer or how tasks are performed. Third-party integrations often have more complexity to secure, as there are multiple systems, vendors and permissions to consider.

This shows that OpenAI is looking at security as an ecosystem issue, rather than with individual products. As AI agents become more powerful, they’re able to interact with services, increasing the exposure.

Collaboration on security

This is a move away from a traditional “security audit”. OpenAI will collaborate with Hacktron to find the most critical areas to test.

Having internal knowledge of the architecture, security controls and product roadmap combined with external hacking expertise means they can identify potential areas of weakness that may have otherwise been missed by internal teams.

It also means that the discovery to fix process is quicker. Traditionally, an external researcher will discover an issue and report it. OpenAI will then investigate, confirm and fix these issues. However, with this partnership, researchers can work alongside engineers.

Hacktron Overview

Hacktron provide tools using AI to help with code reviews, penetration testing and red teaming. This fits well with OpenAI’s current security goals.

Hacktron were first introduced to OpenAI via their Security Bug Bounty programme. Their researchers provided reports on vulnerabilities that stood out to the security team.

This provides some credibility to the partnership. Rather than just outsourcing to a general cyber security provider, they’re working with researchers who have already identified meaningful issues.

They have experience with applications, cloud and identity systems. As identity and access management are key areas of AI security, this fits well.

Increased Access to Frontier Models

An important aspect of the partnership is that Hacktron will be gaining increased access to OpenAI’s frontier models. OpenAI haven’t specified exactly what models they are providing access to, or what the conditions are.

However, it shows that they do see advanced AI models as both a risk and an asset. The same features that could be abused can be used to quickly identify complex issues.

For example, an AI assisted security process could analyse large amounts of code, identify odd behaviours, attack scenarios or help researchers understand how permissions are passed between services. While the findings will need human validation, the use of AI will speed up the process of checking large and complex systems.

This means that OpenAI are not only looking to see if they can be abused, but also if they can help protect against abuse.

Other Security Actions

This isn’t the only action OpenAI have taken to improve their external security. They’ve stated that researchers from Cathedral will be participating in their next internal security sprint, and they’ve increased the payouts for their Security Bug Bounty programme.

Overall, this shows that they’re trying to increase the range of people who can test their systems. Internal security teams can only go so far, especially as products become more complex and integrated. Having external researchers can provide alternative assumptions, tools and attack methods.

They’ve also pointed researchers to their Security Bug Bounty programme to understand what they can legally test and how to report findings.

What we don’t know

There are a few aspects that we don’t know about this announcement. We don’t know how much money is being paid to Hacktron, how long the partnership will last, how often they’ll be testing or when results will be published.

We also don’t know how many vulnerabilities have been found and fixed. There’s no public information on what third-party services they’re testing, or how the findings will be reported/ fixed.

Overall, this shows that they’re making a commitment to testing more deeply. However, it won’t mean much unless they publish meaningful results, quickly fix any vulnerabilities and increase the scope of the programme.

Why it matters for AI companies

This shows a shift in how companies need to think about security. In the early days of generative AI, the focus was on safety. Ensuring models don’t produce harmful outputs, reducing hallucinations and controlling misuse via prompts.

However, now that AI systems are interacting with tools, data and external services, they’re becoming a systems issue. A model may be safe when working alone, however could be risky when interacting with an email service, cloud service, payment system or enterprise database.

By partnering with Hacktron to test third-party integrations, OpenAI are acknowledging this change. They’re showing that security in the future will be determined less by how a model behaves, and instead how the whole chain of systems an AI product interacts with is secured.

Transparency

The main concern around this announcement is the lack of transparency. OpenAI have made a strong commitment to external security testing, however they haven’t shown how much of this will be visible to researchers, customers and the AI community.

To make this partnership truly valuable, OpenAI will need to be transparent. They’ll need to show what kind of vulnerabilities are being found, how they’re being fixed and what lessons can be learnt by other AI developers.

If they can create a repeatable process for security testing, they could set a new benchmark for the industry. However, if they keep it limited and private, it could be seen as a reactive measure.