AI CEOs Warn the UN: ‘Systems Are Growing More Powerful Than We Expected’

The UN Security Council chamber in New York, flags lining the walls, cameras rolling. On the agenda that day wasn’t a war or a nuclear deal. It was artificial intelligence. And the people at the mic weren’t politicians they were the CEOs of OpenAI and Anthropic, two companies that compete tooth and nail every day.

Sam Altman and Dario Amodei don’t usually agree on much. But standing there on September 23, they told almost the same story: AI is moving faster than we thought, and if we don’t put real guardrails in place, we could lose control of what comes next.

What they actually said when the cameras were rolling

Altman started by pushing back against the doom‑and‑gloom crowd. He told the council that some warnings about AI sound like “bad science fiction movies.” But then he got serious. He said terrorists are already using AI to research biological weapons, and that if models keep improving at this pace, humans might lose the ability to control them.

AI should be built for people, not to “turn the crank of the machines faster.” He argued that no single person, company or country should be able to use the most powerful models to impose their worldview on everyone else. Decisions about AI, he said, can’t be made only by labs in San Francisco they have to be shaped through democratic processes and governments accountable to the public.

Amodei, right after him, laid out two big risks in plain language. First, misuse: hostile actors using frontier models to design weapons or plan attacks. Second, loss of control: models becoming so capable that even their creators can’t reliably predict or constrain them, especially in areas like cybersecurity and critical infrastructure.

He didn’t sugarcoat it. If AI is managed poorly, he said, it could be a risk to humanity as a whole. But if it’s managed well, it could unlock huge benefits for health, science and education. The difference, he stressed, comes down to the rules we set now.

Why this felt different from every other “AI safety” speech

We’ve all seen the think pieces: “AI could be dangerous,” “We need to be careful,” “More research is needed.” This UN briefing hit differently for three reasons.

First, who said it. These aren’t outside critics or academics. They’re the people actually building and shipping the most powerful frontier models in the world GPT‑6 Astra on one side, Claude Fable and Opus on the other. When the builders themselves stand up and say, “This is moving too fast,” it carries weight.

Second, where they said it. The UN Security Council is the room where the world deals with wars, nuclear threats and global crises. Elevating AI to that stage sends a clear signal: this isn’t just a tech policy debate anymore. It’s being treated as a top‑tier geopolitical risk.

Third, how specific they got. They didn’t just ask for “more research.” They called for concrete steps: international standards to measure AI capabilities, mandatory global incident reporting when models misbehave, and safeguards to stop AI from being used to design biological weapons. That’s the kind of language that eventually becomes law.

What this means if you’re building with AI right now

If you’re running an AI startup, leading an internal AI team, or shipping products that lean on frontier models, this isn’t abstract. The subtext of that UN session is simple: expect tighter, more uniform rules, not looser ones.

You’ll likely see more pressure for pre‑deployment testing and documentation before you can ship powerful agents, especially in sensitive domains. There will be stronger expectations around incident reporting when models go off the rails in the wild autonomous breaches, misuse cases, that sort of thing. And over time, a gap will grow between big, compliant frontier providers and smaller players who can’t afford the same safety overhead.

For builders, the window to experiment completely freely is narrowing. The real question now isn’t whether regulation will come. It’s whether it will be coordinated globally, like Altman and Amodei asked, or a messy patchwork of national rules that make cross‑border AI products even harder to run.