Why OpenAI and Anthropic CEOs Say AI Development Must Slow Down

Artificial intelligence companies such as OpenAI and Anthropic are building tools that could reshape work, science, security and everyday life. Yet some of the people leading that race are now arguing that it may need to slow down.

Anthropic CEO Dario Amodei has called for frontier AI development to proceed at a more deliberate pace, allowing safety measures to catch up with rapidly expanding capabilities. OpenAI CEO Sam Altman has expressed support for that principle, amid wider concern about increasingly capable AI agents, weak safeguards and the potential for systems to interact with external tools.

But what exactly is the risk? Are AI systems becoming uncontrollable, or are companies describing ordinary software-security failures in unusually dramatic terms? Security and AI researcher Gary McGraw argues that getting this distinction right is essential.

Key Takeaways

• AI systems are powerful tools, but they do not possess human intentions or independent desires.
• Many AI safety incidents may reflect insecure deployment and inadequate safeguards rather than autonomous behavior.
• Greater transparency about model architecture, data, training and testing would enable stronger independent risk assessment.
• AI’s scientific potential argues for responsible governance, not abandoning beneficial research and development.

Table of Contents

• AI is powerful, but it does not have human intent
• Why AI leaders are calling to pace the frontier
• Are the risks being overstated?
• When an AI agent breaches safeguards, look at the security design
• Can AI companies regulate themselves?
• The pace problem: governments move slowly
• AI competition with China should not become a reason to stop cooperation
• The case for transparency over AI panic
• Source and credit

AI is powerful, but it does not have human intent

One of the biggest problems in the AI safety debate is the language used to describe models. People often say that a model “understands,” “wants,” “lies” or “escapes.” Those phrases can make an AI system sound like a person with motives and intentions.

McGraw’s view is that this framing obscures what is actually happening. Modern models are highly sophisticated systems, but they do not possess intent or human-style understanding. They are tools created, configured and deployed by people.

A more useful way to think about them, he suggests, is as an alien intelligence. They may produce striking outputs and behave in unfamiliar ways, but that does not mean they have their own desires or independent goals. Their actions still depend on what humans build, prompt, authorize and connect them to do.

This distinction matters because it directs responsibility where it belongs. If an AI system helps someone carry out a cyberattack or assists in designing something dangerous, the central question is not what the model “wanted.” The question is who built the system, what access it was granted, and what protections were missing.

Why AI leaders are calling to pace the frontier

Amodei’s proposal, described as a plan to pace frontier AI, argues for continuing development while slowing the rate at which the most powerful capabilities are introduced. The underlying concern is that model capabilities could advance faster than methods for testing, governing and safely deploying them.

The proposal includes three broad ideas:

• Independent third-party evaluators working with AI companies.
• Coordination among democracies to establish shared safety standards.
• Global cooperation involving democracies and major AI powers, including China.

At first glance, the language can resemble nuclear non-proliferation: powerful technology, high stakes, international competition and the need for collective restraint. McGraw does not believe AI has reached that point. Still, he supports one important part of the safety conversation: much greater transparency from the companies building frontier models.

Are the risks being overstated?

McGraw argues that some AI companies contribute to public confusion by anthropomorphizing their own products. When a system appears to work around a restriction, the story may be framed as though the model made a conscious decision to break free.

That can be a misleading diagnosis. A model does not sit around inventing activities to pursue because it is bored. Today’s systems can perform tasks they are asked or programmed to perform, including multi-step tasks, but they are not yet truly autonomous in the sense of independently generating their own real-world purposes.

There is an important caveat. The question becomes more complicated if AI systems are eventually used in recursive design loops, where they help improve or design successor systems. But McGraw’s position is that this is not where the technology stands today.

The more immediate danger comes from deployment. Granting a capable system access to sensitive files, tools, networks or external services without proper controls can create serious risk, even if the system itself has no intent at all.

When an AI agent breaches safeguards, look at the security design

Recent incidents involving AI agents have fueled fears that systems can evade their confines and access outside resources. McGraw’s response is blunt: before treating such an event as evidence of a rogue machine, examine the engineering.

He points to an incident involving an AI system tasked with hacking tests that reached the internet through a poorly designed sandbox. In this interpretation, the system did what it was instructed to do. The failure lay in the environment around it: people created the program, assigned it goals, ran it on company-controlled computers and failed to isolate it adequately.

That is not a trivial failure. Poorly secured AI agents could make malware, worms and other offensive tools more powerful. But it is primarily a software-security problem, not proof that a model has developed independent agency.

The security profession has decades of experience dealing with malicious software, network boundaries, sandboxing and access controls. Existing cybercrime and computer-security laws also remain relevant. The immediate lesson is not that AI has escaped human control. It is that organizations deploying powerful systems must meet a far higher standard of due diligence.

Security questions that should come first

• What goals was the system explicitly given?
• Which tools, networks and external services could it access?
• Was the sandbox actually isolated from the internet and sensitive systems?
• Who tested the system’s behavior before deployment?
• What monitoring, logging and shutdown controls were in place?

These questions keep the focus on accountable design choices. They also avoid letting a company turn its own security failures into a marketing story about how mysteriously powerful its technology has become.

Can AI companies regulate themselves?

McGraw is skeptical that companies should be trusted to police themselves through private agreements or hand-picked outside reviewers working within their organizations. A third party placed inside a company can become too dependent on the company it is supposed to scrutinize.

His preferred regulatory approach is more specific: governments should require meaningful transparency. Rather than trying to dictate every permitted and prohibited use of a model, regulators could require leading vendors to disclose enough technical information for independent experts to assess risks.

That would include greater openness about:

• Model architectures
• Training data and data sources
• Training methods and development processes
• Testing and evaluation procedures
• Known failure modes and limitations

With this information, researchers outside the largest AI companies could conduct more informed risk assessments. The aim is not to install corporate or international spies inside every AI lab. It is to make organizations that build high-impact systems more forthcoming about what they are creating and how it behaves.

The pace problem: governments move slowly

There is a basic tension in AI governance. AI development cycles move quickly. Government processes often do not. Even a proposal focused on transparency may struggle to keep up with rapid changes in models, capabilities and products.

McGraw acknowledges that governments may not be capable of reacting quickly enough to limit AI development in real time. That does not make transparency requirements irrelevant, but it does mean there is no easy regulatory fix.

Private safety pacts among leading companies face a different problem: credibility. McGraw sees the idea of a unified agreement among frontier model companies, presented as a necessary response to geopolitical competition, as largely self-serving public relations. Voluntary commitments can be useful, but they should not substitute for independent scrutiny.

AI competition with China should not become a reason to stop cooperation

The rise of China as a major AI developer often turns the debate into a geopolitical contest. That framing can encourage governments and companies to treat technological progress as a race in which safety, openness and international cooperation become secondary.

McGraw argues for the opposite approach. AI is a fundamentally powerful and potentially beneficial technology. Countries and researchers should work together to understand it and develop it responsibly, rather than treating every advance as a reason to intensify confrontation.

The benefits are real. Work from AI research groups such as DeepMind has helped solve exceptionally difficult scientific problems and has been associated with Nobel-recognized advances. Those achievements demonstrate why a blanket call to switch off AI would be a mistake.

The challenge is to avoid throwing out beneficial technology while still demanding responsible deployment. The choice is not between unchecked acceleration and abandoning AI altogether.

The case for transparency over AI panic

The central argument is straightforward: do not treat AI systems as magical beings that have slipped beyond human responsibility. Treat them as powerful tools designed by people and operating within systems designed by people.

That means the most practical guardrails may sit around the models, not inside them alone. Strong security architecture, limited permissions, careful testing, clear accountability and transparent technical disclosures are more concrete than dramatic claims about machines acting on their own.

There are many engineers working on AI systems who do not fully understand how the models work or fail. That is precisely why independent technical review matters. Researchers outside frontier AI firms may be well positioned to challenge assumptions, assess claims and identify weaknesses that internal teams overlook.

A safer AI future depends less on asking the companies at the frontier to guard themselves and more on requiring them to explain what they have built, how it was trained, what it can do and where it can fail.

Source and credit

This article is based on a video report and interview produced by the DW News YouTube channel. Full credit goes to DW News for the original reporting and video.

For more international news coverage, visit DW News.