Minutes From Nuclear War? AI Threat Sparks New Fears

An AI system that disrupts a nuclear command network could leave Washington or Beijing with just minutes to determine if the other has launched an attack, according to US and Chinese security experts.

Experts in US-China dialogue are advising both governments to prepare for scenarios involving powerful, autonomous systems. They recommend setting clear boundaries for nuclear systems, ensuring human oversight of major cyberattacks, and establishing a hotline for autonomous AI incidents.

Last week, the recommendations were released ahead of government discussions on artificial intelligence and a meeting on September 24 in Washington, D.C., between U.S. President Donald Trump and Chinese President Xi Jinping. A senior fellow from the Brookings Institution and an associate professor at Fudan University participated in the ongoing U.S.-China AI and national security dialogue that started in 2019.

Their proposals highlight the evolving U.S.-China AI rivalry, shifting from chip competition to key arms-control issues. These include preventing misinterpretations of accidents as attacks, limiting autonomous military decisions, and establishing communication protocols for nuclear-armed rivals amidst technological unpredictability.

This week, China’s State Security Minister stated that artificial intelligence (AI) fundamentally changes military conflict, moving from a supportive role to a critical one, especially during the US-Israeli conflict with Iran. The Brookings Institution argued that only humans should have the authority to initiate AI-enabled cyberattacks on another country’s nuclear and strategic systems.

An autonomous system can malfunction or be compromised, making it hard for a country under attack to determine if the aggression was accidental or intentional. Jiang from Fudan University recommends setting clear boundaries to prevent AI from launching nuclear weapons or targeting command systems and stresses the need to protect critical infrastructure like energy, finance, and healthcare.

He stressed the importance of a shared definition of “meaningful human control” to prevent the US and China from using similar language to justify different levels of machine autonomy. An associate senior fellow at the Stockholm International Peace Research Institute pointed out in February that this convergence is significant, even as both countries increasingly integrate AI into their nuclear systems.

He proposed a military hotline between the U.S. and China to address AI incidents, cautioning that automated cyber exchanges could escalate beyond human control. If one country’s defensive AI interprets suspicious activity and responds, the other might mistake it for an attack. A hotline would help clarify accidental or unauthorized AI actions and prevent misunderstandings of aggression.

Experts at Johns Hopkins School of Advanced International Studies have raised concerns about the US-China military crisis communication hotline’s effectiveness, noting that Chinese officials did not respond to US calls during the “spy balloon incident” in February 2023. They also highlighted that China’s strict hierarchy limits lower-ranking officers’ initiative during these interactions.

“Communication between China and the United States will not occur until top leadership consults key officials, which may take several days,” she stated. While neither government has publicly supported the Brookings-Tsinghua proposals, China is increasingly incorporating artificial intelligence into its arms control framework, particularly within the Foreign Ministry’s Department of Arms Control, which handles nuclear weapons and missile issues.

At a UN meeting in Geneva in June, China’s ambassador for disarmament affairs said military AI could jeopardize strategic stability and heighten the risk of conflict escalation. He insisted that weapons should remain under human control and warned against using AI safety as a justification for technological barriers.

Washington has no centralized office for AI arms control; national security policy is split among the White House National Security Council, the State Department, and the Pentagon. Both the U.S. and China are wary of measures that may hinder technological progress, with Trump warning that broad restrictions could benefit China, while Beijing views U.S. technology controls as efforts to preserve American dominance.