As an organization design consultant, I pay particular attention to how people’s judgment about “who is accountable if something goes wrong” shapes the decisions they make. Every time I hear a claim along the lines of “AI can now make decisions on its own, with no human involved,” I become wary. I do not believe AI has reached that degree of autonomy. But I worry that continually playing up the idea that it has may, by itself, increase the risk of war.
The Appeal of the “AI Autonomy” Narrative Runs Deeper Than It Looks
I suspect some companies are happy to play up AI autonomy because such stories keep the buzz alive, raise expectations of capability, and leave room for imagination in fundraising, valuation, and eventually an IPO. From a commercial standpoint, the motive is easy to understand. But what happens when the AI autonomy these business leaders describe so fluently is transplanted into the military domain?
AI’s appeal to military organizations is very real. Even without “autonomy,” purely as a decision-support tool, AI has the potential to process vast volumes of intelligence faster, link different sensors and combat units together, coordinate more unmanned platforms, and sustain tasks beyond human endurance. Once AI can make decisions on its own, it could directly change who wins on the battlefield.[1]
The risks of such autonomous decision-making are no secret. But the potential rewards and the constant competitive pressure may outweigh the risk concerns of the world’s major players. It is hard to expect warnings about AI risk to stop capable states and military organizations from exploring its uses.
How an Excuse Changes the Choices People Make
In this environment of continuous competition, “the AI decided on its own” can easily become a convenient excuse. Imagine a military probe. If it succeeds, it deals a heavy blow to the adversary and the rewards are considerable. If it fails, or draws a strong backlash, the immediate line is that the operation was a bug produced by AI acting autonomously, with no human involved in the decision.
An excuse does not necessarily bring immunity. But as long as it can delay investigation, scatter suspicion, and make accountability harder to pin down, it may lower certain people’s concern about the consequences.
We have no reason to assume that everyone genuinely loves peace. As long as someone believes a limited probe can pay off, and believes that if something goes wrong the blame can be shifted onto AI, he may take the gamble.
Nor is such military risk-taking confined to the attacker and the attacked. Suppose Country A’s system attacks Country B, and A claims the AI acted on its own. The truth might be a deliberate probe by A, a malfunction in A’s system, or a third party that broke into A’s network and launched the attack through A’s AI. The third party does not even need to pin the blame successfully; creating enough confusion may be sufficient to set A and B suspecting each other.
An investigation is of course necessary, but agreeing to investigate does not produce a credible answer right away. Key evidence may be scattered across different agencies; the parties involved may be unwilling to disclose everything, and unwilling to accept one another’s explanations. “The AI decided on its own,” “the system was compromised,” and “someone attacked deliberately” can coexist in the same dispute for a long time.
Attribution of accountability was often contested before the AI era too. But once “the AI decided on its own” is widely accepted by the public, those with ulterior motives gain one more story to tell, those trying to read the situation gain one more layer of doubt, and accountability becomes harder to establish and enforce.
When I wrote about the Decision Authority Matrix this March, I discussed a Reverse Moral Crumple Zone: in some automation settings, a human is pushed forward to take the blame for the system; in the military scenario I worry about, the system may be pushed forward to take the blame for humans.[2][3] If someone factors this “liability cushion” into their risk calculation before acting, the appeal of a military gamble may grow.
Retaliation Won’t Necessarily Wait for the Investigation
A military response will not necessarily wait for an investigation. The attacked side may worry that the adversary is using “the AI went rogue” to buy time; the apparent attacker may indeed have had its system hijacked yet be unable to prove it quickly. What one side sees as a necessary countermeasure, the other sees as a fresh attack. Action and counteraction can happen fast; verification and communication take time.
If such a local incident keeps escalating and eventually draws in major powers, the consequences could far exceed the original probe. A direct conflict between great powers does not require every participant to have set out to start one.
On September 20, the latest round of US-China economic and trade talks in New York also included a dialogue on AI.[4] Treasury Secretary Scott Bessent afterward spoke about a communication mechanism for AI safety incidents, and, referring to the Hugging Face incident, stressed that the accountability lay with “the OpenAI management, not a bunch of agents.”[5] That statement about corporate accountability is the direction I hope to see. When we discuss AI risk, human accountability must not be diluted by talk of “AI autonomy.”
This is why I am so wary of playing up the claim that “AI is now truly autonomous.” Even if AI’s actual capability has not changed at all, people’s judgments about accountability and risk may be changed first by the publicity. That could make some people bolder in probing, make accountability harder to pin down, and leave more room for those who would exploit confusion to provoke conflict.
Even if AI never actually gains that autonomy, humans may first turn it into a reason to gamble and an excuse to shed accountability.
Sources
[1] Congressional Research Service, Artificial Intelligence and National Security, November 10, 2020
[2] Hector Sun, Sun’s Decision Authority Matrix, March 3, 2026
[3] Madeleine Clare Elish, Moral Crumple Zones, March 23, 2019