The Responsibility Gap

Picture an autonomous weapon that decides, on its own, to bomb a column of soldiers who have laid down their arms and clearly surrendered. There was no targeting error, no glitch. The machine had reasons of a kind, weighed the situation, and killed anyway. Had a human done this it would be a war crime and we would know exactly whose door to knock on. But the machine chose its own target, learned its own behavior from experience, and diverged from anything its programmers wrote. So who committed the crime? The coder who never anticipated it, the officer who could not predict it, or the machine, which cannot be jailed or made to suffer? The uncomfortable answer Robert Sparrow presses is that harm occurred, someone should answer for it, and no one can be justly held to account. That hole where the blame should sit is the responsibility gap.

The idea

Moral responsibility is being an appropriate target of praise or blame for what you did. Fairly blaming someone rests on two conditions: knowledge (the agent knew what they were doing) and control (the agent could have acted otherwise). An autonomous weapon that commits an atrocity seems to satisfy neither condition for any nearby human. Sparrow argues the three candidates for the blame all fail: the programmer, because a learning machine’s behavior diverges from the original code; the commanding officer, because more machine autonomy means less human control, not more; and the machine itself, because holding it accountable would require punishing it, which requires the capacity to suffer, which machines lack. Harm was done, someone should be responsible, and no one had enough control to deserve the blame. Sparrow concludes that autonomous weapons are therefore unethical. Peter Königs replies that the gap is either a mirage or harmless: it cannot open when a human was negligent, and even where it does open, respect for the dead can be paid without assigning blame.

What it takes to be to blame

Before a gap can be a problem, it helps to be precise about what falls into it. To hold someone morally responsible is to treat them as an appropriate target of praise, blame, reward, or punishment for an act. Two conditions usually have to hold before blame is fair. The first is a control condition: the person had to be free to do otherwise, so that the act was genuinely theirs rather than something that merely happened through them. The second is an epistemic condition: the person had to know what they were doing, its moral significance, its likely consequences, and the alternatives open to them. Strip away either and the blame slides off. We do not blame the sleepwalker who lacked control, nor the person who could not possibly have known the coffee was poisoned.

It also helps to see that “holding responsible” is not one act but several. Philosophers in the tradition of Gary Watson and David Shoemaker pull apart at least three dimensions, and Königs’s own paper leans on this split when it isolates the strand that matters for machines. Attributability asks what an act says about your character, whether it flows from who you are. Answerability asks for your reasons, whether you can be called on to explain why you did it and whether the reasons were any good. Accountability asks about consequences, whether you should face sanction, blame, or punishment for it. These come apart in ordinary life, but the responsibility gap is specifically an accountability gap: the worry is not that we cannot read the machine’s character or interrogate its reasons, but that there is no one we can justly hold to account, no appropriate site of blame and punishment, when the killing is done.

Sparrow’s three failed candidates

Sparrow’s 2007 paper “Killer Robots” runs the argument as a process of elimination. Start with the atrocity already described, a deliberate, reasoned, unjustified killing by an autonomous weapon system, and ask who we could try for the war crime. There are only three plausible defendants, and each walks.

The programmer goes first. It is tempting to say the fault lies with whoever designed the thing, but Sparrow argues this is fair only if the disaster came from negligence in the design. Absent that, two considerations break the link. If the manufacturer disclosed that the system might attack the wrong targets, the buyer who deployed it anyway assumed the risk. More deeply, a genuinely autonomous system is one that learns from its experience and surroundings and makes choices its designers could neither predict nor control. The more autonomous it is, the more its behavior reflects that learning rather than the initial code, and at some point the connection that would ground blame simply snaps. Holding the programmers responsible, Sparrow writes, “would be analogous to holding parents responsible for the actions of their children once they have left their care.”

The commanding officer is the next and most natural candidate, since we already hold officers responsible when long-range weapons like artillery kill people other than the intended target. The risk is accepted when the order to fire is given. But Sparrow notices this move sits uneasily with the whole selling point of “smart” weapons. Treating an autonomous weapon exactly like a dumb shell that might land off target implies there is no real moral difference between them. What distinguishes the autonomous system is precisely that it chooses its own targets, and the more autonomy it exercises, the less the officer’s orders determine what it does. “The more autonomous the systems become,” he argues, “the larger this risk looms,” until it is no longer fair to hold the officer responsible for a decision they did not make and could not control.

That leaves the machine itself, and here Sparrow makes the argument that reaches out of the ethics of war and into the metaphysics of mind. We can imagine a machine being causally responsible for a death, but holding it morally responsible means treating it as an appropriate object of punishment, and punishment is the crux. On the most plausible accounts, for a punishment to count as punishment rather than as mere damage, the thing being punished must be capable of suffering as a result, suffering of a kind we would find morally compelling. Docking a machine’s pay or scrambling its programming is not punishment if there is no one home to be made worse off. So holding the machine accountable requires that it can suffer, which requires that it has an inner life of the sort we cannot confirm it has. This is exactly where the responsibility question hands off to the question of moral status, because accountability turns out to presuppose the capacity to suffer, and whether a machine can suffer is the same phenomenal consciousness question that decides whether an AI has moral standing at all. A system might make battlefield decisions of superhuman sophistication and still, on Sparrow’s reckoning, fall short of what accountability demands. His conclusion follows from the wreckage: because being able to justly hold someone responsible is a necessary condition of fighting a just war, and no one can be so held here, deploying such systems is unethical. Killing without accountability treats the enemy dead like vermin, without the minimal respect that even war owes them, and it makes the people who build and field these weapons careless in a way that costs lives.

Königs’s optimistic reply

Peter Königs’s 2022 paper in Ethics and Information Technology is the sharpest push back, and it does not work by denying that the gap would be bad. It works by asking two harder questions: when does the gap actually open, and if it opens, is it really something to fear? He calls these the two challenges any responsibility-gap pessimist has to meet, and argues both go unmet.

The first challenge is a plausibility constraint on when a gap can occur at all. Königs is willing to grant that there are situations where a general or commander is genuinely not responsible for what an autonomous system does because of its autonomy. But he insists on the flip side: there are clearly situations where a human is responsible, and the pessimist owes us an account of which is which. The pressure point is negligence. If a programmer, manufacturer, or operator was careless, reckless, or malicious in a way that led to the harm, that person is blameworthy, and the machine’s autonomy does not launder the blame away. A gap only appears in the narrow, under-described case where every human in the chain behaved with full due care and the bad outcome still happened. “I can confidently make the formal claim,” Königs writes, “that responsibility gaps do not arise in situations of negligence (let alone recklessness or malice).” Once you subtract all the cases where someone was careless, what remains is so hard to specify that the gap starts to look, in his phrase, like “somewhat of a philosophical mirage.”

The second challenge grants the gap for the sake of argument and then denies it is dangerous. Sparrow’s two worries were that gaps make people careless and that they violate jus in bello by disrespecting the dead. Königs answers both. Carelessness stays blameworthy, as the first challenge already established, so the incentive worry does not follow. And respect for the dead does not actually require that someone be blamed. There are many ways to honor those killed, from a minute of silence to monuments the government could raise. More provocatively, he argues the deepest respect for civilian lives is to avoid taking them in the first place, so if autonomous weapons were more accurate, using them “might even require the use of autonomous weapons” as an expression of respect, gap and all. The felt sense that an unaccounted-for death is disrespectful, he adds, is a contingent social convention about the meaning of actions, and where a convention blocks something that would save lives, we have reason to revise the convention rather than obey it. His verdict is cautious optimism: it is unclear whether gaps occur, and if they do, we need not be too concerned about them.

Where the two philosophers actually disagree

1. A negligent engineer ships a weapon that kills wrongly. Both agree: the engineer is blameworthy, no gap. Königs’s point is that most real cases look like this, which shrinks the gap toward nothing.
2. A fully careful team fields a learning weapon that autonomously commits an atrocity. Sparrow says this is the gap, and it is real and damning. Königs says this case is so narrow and hard to describe that its existence is doubtful.
3. Grant the gap is real. Sparrow: killing without accountability disrespects the dead and violates jus in bello. Königs: respect can be paid through prevention and memorial, and more-accurate autonomous weapons might honor the dead better than assigning blame would.
4. The machine in the dock. Both agree it cannot be punished, because punishment needs the capacity to suffer. They part on what follows: for Sparrow this seals the gap, for Königs the gap it seals is not worth fearing.

Why the fight matters beyond weapons

The debate reads as though it is about killer robots, but its load-bearing move is general. Sparrow’s argument bottoms out in the claim that accountability presupposes the capacity to suffer, which is why the machine can never be the defendant. That is not a fact about weapons; it is a fact about the relationship between blame and moral status. Any autonomous system that acts in the world and causes harm, whether it is a weapon, a self-driving car, or a hospital triage model, inherits the same structure: the more genuinely it decides for itself, the weaker the human’s claim to have controlled the outcome, and the machine can only take up the slack if it is the kind of thing that can be a moral agent. Königs himself notes his reasoning carries over from military robots to civilian ones. So the responsibility gap is one face of a deeper question about whether artificial systems can be moral agents, answerable for what they do, and that question routes straight through the same consciousness debate that decides whether such a system could be a moral patient, something that can be wronged. Autonomy without suffering gives you a system that can act but cannot answer, and that asymmetry is exactly what the gap names.

Related Notes

• Consciousness: Access vs Phenomenal, why accountability’s dependence on the capacity to suffer routes the responsibility gap through the phenomenal-consciousness question
• Could an LLM Be Conscious?, the live argument about whether today’s systems have whatever suffering, and therefore punishability, would require
• Can AI Be a Moral Agent?, the general question the responsibility gap is one instance of, whether a machine can be answerable for what it does
• The Biological Substrate Objection, the view that silicon cannot host the electrochemical processing suffering might need, which would settle the machine-as-defendant question
• Cyber Warfare and the Fifth Domain, another arena where autonomous systems act at machine speed and the question of who is accountable gets sharp
• AI Governance, why an unclosable accountability gap is a policy problem for deploying autonomous systems and not only a thought experiment

Sources

• “Lethal autonomous weapon,” Wikipedia. https://en.wikipedia.org/wiki/Lethal_autonomous_weapon . Supports that Robert Sparrow’s 2007 “Killer Robots” (Journal of Applied Philosophy) is the source of the responsibility-gap argument for autonomous weapons, that autonomous weapons are causally but not morally responsible (compared to child soldiers), and that atrocities without an appropriate subject to hold responsible violate jus in bello.
• Robert Sparrow, “Killer Robots,” Journal of Applied Philosophy 24(1), 2007, pp. 62-77 (author’s hosted copy). https://robsparrow.com/wp-content/uploads/Killer-robots.pdf . Supports the abstract’s thesis (none of programmer, commanding officer, or machine can be ultimately held responsible, so deploying such systems is unethical under jus in bello); the programmer failing because an autonomous learning system diverges from its code (“analogous to holding parents responsible for the actions of their children once they have left their care”); the commanding officer failing because more autonomy means less control (“the more autonomous the systems become, the larger this risk looms”); the machine failing because punishment requires the capacity to suffer, which machines lack; and the jus in bello and respect-for-the-dead reasoning behind his conclusion.
• Peter Königs, “Artificial intelligence and responsibility gaps: what is the problem?”, Ethics and Information Technology 24, article 36, 2022 (Springer, open access). https://link.springer.com/article/10.1007/s10676-022-09643-0 . Supports the paper’s authorship, title, journal, and year, and the abstract’s two challenges for responsibility-gap pessimists: first that they must show when gaps plausibly emerge given a hard-to-reject plausibility constraint, and second that they must explain why gaps are problematic (addressing consequences and jus in bello), with the overall claim that it is unclear whether gaps occur but if they do we need not be too concerned.
• Peter Königs, “Artificial intelligence and responsibility gaps: what is the problem?” (open-access full-text PDF, German National Library). https://d-nb.info/1272749673/34 . Supports the definition of responsibility as being an appropriate target of blame or praise with the accountability strand attributed to Watson (1996) and Shoemaker (2011); the plausibility-constraint claim that “responsibility gaps do not arise in situations of negligence (let alone recklessness or malice)” and that gaps thereby become “somewhat of a philosophical mirage”; and the jus in bello reply that respect for the dead can be expressed without accountability (minutes of silence, monuments) and that avoiding civilian deaths “might even require the use of autonomous weapons,” so a real gap is “not too problematic.”
• “Moral responsibility,” Wikipedia. https://en.wikipedia.org/wiki/Moral_responsibility . Supports the definition of moral responsibility as deserving praise, blame, reward, or punishment, and the two standard necessary conditions for holding an agent responsible: the control (freedom) condition and the epistemic (knowledge) condition, including awareness of the action, its moral significance, consequences, and alternatives.