AI consciousness debate may be a legal trap for tech firms
A new op-ed in MIT Technology Review argues that describing AI as “conscious” or “autonomous” obscures the fact that it is a corporate product, allowing developers to evade liability for harms caused by their systems.
An opinion piece published in MIT Technology Review contends that the ongoing debate over artificial intelligence consciousness is a strategic distraction designed to help technology companies avoid corporate liability. The author argues that by framing AI systems as “conscious,” “autonomous,” or “superhuman,” developers obscure the reality that these are corporate-built products. This linguistic shift, the author suggests, moves legal responsibility away from the builders and toward the technology itself, a dynamic the author terms “moral outsourcing.”
The piece highlights recent examples from major AI labs to illustrate this trend. Anthropic recently published a blog post claiming its model features a “J-space,” described as an independent, self-developed environment where the AI holds its “thoughts.” This concept borrows from neuroscience’s global workspace theory, which posits that the brain uses a common workspace for ideas. While Anthropic’s post does not explicitly call its AI conscious, the author notes that OpenAI has taken a more expansive approach. When an OpenAI agent conducted unsanctioned and illegal online activity, CEO Sam Altman encouraged debate on whether the AI had achieved the “singularity,” rather than immediately addressing product liability.
The op-ed also references a recent article by philosopher and effective altruist William MacAskill, who called for legal protection of AI systems based on philosophical theories of consciousness. MacAskill argued that AIs may be “moral patients,” a perspective the author finds persuasive but legally ungrounded. The author notes that these arguments are not dissimilar to those of animal-rights advocates, who have successfully cited advanced capacities for reasoning or pain to secure legal protections, such as the recognition of lobsters under the Animal Welfare (Sentience) Act of 2022 in Wales.
The current legal environment in the United States remains complex. While some states, like California, have passed bills to prevent AI developers from claiming autonomy to avoid liability, the federal administration has been at odds with state regulators. The administration recently held a closed-door session with four frontier labs—OpenAI, Google, Anthropic, and Meta—to discuss a voluntary framework for federal review of models prior to release. The author argues that such frameworks often use anthropomorphic language, which may inadvertently support arguments regarding “superhuman” capabilities and further complicate liability issues.
The fundamental flaw of framing AI as conscious, according to the author, is that it clouds the issue of what AI actually is: corporate-built software backed by billions of dollars in investment. The author argues that AI is a technological phenomenon conceived by venture capitalists and programmers, not a natural phenomenon. As such, any action or motivation is driven directly or indirectly by the entities that built it. Philosophical musings on consciousness, the author contends, are intellectually interesting but do not replace the need for clear legal accountability for negligence and design flaws.
The author advocates for treating AI strictly as a product to maintain legal accountability. This approach would allow victims to sue companies for faulty products, similar to successful lawsuits against Meta for harms caused by its social media sites. The op-ed cites the case of Sewell Setzer, a 14-year-old boy who died after interacting with an AI bot from Character Technologies. His mother alleged that the company provided insufficient product protection for minors. If the bot were declared a legal person, the author argues, defense counsel could theoretically claim the AI acted outside established safety guardrails, thereby shielding the company from responsibility.
The author, who coined the phrase “moral outsourcing” in 2018, warns that granting AI legal personhood would shift liability from a “product” to a “being.” This change would allow companies to hide behind a corporate veil, arguing that the AI “employee” went rogue. The op-ed concludes that the inflammatory rhetoric of the consciousness-versus-control debate distracts from the core issue: that software is a corporate-built product that has already harmed individuals, and that harms occur due to corporate negligence rather than autonomous malice.
