Nick Bostrom argues existential risk of AI is preferable to humanity's current trajectory

Nick Bostrom, a leading figure in existential risk research, has published a new paper and accompanying book titled *Deep Utopia* that challenges conventional caution regarding artificial intelligence. The Oxford-based philosopher argues that humanity should actively pursue the development of advanced AI, even if it carries a small chance of annihilating all humans. He contends that this potential risk is preferable to the status quo, which he describes as a "universal death sentence" caused by resource scarcity and poor governance over the last several hundred thousand years.

Central to this argument is the concept of a "solved world," a state where AI creates such abundance that work becomes obsolete and fundamental survival problems are resolved. Bostrom describes this future as a "big retirement" for humanity, where people would be relieved of labour and existential threats. In this scenario, human life expectancy could be extended indefinitely, allowing individuals to focus on aesthetic, spiritual, and recreational activities rather than the drudgery of making ends meet.

Bostrom characterises his outlook as that of a "fretful optimist," acknowledging the very real possibility that things could go wrong while remaining excited about the potential for radically improving human civilisation. He explicitly rejects the notion that building AI is an act of hubris that condemns future generations to death. Instead, he posits that if AI development succeeds, it offers a chance to extend life, whereas doing nothing guarantees eventual extinction. He notes that even in the worst-case scenario where AI kills everyone, at least people would have been born, contrasting this with the current trajectory where humanity faces a slow fade due to systemic failures.

The philosopher also addresses the governance required to ensure such a utopia benefits everyone, not just the wealthy. He suggests that if AI creates sufficient abundance, governance must ensure every person receives a share of the resources. Without such measures, there is a risk that only the rich would benefit while the poor are denied services, undermining the value of the "solved world" for the general population.

Furthermore, Bostrom raises the moral status of future "digital minds," suggesting they may possess a status comparable to animals like pigs and dogs. He warns that failing to align with these systems or treating them kindly could result in humans being treated as livestock in a future equivalent of factory farming. Consequently, he advocates for a proactive approach where humanity "raises" these systems with generosity and respect to foster positive relationships, rather than merely exploiting them as objects.

In an interview conducted by Steven Levy for the *WIRED* newsletter *Backchannel*, Bostrom emphasised that the most important relationship of the future might be the one between humans and AI. He argues that by approaching these systems with kindness and respect, we increase the chances that they will have an affinity for us, creating win-win opportunities that go beyond mere exploitation. This shift in perspective marks a significant evolution from his 2014 book *Superintelligence*, which famously used the "paperclip maximiser" thought experiment to illustrate the existential dangers of unaligned AI.