L. M. Sacasas in the latest edition of The Convivial Society suggests the nature of the real AI apocalypse:
AI is apocalyptic in exactly one narrow sense: it is not causing, but rather revealing the end of a world. We get our word apocalypse from a Greek word meaning “to reveal, to disclose, or to uncover.” What I am suggesting is that AI, as it is being developed, deployed, and hyped (and criti-hyped), forces us to reckon with the fact that modernity is expiring, and it is expiring precisely to the degree that it no longer serves the interest of and is at various points, particularly in its techno-economic dimensions, openly hostile to the human person. As a second nature, the culture of technological modernity, while undoubtedly improving the lot of humanity in important ways, has become, in other respects, inhospitable to our species. AI can thus be read as a last ditch effort to shore up the old decrepit structures and to double down on the promise of scale, efficiency, rationality, control, and prediction. It can also be read as an effort to extend the logic of late modernity to a point of absurdity. So where we see a proposed or actual application of AI, we might do well to ask how it relates to the end of the world we have called modern.
It’s not that AI itself is some sort of machine overlord threat, but that it’s seen by the human ones as the apotheositic extension of their own control.