Does Advanced AI Pose a Real Existential Threat?

“AI capability gains have repeatedly surprised the experts building these systems. GPT-4 passed the bar exam, aced medical licensing tests, and wrote publishable academic papers — none of which was predicted one year before it happened. We cannot assume alignment will be solved in time when the capability curve is consistently ahead of schedule.”

“A sufficiently advanced system optimizing for a misspecified goal could pursue that goal in ways catastrophically harmful to humans. The paperclip maximizer is a toy example, but the underlying logic — that an optimizer doesn't intrinsically value human welfare — applies to any system whose objective function diverges from what we actually want.”

“There is no credible path from current LLMs to a system capable of outmaneuvering human civilization. Current models are pattern-matching engines that can't reliably count to ten without error. The gap between stochastic text prediction and autonomous world-altering agency is not an engineering problem being actively solved; it is a conceptual one.”

“Existential risk framing distracts from concrete AI harms happening today: algorithmic discrimination in hiring, credit, and criminal justice; deepfake disinformation at scale; and surveillance infrastructure sold to authoritarian governments. These harms are measurable, affect real people now, and receive less attention and funding because of the existential-risk narrative.”