Gradual AI-Driven Disempowerment
Rather than a sudden takeover, humans may gradually cede decision-making to AI systems in ways that are individually rational but collectively undermine human agency and democratic governance.
Scale of the Problem
Over 40% of major financial trades are already executed by autonomous AI systems with minimal human oversight.
Risk is civilization-level: humans collectively lose the ability to course-correct institutions that run their lives. The 2024 paper Gradual Disempowerment by Kulveit and colleagues articulates the dynamic. Such erosion could unfold over years to decades without any AI takeover event and could end with human values no longer meaningfully represented in practice.
Why This Is Pressing
Even without a single catastrophic AI event, broad automation of cognition and eventually labor and militaries could gradually erode the feedback loops (elections, labor markets, civil society) that keep human values represented in major institutions. Early signs include AI-mediated media ecosystems, AI decision tools inside government, and concentration of AI-generated economic value. Unlike acute catastrophes, gradual disempowerment is easy to miss until correction becomes infeasible.
Why It's Neglected
Very new framing, explicitly developed only in the past few years. Core researchers include Jan Kulveit, Raymond Douglas, David Duvenaud, and David Krueger. Dedicated funding is minimal, and institutional attention lags. Gradual risks systematically receive less attention than acute ones.
Can It Be Solved?
Speculative. Levers include AI governance that preserves human agency, supporting democratic and deliberative institutions, fair distribution of AI gains, and research into post-AGI economic arrangements. Concrete interventions are less developed than for other AI risks; meta-governance research is a priority.
Research & Solutions
The Boiling Frog Scenario: Gradual Human Disempowerment in the Age of AI
WorldProblems Solved
Sign in to read full documents, vote on solutions, and submit your own research.
What You Can Do
What this problem still needs
Research at Forethought, GovAI, the Centre for Long-Term Resilience, or academic political economy programs. Policy work on AI and democracy. Support research on post-AGI economic and political arrangements. Journalism and civic-tech work that keeps institutions legible to citizens.
Priority Score
20
Breakdown
Fund this problem
Support the researchers and organizations working on Gradual AI-Driven Disempowerment.
Donate to this causeContribute to solving this
Create a free account to submit research, vote on solutions, and follow this problem.
Join for free