S-Risks (Suffering Risks)
Scenarios where the long-run future contains vast amounts of suffering rather than flourishing — including scenarios with digital minds in negative hedonic states, or stable dystopias locked in by powerful AI.
Scale of the Problem
A stable dystopian attractor could affect 10^30+ future minds if civilization survives for cosmic timescales.
Potentially much worse than extinction. Possible pathways include sentient subroutines in AI training, malicious AI optimization, stable dystopias, and acausal conflict scenarios. Scale is limited only by available computation across the long-term future, which could be astronomical.
Why This Is Pressing
Risks of astronomically bad futures involving widespread suffering receive even less attention than extinction risks, despite potentially being worse. AI-enabled optimization, conflict between advanced agents, and lock-in of malicious values could create suffering at unprecedented scale. The field is small, pre-paradigmatic, and especially in need of careful conceptual work now, before transformative AI makes course corrections harder or impossible.
Why It's Neglected
Global funding for suffering-focused work is under 5 million dollars per year. The Center on Long-Term Risk and the Center for Reducing Suffering are the two core institutions, with roughly 10 to 20 researchers between them. Most longtermist funding goes to extinction-focused work.
Can It Be Solved?
Early-stage. Useful work includes cooperative AI research, bargaining theory, suffering-focused ethics, and overlap with power-seeking AI and misuse prevention. Direct interventions are few; much current work is agenda-setting, conceptual, and aimed at building the field.
Research & Solutions
Astronomical Stakes: Understanding and Mitigating Suffering Risks (S-Risks)
WorldProblems Solved
Sign in to read full documents, vote on solutions, and submit your own research.
What You Can Do
What this problem still needs
Research at the Center on Long-Term Risk or the Center for Reducing Suffering. Donate to either organization. Cooperative AI research at CLR or the Cooperative AI Foundation. Adjacent work on AI safety, bargaining theory, and game-theoretic robustness.
Priority Score
21
Breakdown
Fund this problem
Support the researchers and organizations working on S-Risks (Suffering Risks).
Donate to this causeContribute to solving this
Create a free account to submit research, vote on solutions, and follow this problem.
Join for free