Rank #8 ITN Score: 21/30

S-Risks (Suffering Risks)

Scenarios where the long-run future contains vast amounts of suffering rather than flourishing — including scenarios with digital minds in negative hedonic states, or stable dystopias locked in by powerful AI.

Submit New Draft

1 approved

Sort

Astronomical Stakes: Understanding and Mitigating Suffering Risks (S-Risks)v1Accepted

WorldProblems Consortium

S-risks — scenarios in which the long-run future contains vast quantities of suffering rather than flourishing — represent what may be the worst possible category of outcome for civilization. Unlike extinction, S-risks involve not the end of conscious experience but its continuation under conditions of extreme and irreversible negative hedonic states. This report examines the theoretical basis for S-risk concern, surveys plausible scenarios, and proposes a research agenda for identification and mitigation.

+0 votes

Followers

Approved Docs

Evidence & References

80,000 Hours Problem Profile

https://80000hours.org/problem-profiles/s-risks/

Problem Profile

Why Pressing

Risks of astronomically bad futures involving widespread suffering receive even less attention than extinction risks, despite potentially being worse. AI-enabled optimization, conflict between advanced agents, and lock-in of malicious values could create suffering at unprecedented scale. The field is small, pre-paradigmatic, and especially in need of careful conceptual work now, before transformative AI makes course corrections harder or impossible.

Scale

Potentially much worse than extinction. Possible pathways include sentient subroutines in AI training, malicious AI optimization, stable dystopias, and acausal conflict scenarios. Scale is limited only by available computation across the long-term future, which could be astronomical.

Neglectedness

Global funding for suffering-focused work is under 5 million dollars per year. The Center on Long-Term Risk and the Center for Reducing Suffering are the two core institutions, with roughly 10 to 20 researchers between them. Most longtermist funding goes to extinction-focused work.

Tractability

Early-stage. Useful work includes cooperative AI research, bargaining theory, suffering-focused ethics, and overlap with power-seeking AI and misuse prevention. Direct interventions are few; much current work is agenda-setting, conceptual, and aimed at building the field.