This post benefitted greatly from comments, suggestions, and ongoing discussions with David Duvenaud, David Krueger, and Jan Kulveit. All errors are my own.
A few months ago, I and my coauthors published Gradual Disempowerment (GD hereafter). It was mostly about how things might go wrong, but naturally a lot of the resulting interest has been about solutions.
We have some more formal followup work coming: in the meantime, this is my 80/20 for ‘what would I do if I had way more time’ / ‘what would I find it helpful if someone else had done well’. This document is very much breadth over depth, and still missing a lot of details; I hope it is nonetheless helpful. For many of these, I expect even a pretty motivated and smart undergraduate could make useful progress in 10-20 hours.
I would be excited about people doing good work on [...]
Outline:
(01:15) Conceptual / High-Level
(01:19) Interaction with other x-risk concerns
(02:37) Responding to counterarguments
(04:10) Beyond competitive pressures
(06:12) Clarifying the goal
(07:42) Social Science-y
(07:45) Robustness of societal fundamentals
(08:59) Studying Historical Parallels
(10:12) Indicators and Policy
(11:32) Technical / Mathematical
(11:36) Simulating entire civilizations
(12:46) AI cognition and agency
(13:26) Civilisational Alignment / Hierarchical Agency
(16:41) Differential Progress / Differential Empowerment
(17:38) AI Complementarity
(18:48) Concluding thoughts
The original text contained 1 footnote which was omitted from this narration.
First published:
May 29th, 2025
Source:
https://www.lesswrong.com/posts/GAv4DRGyDHe2orvwB/gradual-disempowerment-concrete-research-projects)
---
Narrated by TYPE III AUDIO).
