When discussing AI control, we often talk about levels of AI capabilities where we think control can probably greatly lower risks and where we can probably estimate risks. However, I think it's plausible that an important application of control is modestly improving our odds of surviving significantly superhuman systems which are misaligned. This won't involve any real certainty, it would only moderately reduce risk (if it reduces risk at all), and it may only help with avoiding immediate loss of control (rather than also helping us extract useful work). In a sane world, we wouldn't be relying on control for systems which have significantly superhuman general purpose capabilities. Nevertheless, I think applying control here could be worthwhile. I'll discuss why I think this might work and how useful I think applying control in this way would be.
As capabilities advance, the level of risk increases if we hold [...]
Outline:
(02:49) (Limited) hopes for controlling superintelligence
(05:59) How much time can control buy?
(12:39) Conclusion
The original text contained 9 footnotes which were omitted from this narration.
First published:
June 27th, 2025
Source:
https://www.lesswrong.com/posts/ainn5APCKHTFxuHKv/jankily-controlling-superintelligence)
Narrated by TYPE III AUDIO).
