“Handling schemers if shutdown is not an option” by Buck
23:56 Share

What if getting strong evidence of scheming isn't the end of your scheming problems, but merely the middle?

In most of our research and writing on AI control, we’ve emphasized the following situation:

• The AI developer is deploying a model that they think might be scheming, but they aren’t sure.
• The objective of the safety team is to ensure that if the model is scheming, it will be caught before successfully causing a catastrophe.

For example, in our original paper on AI control, the methodology we used for evaluating safety protocols implies that once the developer is confident that the model is a schemer, they’re totally fine with undeploying the untrusted model.[1][2] But that seems kind of unlikely. Even if AI companies find strong evidence that their AIs are trying to escape, they won’t necessarily stop deploying them. This might be justified: even a responsible actor might think that [...]

Outline:

(03:18) Behavioral observations vs other ways of detecting scheming

(08:29) What does control look like if you need to keep deploying schemers?

(15:06) How does the prospect of needing to deploy known schemers affect the relative value of control research?

(16:56) What research should be done that focuses specifically on this?

(17:02) Sample-efficient ML

(17:27) Training recalcitrant policies

(19:56) A few other historical reasons for focusing on before-you-catch

The original text contained 4 footnotes which were omitted from this narration.

First published: April 18th, 2025

Source: https://www.lesswrong.com/posts/XxjScx4niRLWTfuD5/handling-schemers-if-shutdown-is-not-an-option)

    ---
    

Narrated by TYPE III AUDIO).

Images from the article: ) Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts), or another podcast app.