Shownotes Transcript
Audio note: this article contains 172 uses of latex notation, so the narration may be difficult to follow. There's a link to the original text in the episode description.
This post was written during Alex Altair's agent foundations fellowship program, funded by LTFF. Thank you Alex Altair, Alfred Harwood, Daniel C for feedback and comments.
** Introduction** The selection theorems agenda aims to prove statements of the following form: "agents selected under criteria <span>X</span> provably has property <span>Y</span>," where <span>Y</span> are things such as world models, general purpose search, modularity, etc. We're going to focus on world models. But what is the intuition that makes us expect to be able to prove such things in the first place? Why expect world models? Because: assuming the world is a Causal Bayesian Network with the agent's actions corresponding to the <span>D</span> (decision) node, if its actions can robustly control the [...]
Outline:
(00:30) Introduction
(03:07) Basic Setup
(03:11) World
(03:42) Agent as a Policy Oracle
(05:14) Robustness as \delta-optimality under interventions
(10:27) Assumptions
(11:30) Main Theorem
(12:31) High-level argument
(16:34) Discussion
(17:08) Policy oracle is not a good model of an agent.
(18:03) The Causal Good Regulator Theorem isnt a structural theorem.
(20:51) Conclusion
The original text contained 1 footnote which was omitted from this narration.
The original text contained 11 images which were described by AI.
First published: December 14th, 2024
Source: https://www.lesswrong.com/posts/Rwjrrmn6LBzHrfen7/summary-of-robust-agents-learn-causal-world-model)
---
Narrated by TYPE III AUDIO).
Images from the article:
