In our jobs as AI safety researchers, we think a lot about what it means to have reasonable beliefs and to make good decisions. This matters because we want to understand how powerful AI systems might behave. It also matters because we ourselves need to know how to make good decisions in light of tremendous uncertainty about how to shape the long-term future.

It seems to us that there is a pervasive feeling in this community that the way to decide which norms of rationality to follow is to pick the ones that win. When it comes to the choice between CDT vs. EDT vs. LDT…, we hear we can simply choose the one that gets the most utility. When we say that perhaps we ought to be imprecise Bayesians, and therefore be clueless about our effects on the long-term future, we hear that imprecise Bayesianism is “outperformed” by [...]

---

Outline:

(01:31) “Winning” gives little guidance

(01:50) Avoiding dominated strategies

(04:36) Heuristics

(11:27) Non-pragmatic principles

(15:25) Conclusion

(16:48) Acknowledgments

The original text contained 6 footnotes which were omitted from this narration.

---

First published:
November 5th, 2024

Source:
https://www.lesswrong.com/posts/QxoGM89f8zr3JmNrz/winning-isn-t-enough

---

Narrated by TYPE III AUDIO.

“Winning isn’t enough” by Anthony DiGiovanni, JesseClifton 17:23 Share

LessWrong (30+ Karma)

Shownotes Transcript

“Winning isn’t enough” by Anthony DiGiovanni, JesseClifton