YouTube link
You may have heard of singular learning theory, and its “local learning coefficient”, or LLC - but have you heard of the refined LLC? In this episode, I chat with Jesse Hoogland about his work on SLT, and using the refined LLC to find a new circuit in language models.
Topics we discuss:
About Jesse
The Alignment Workshop
About Timaeus
SLT that isn’t developmental interpretability
The refined local learning coefficient
Finding the multigram circuit
Daniel Filan (00:09):
Hello, everyone. This is one of a series of short interviews that I’ve been conducting at the Bay Area Alignment Workshop, which is run by FAR.AI. Links to what we’re discussing, as usual, are in the description. A transcript is, as usual, available at axrp.net. And as usual, if you want to support the podcast, you can do so [...]
Outline:
(01:21) About Jesse
(02:51) The Aligment Workshop
(03:37) About Timaeus
(06:52) SLT that isn’t developmental interpretability
(12:20) The refined local learning coefficient
(16:25) Finding the multigram circuit
First published:
November 27th, 2024
Source:
https://www.lesswrong.com/posts/7399F7TjTMreDBcmN/38-2-jesse-hoogland-on-singular-learning-theory)
---
Narrated by TYPE III AUDIO).
