“‘Superhuman’ Isn’t Well Specified” by JustisMills
07:46 Share

** Strength**

In 1997, with Deep Blue's defeat of Kasparov, computers surpassed human beings at chess. Other games have fallen in more recent years: Go, Starcraft, and League of Legends among them. AI is superhuman at these pursuits, and unassisted human beings will never catch up. The situation looks like this:[1] At chess, AI is much better than the very best humans The average serious chess player is pretty good (1500), the very best chess player is extremely good (2837), and the best AIs are way, way better (3700). Even Deep Blue's estimated Elo is about 2850 - it remains competitive with the best humans alive.

A natural way to describe this situation is to say that AI is superhuman at chess. No matter how you slice it, that's true.

For other activities, though, it's a lot murkier. Take radiology, for example: Graph derived from figure one of CheXNet: Radiologist-Level Pneumonia Detection [...]

Outline:

(00:10) Strength

(02:28) Effort

(04:35) And More

(06:36) Beyond Superhuman

The original text contained 1 footnote which was omitted from this narration.

First published: May 3rd, 2025

Source: https://www.lesswrong.com/posts/R7r8Zz3uRyjeaZbss/superhuman-isn-t-well-specified)

    ---
    

Narrated by TYPE III AUDIO).

Images from the article: ))![Graph showing performance scores versus cost per task for O-series models.

The scatter plot displays various model versions (O1-Mini through O3 High) with their performance scores (0-100%) plotted against logarithmic cost per task ($1.0-$1,000.0).](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fd36df936-b8fd-4516-b5a2-190dac733612_1200x675.jpeg)) Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts), or another podcast app.