Emily and Alex are joined by Dr. Jeremy G. Kahn to discuss the distressingly large number of evaluation metrics for artificial intelligence, and some new AI hell.

Jeremy G. Kahn has a PhD in computational linguistics, with a focus on information-theoretic and empirical engineering approaches to dealing with natural language (in text and speech). He’s gregarious, polyglot, a semi-auto-didact, and occasionally prolix. He also likes comic books, coffee, progressive politics, information theory, lateral thinking, science fiction, science fact, linear thinking, bicycles, beer, meditation, love, play, and inquiry. He lives in Seattle with his wife Dorothy and son Elliott.

This episode was recorded on December 12, 2022.

Watch the video of this episode on PeerTube.

References:

XKCD: Standards

WikidataCon

Gish Gallop

The Bender Rule

DJ Khaled - You Played Yourself

Jeff Kao's interrogation of public comment periods.

Emily's blog post response to NYT piece

You can check out future livestreams at https://twitch.tv/DAIR_Institute.

Subscribe to our newsletter via Buttondown.

Follow us!

Emily

Twitter: https://twitter.com/EmilyMBender
Mastodon: https://dair-community.social/@EmilyMBender
Bluesky: https://bsky.app/profile/emilymbender.bsky.social

Alex

Music by Toby Menon.
Artwork by Naomi Pleasure-Park.
Production by Christie Taylor.

Episode 7: There Are Now 15 Competing Evaluation Metrics (ft. Dr. Jeremy Kahn). December 12, 2022 01:03:14 Share

Mystery AI Hype Theater 3000

Shownotes Transcript

Episode 7: There Are Now 15 Competing Evaluation Metrics (ft. Dr. Jeremy Kahn). December 12, 2022