The future of AI Chat: Foundation models and responsible innovation
The Future of Everything
Shownotes Transcript
Guest Percy Liang) is an authority on AI who says that we are undergoing a paradigm shift in AI powered by foundation models, which are general-purpose models trained at immense scale, such as ChatGPT. In this episode of Stanford Engineering’s The Future of Everything)* *podcast, Liang) tells host Russ Altman) about how foundation models are built, how to evaluate them, and the growing concerns with lack of openness and transparency.
Connect With Us:
- Episode Transcripts >>> The Future of Everything Website)
- Connect with Russ >>> Threads) / Bluesky) / Mastodon)
- Connect with School of Engineering >>> Twitter/X) / Instagram) / LinkedIn) / Facebook)
Chapters:
(00:00:00) IntroductionHost Russ Altman introduces Percy Liang, who runs the Stanford Center on Foundation Models
(00:02:26) Defining Foundation Models
Percy Liang explains the concept of foundation models and the paradigm shift they represent.
(00:04:22) How are Foundation Models Built & Trained?
Explanation of the training data sources and the scale of training data: training on trillions of words. Details on the network architecture, parameters, and the objective function.
(00:10:36) Context Length & Predictive Capabilities
Discussion on context length and its role in predictions. Examples illustrating the influence of context length on predictive accuracy.
(00:12:28) Understanding Hallucination
Percy Liang explains how foundation models “hallucinate”, and the need for both truth and creative tasks which requires “lying”.
(00:15:19) Alignment and Reinforcement in Training
The role of alignment and reinforcement learning from human feedback in controlling model outputs.
(00:18:14) Evaluating Foundation Models
The shift from task-specific evaluations to comprehensive model evaluations, Introduction of HELM & the challenges in evaluation these models.
(00:25:09) Foundation Models Transparency Index
Percy Liang details the Foundation Models Transparency Index, the initial results and reactions by the companies evaluated by it.
(00:29:42) Open vs. Closed AI Models: Benefits & Risks
The spectrum between open and closed AI models , benefits and security impacts
Connect With Us:
Episode Transcripts >>> The Future of Everything Website)
Connect with Russ >>> Threads) / Bluesky) / Mastodon)
Connect with School of Engineering >>>Twitter/X) / Instagram) / LinkedIn) / Facebook)