We're sunsetting PodQuest on 2025-07-28. Thank you for your support!
Export Podcast Subscriptions
cover of episode Ep20. AI Scaling Laws, DOGE, FSD 13, Trump Markets | BG2 w/ Bill Gurley & Brad Gerstner

Ep20. AI Scaling Laws, DOGE, FSD 13, Trump Markets | BG2 w/ Bill Gurley & Brad Gerstner

2024/11/21
logo of podcast BG2Pod with Brad Gerstner and Bill Gurley

BG2Pod with Brad Gerstner and Bill Gurley

AI Deep Dive AI Chapters Transcript
People
B
Bill Gurley
B
Brad Gerstner
Topics
Bill Gurley认为，当前人工智能模型的预训练效果提升可能遇到瓶颈，规模法则的有效性受到质疑。某些迹象表明，像OpenAI、Google和Anthropic等公司在构建更高级的AI模型时，未能达到内部预训练目标。同时，多位AI领域重要人物也对规模法则的持续性表示怀疑。然而，Dario Amodei在最近的采访中对规模法则持积极态度，这与其他人的观点存在差异。Brad Gerstner则认为，对规模法则的质疑主要集中在大型语言模型（LLMs），并且这种质疑由来已久，主要关注参数数量、上下文窗口大小和数据量三个方面。即使预训练的改进速度放缓，其他扩展智能的向量（如后训练、推理时间推理、数据质量等）组合起来也可能带来同样好甚至更好的结果。模型的改进将来自后训练、数据质量和推理时间推理等多个方面，而不是仅仅依赖于预训练。

Deep Dive

Chapters
Discussion on whether AI scaling laws are reaching their limits, focusing on pre-training improvements, data quality, and inference time reasoning.
  • Bloomberg report suggests AI models like Google's and Anthropic's are struggling to advance.
  • Dario Amodei questions the continued validity of scaling laws.
  • Mark Benioff and Horacio Gutierrez also express doubts about scaling laws.
  • Elon Musk remains positive on scaling laws in his recent interview with Lex Fridman.
  • Concerns about running out of data and the effectiveness of synthetic data.

Shownotes Transcript

Open Source bi-weekly convo w/ Bill Gurley and Brad Gerstner on all things tech, markets, investing & capitalism. This week they discuss AI Scaling Laws, the future of inference, AI memory and actions, National Full Self-Driving Regulations, FSD 13, Robotaxi and Waymo, Department of Government Efficiency (DOGE), & more. Enjoy another episode of BG2.

Timestamps:

(00:00) Intro

(02:00) AI Scaling Laws

(05:54) Implications of AI Scaling Trends

(12:20) The Future of Inference and Data Quality

(17:51) AI Memory and Actions

(21:47) National Full Self-Driving Regulation

(23:14) FSD 13

(29:41) Market Disruption: Winners and Losers in the Auto Industry

(31:18) The Future of OEM’s

(40:41) Department of Government Efficiency (DOGE)

(54:47) Tech Market Check

Available on Apple, Spotify, www.bg2pod.com)

Follow:

Brad Gerstner @altcap https://x.com/altcap

Bill Gurley @bgurley https://x.com/bgurley

BG2 Pod @bg2pod https://x.com/BG2Pod)

#BillGurley #BradGerstner #Bg2Pod