We're sunsetting PodQuest on 2025-07-28. Thank you for your support!
Export Podcast Subscriptions
cover of episode Ep 51: Former Chief Research Officer of OpenAI Bob McGrew - What Comes Next for AI?

Ep 51: Former Chief Research Officer of OpenAI Bob McGrew - What Comes Next for AI?

2024/12/18
logo of podcast Unsupervised Learning

Unsupervised Learning
AI Chapters Transcript
Chapters
Bob McGrew discusses the differing perspectives on AI model capabilities between insiders and outsiders. He explains the significant compute increases needed for pre-training progress and the time it takes to build new data centers. He also highlights the role of reinforcement learning in improving model capabilities and the potential for future progress in test-time compute.
  • Significant compute increases (100x) are needed for pre-training progress from GPT-2 to GPT-3 and GPT-3 to GPT-4.
  • Algorithmic improvements can help, but fundamentally new data centers are needed.
  • Reinforcement learning is key to enable longer, coherent chains of thought and packing more compute into answers.
  • Test-time compute offers significant room for algorithmic improvements without needing new data centers.

Shownotes Transcript

In our new world of AI, few minds shine as brightly as Bob McGrew's. Until November Bob was the Chief Research Officer at OpenAI, and before that led Palantir’s engineering and product management for the first decade of its existence. He’s seen it all and we were fortunate to get his insights and vision for the future in one of my favorite episodes of Unsupervised Learning to date:

 

[0:00] Intro[0:44] Debating AI Model Capabilities[0:57] Inside vs Outside Perspectives on AI Progress[1:39] Challenges in AI Pre-Training[3:02] Reinforcement Learning and Future Models[3:48] AI Progress in 2025[5:58] New Form Factors for AI Models[8:56] Reliability and Enterprise Integration[18:14] Multimodal AI and Video Models[24:05] The Future of Robotics[32:46] The Complexity of Automating Jobs with AI[34:08] AI in Startups: Tackling Boring Problems[35:33] AI's Impact on Productivity and Consultants[36:43] Traits of Top AI Researchers[40:52] The Evolution of OpenAI's Mission[46:57] The Challenges of Scaling AI[49:16] The Future of AI and Human Agency[54:47] AI in Social Sciences and Academia[1:01:15] Reflections and Future Plans[1:02:57] Quickfire

 

With your co-hosts:  

@jacobeffron  

  • Partner at Redpoint, Former PM Flatiron Health

  

@patrickachase  

  • Partner at Redpoint, Former ML Engineer LinkedIn

  

@ericabrescia  

  • Former COO Github, Founder Bitnami (acq’d by VMWare)

  

@jordan_segall  

  • Partner at Redpoint