Shownotes Transcript
OpenAI has finally introduced us to the full o3 along with o4-mini. Greg Brockman (OpenAI): Just released o3 and o4-mini! These models feel incredibly smart.
We’ve heard from top scientists that they produce useful novel ideas.
Excited to see their positive impact on people's daily lives and humanity's hardest problems!
Sam Altman: we expect to release o3-pro to the pro tier in a few weeks By all accounts, this upgrade is a big deal. They are giving us a modestly more intelligent model, but more importantly giving it better access to tools and ability to discern when to use them, to help get more practical value out of it. The tool use, and the ability to string it together and persist, is where o3 shines.
The highest praise I can give o3 is that this was by far the most a model has been used as part of writing its [...]
Outline:
(01:56) What's In a Name
(02:51) My Current Model Use Heuristics
(04:21) Huh, Upgrades
(05:31) Use All the Tools
(09:47) Search the Web
(10:27) On Your Marks
(18:15) The System Prompt
(19:00) The o3 and o4-mini System Card
(23:17) Tests o3 Aced
(25:14) Hallucinations
(31:41) Instruction Hierarchy
(32:52) Image Refusals
(33:18) METR Evaluations for Task Duration and Misalignment
(42:45) Apollo Evaluations for Scheming and Deception
(44:40) We Are Insufficiently Worried About These Alignment Failures
(47:16) GPT-4.1 Also Has Some Issues
(50:08) Pattern Lab Evaluations for Cybersecurity
(51:45) Preparedness Framework Tests
(52:14) Biological and Chemical Risks (4.2)
(58:20) Cybersecurity (4.3)
(59:27) AI Self-Improvement (4.4)
(01:00:51) Perpetual Shilling
(01:01:54) High Praise
(01:09:31) Syncopathy
(01:11:58) Mundane Utility Versus Capability Watch
(01:16:33) o3 Offers Mundane Utility
(01:24:10) o3 Doesn't Offer Mundane Utility
(01:30:54) o4-mini Also Exists
(01:31:31) Colin Fraser Dumb Model Watch
(01:32:52) o3 as Forecaster
(01:34:31) Is This AGI?
First published: April 18th, 2025
Source: https://www.lesswrong.com/posts/u58AyZziQRAcbhTxd/o3-will-use-its-tools-for-you)
---
Narrated by TYPE III AUDIO).
Images from the article:
)
)
)
)
)
)
)
)
)
)
)
)
))))))!%5BBar){kind=link}
)!%5BFive){kind=link}
)!%5BPerformance){kind=link}
)!%5BComparison){kind=link}
The chart displays eight different AI models ranked by their scores, with ](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F501cc74b-c441-474e-baed-214258a0a8a4_673x636.png))))!%5BBar){kind=link}
)!%5BTable){kind=link}
The table shows ](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F1da25928-cc68-4784-b7f7-38eaa4dd5d40_900x672.jpeg)))))))))))))))))!%5BCode){kind=link}
)!%5BGraph){kind=link}
)!%5BTable){kind=link}
)!%5BGraph){kind=link}
)!%5BMinimalist){kind=link}
)!%5BTable){kind=link}
)!%5BTable){kind=link}
)!%5BTable){kind=link}
)!%5BScreenshot){kind=link}
)!%5BBar){kind=link}
)!%5BThis){kind=link}
)!%5BGraph){kind=link}
)!%5BBar){kind=link}
)!%5BScatter){kind=link}
)!%5BBar){kind=link}
This appears to be Figure 3 showing pre-mitigation model responses across different metrics, with various colored bars representing different models and conditions.](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F69d9c5c0-7e31-41bd-a385-00a26f9f8dcc_1196x341.png)))))!%5BTwo){kind=link}
)!%5BTable){kind=link}
)!%5BBenchmark){kind=link}
The image shows four panels comparing different ChatGPT model versions' responses when asked to rate attractiveness. Three earlier versions consistently refused to rate (100% refusal), while the latest version shows consistently high ratings of 8/10 across five trials, displayed in a green bar graph.](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fb1a976cf-4e36-400f-a7c3-de339c89ffdd_900x764.jpeg)))))))))!%5BTable){kind=link}
)!%5BSamuel){kind=link}
)!%5BPoll){kind=link}
)!%5BTable){kind=link}
)!%5BHeatmap){kind=link}
)!%5BCyberpunk-style){kind=link}
)!%5BStatistical){kind=link}
The image shows a detailed table with risk factors affecting military scenarios, including PLA capabilities, political signals, deterrence postures, and economic conditions, with associated probability weights.](https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F0caa3fbe-2425-41c9-a387-683e1cd6989f_2614x1552.png))))))) Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts), or another podcast app.
)!%5BZvi){kind=link}
)!%5BPeter){kind=link}
)!%5BAn){kind=link}
)!%5BDefault){kind=link}
)!%5BTrademark){kind=link}
