About this episode
<p>New episode with my good friends <a target="_blank" href="https://x.com/_sholtodouglas?lang=en">Sholto Douglas</a> & <a target="_blank" href="https://x.com/trentonbricken?lang=en">Trenton Bricken</a>. Sholto focuses on scaling RL and Trenton researches mechanistic interpretability, both at Anthropic.</p><p>We talk through what’s changed in the last year of AI research; the new RL regime and how far it can scale; how to trace a model’s thoughts; and how countries, workers, and students should prepare for AGI.</p><p>See you next year for v3. <a target="_blank" href="https://www.youtube.com/watch?v=UTuuTTnjxMQ">Here’s last year’s episode</a>, btw. Enjoy!</p><p>Watch on <a target="_blank" href="https://youtu.be/64lXQP6cs5M">YouTube</a>; listen on <a target="_blank" href="https://podcasts.apple.com/us/podcast/dwarkesh-podcast/id1516093381?i=1000709488338">Apple Podcasts</a> or <a target="_blank" href="https://open.spotify.com/episode/3H46XEWBlUeTY1c1mHolqh?si=b645971b1af546fa">Spotify</a>.</p><p>----------</p><p>SPONSORS</p><p>* <a target="_blank" href="https://workos.com/">WorkOS</a> ensures that AI companies like OpenAI and Anthropic don't have to spend engineering time building enterprise features like access controls or SSO. It’s not that they don't need these features; it's just that WorkOS gives them battle-tested APIs that they can use for auth, provisioning, and more. Start building today at <a target="_blank" href="https://workos.com/">workos.com</a>.</p><p>* <a target="_blank" href="https://scale.com/dwarkesh">Scale</a> is building the infrastructure for safer, smarter AI. Scale’s Data Foundry gives major AI labs access to high-quality data to fuel post-training, while their public leaderboards help assess model capabilities. They also just released Scale Evaluation, a new tool that diagnoses model limitations. If you’re an AI researcher or engineer, learn how Scale can help you push the frontier at <a target="_blank" href="https://scale.com/dwarkesh">scale.com/dwarkesh</a>.</p><p>* <a target="_blank" href="https://www.lighthousehq.com/ref/Dwarkesh">Lighthouse</a> is THE fastest immigration solution for the technology industry. They specialize in expert visas like the O-1A and EB-1A, and they’ve already helped companies like Cursor, Notion, and Replit navigate U.S. immigration. Explore which visa is right for you at <a target="_blank" href="https://www.lighthousehq.com/ref/Dwarkesh">lighthousehq.com/ref/Dwarkesh</a>.</p><p>To sponsor a future episode, visit <a target="_blank" href="https://www.dwarkesh.com/advertise">dwarkesh.com/advertise</a>.</p><p>----------</p><p>TIMESTAMPS</p><p>(00:00:00) – How far can RL scale?</p><p>(00:16:27) – Is continual learning a key bottleneck?</p><p>(00:31:59) – Model self-awareness</p><p>(00:50:32) – Taste and slop</p><p>(01:00:51) – How soon to fully autonomous agents?</p><p>(01:15:17) – Neuralese</p><p>(01:18:55) – Inference compute will bottleneck AGI</p><p>(01:23:01) – DeepSeek algorithmic improvements</p><p>(01:37:42) – Why are LLMs ‘baby AGI’ but not AlphaZero?</p><p>(01:45:38) – Mech interp</p><p>(01:56:15) – How countries should prepare for AGI</p><p>(02:10:26) – Automating white collar work</p><p>(02:15:35) – Advice for students</p> <br/><br/>Get full access to Dwarkesh Podcast at <a href="https://www.dwarkesh.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4">www.dwarkesh.com/subscribe</a>