From logic gates to GPUs: a visual tour of how chips are built
This preview is based only on the published show notes. The episode appears to be a blackboard-style lecture with Reiner Pope that starts from basic logic gates and builds up to TPUs, GPUs, FPGAs, memory tradeoffs, and even comparisons with the brain.
This episode looks like a strong pick if you want a first-principles walkthrough of chip design rather than a newsy industry conversation. Based on the show notes, Reiner Pope takes a bottom-up approach: starting with simple logic gates, then building toward multiply-accumulate units, data movement, systolic arrays, pipelines, and the architectural differences between CPUs, GPUs, TPUs, and FPGAs. The notes suggest a very visual, engineering-heavy format. Since it’s described as a blackboard lecture, this seems especially useful for listeners who like seeing concepts built step by step instead of hearing only high-level analogies. The recommendation to watch on YouTube so you can see the chalkboard is a good clue that diagrams and visual reasoning are central to the episode. Reiner’s background also hints at the angle: he’s the CEO of MatX and previously worked at Google on software efficiency, compilers, and TPU architecture. That makes this sound less like a broad business discussion and more like a practical explanation of why modern compute systems “look the way they do.” If the topic list appeals to you, the episode seems to cover several especially relevant questions for AI hardware: the cost of moving data, how systolic arrays work, FPGA versus ASIC tradeoffs, cache versus scratchpad, and why CPU cores are much larger than GPU cores. The final sections also appear to widen the lens by comparing brains and chips, and by framing a GPU in relation to many tiny TPUs. In short: this preview suggests a dense but accessible hardware primer for technically curious listeners, especially those interested in AI compute.
About this episode
<p>New blackboard lecture with Reiner Pope: how do chips actually work - starting with basic logic gates, and working up to why GPUs, TPUs, FPGAs, and the human brain each look the way they do.</p><p><a target="_blank" href="https://reiner.org/">Reiner</a> is CEO of <a target="_blank" href="https://matx.com/">MatX</a>, a new chip startup (full disclosure - I’m an angel investor). He was previously at Google, where he worked on <a target="_blank" href="https://arxiv.org/abs/2211.05102">software</a> <a target="_blank" href="https://jax-ml.github.io/scaling-book/">efficiency</a>, compilers, and TPU architecture.</p><p>Watch this one on <a target="_blank" href="https://youtu.be/oIk3R-sMX5o">YouTube</a> so you can see the chalkboard. Read the <a target="_blank" href="https://www.dwarkesh.com/p/reiner-pope-2">transcript</a>.</p><p>Sponsors</p><p>* <a target="_blank" href="https://crusoe.ai/dwarkesh">Crusoe</a> was one of only five GPU clouds that made the gold tier in SemiAnalysis' most recent ClusterMAX report. Gold-tier providers like Crusoe delivered 5-15% lower TCO than silver-tier clouds, even with identical GPU pricing. This is because optimizations like early fault detection and rapid node replacement don't necessarily show up in the sticker price, but still matter a ton in the real world. Learn more at <a target="_blank" href="https://crusoe.ai/dwarkesh">crusoe.ai/dwarkesh</a></p><p>* <a target="_blank" href="https://cursor.com/dwarkesh">Cursor</a> is where I do most of my work—from reading research papers to visualizing technical concepts to coding up internal tools for the podcast. Most recently, I used it to build two different review interfaces for my essay contest, one that anonymizes submissions for scoring and another that lets me see applicants' essays next to their resumes and websites. Whatever you're working on, you should try doing it in Cursor. Get started at <a target="_blank" href="https://cursor.com/dwarkesh">cursor.com/dwarkesh</a></p><p>* <a target="_blank" href="https://janestreet.com/dwarkesh">Jane Street</a> let me ask Ron Minsky and Dan Pontecorvo, two senior Jane Streeters, a bunch of questions about how they use AI. We discussed everything from the types of models they're training to how they think about the future of trading to why they're more bullish than ever on hiring technical talent. You can watch the full conversation and learn more about their open positions at <a target="_blank" href="https://janestreet.com/dwarkesh">janestreet.com/dwarkesh</a></p><p>Timestamps</p><p>00:00:00 – Building a multiply-accumulate from logic gates</p><p>00:16:31 – Muxes and the cost of data movement</p><p>00:26:10 – How systolic arrays work</p><p>00:39:11 – Clock cycles and pipeline registers</p><p>00:51:51 – FPGAs vs ASICs</p><p>01:03:25 – Cache vs scratchpad</p><p>01:07:27 – Why CPU cores are much bigger than GPU cores</p><p>01:12:00 – Brains vs chips</p><p>01:15:33 – A GPU is just a bunch of tiny TPUs</p> <br/><br/>Get full access to Dwarkesh Podcast at <a href="https://www.dwarkesh.com/subscribe?utm_medium=podcast&utm_campaign=CTA_4">www.dwarkesh.com/subscribe</a>