Flashcards for the Dwarkesh Podcast.

Blog posts & side projects

• Pretraining parallelisms and failed runs

  18 cards

  Why pretraining runs fail, and the chain of problems and fixes behind FSDP / pipeline / tensor parallelism.

• Transistors

  16 cards

  Diodes, BJTs, MOSFETs, Dennard scaling, and why silicon won.

• Chips

  9 cards

  Multiplexers, adders, MACs, RTL-to-GDS, and the SRAM vs. register file trade-off.