AI From Scratch/Phase 19/Lesson 31/~90 minutes

Tokenized Dataset with Sliding Window

BuildPython

A pretraining run is a function from token ids to gradients. This lesson builds the conveyor that feeds the ids in.

Loading lesson page...