AI From Scratch/Phase 19/Lesson 33/~90 minutes

Multi-Head Self-Attention

BuildPython

One linear projection, three views, H parallel heads, one mask. The attention block as the model actually uses it.

Loading lesson page...