Skip to main content

Learn Tools Games Exams Newsletter Community

Learn Tools Games Exams Newsletter Community Search

Phase 01: Math Foundations

AI From Scratch/Lesson 05/~90 minutes

Chain Rule & Automatic Differentiation

The chain rule is the engine behind every neural network that learns.

Build

Back to phaseMath Foundations Next lessonProbability and Distributions

Phase 01

Math Foundations

01Linear Algebra Intuition 02Vectors, Matrices & Operations 03Matrix Transformations 04Calculus for Machine Learning 05Chain Rule & Automatic Differentiation 06Probability and Distributions 07Bayes' Theorem 08Optimization 09Information Theory 10Dimensionality Reduction 11Singular Value Decomposition 12Tensor Operations 13Numerical Stability 14Norms and Distances 15Statistics for Machine Learning 16Sampling Methods 17Linear Systems 18Convex Optimization 19Complex Numbers for AI 20The Fourier Transform 21Graph Theory for Machine Learning 22Stochastic Processes

Loading lesson page...

Progress

0 / 22 phase lessons0%

Continue target

Phase 01

Math Foundations

01Linear Algebra Intuition 02Vectors, Matrices & Operations 03Matrix Transformations 04Calculus for Machine Learning 05Chain Rule & Automatic Differentiation 06Probability and Distributions 07Bayes' Theorem 08Optimization 09Information Theory 10Dimensionality Reduction 11Singular Value Decomposition 12Tensor Operations 13Numerical Stability 14Norms and Distances 15Statistics for Machine Learning 16Sampling Methods 17Linear Systems 18Convex Optimization 19Complex Numbers for AI 20The Fourier Transform 21Graph Theory for Machine Learning 22Stochastic Processes

On this page

Resources

3Blue1Brown: Backpropagation calculusReading PyTorch Autograd mechanicsReading Baydin et al., Automatic Differentiation in Machine Learning: a SurveyReading AutodiffSkill autodiff.pyCode main.jlCode