Zero to Hero — 1) Intro to Neural Networks and Backpropagation

About This Tutorial

\lim_{h o 0} \frac{f(x + h) - f(x)}{h} h 0 lim ​ h f ( x + h ) − f ( x ) ​ Differentiation and derivatives tell us how much f(x)f(x) f ( x ) changes when xx x changes. That is, they show the effect of a variable and the slope or gradient of a function. Then, if the derivative is 0, that point may be a maximum or minimum of the function - not always, but it can be. Actually, Karpathy didn't explain differentiation and derivatives in detail in the video. However, I think this is one of the most important aspects for understanding neural networks.