Why Neural Networks can learn (almost) anything

A video about neural networks, how they work, and why they're useful. My twitter: https://twitter.com/max_romana SOURCES Neural network playground: https://playground.tensorflow.org/ Universal Function Approximation: Proof: https://cognitivemedium.com/magic_paper/assets/Hornik.pdf Covering ReLUs: https://proceedings.neurips.cc/paper/2017/hash/32cbf687880eb1674a07bf717761dd3a-Abstract.html Covering discontinuous functions: https://arxiv.org/pdf/2012.03016.pdf Turing Completeness: Networks of infinite size are turing complete: Neural Computability I & II (behind a paywall unfourtunately, but is cited in following paper) RNNs are turing complete: https://binds.cs.umass.edu/papers/1992_Siegelmann_COLT.pdf Transformers are turing complete: https://arxiv.org/abs/2103.05247 More on backpropagation: https://www.youtube.com/watch?v=Ilg3gGewQ5U More on the mandelbrot set: https://www.youtube.com/watch?v=NGMRB4O922I Additional Sources: Neat explanation of universal function approximation proof: https://www.youtube.com/watch?v=Ijqkc7OLenI Where I got the hard coded parameters: https://towardsdatascience.com/can-neural-networks-really-learn-any-function-65e106617fc6 Reviewers: Andrew Carr https://twitter.com/andrew_n_carr Connor Christopherson TIMESTAMPS (0:00) Intro (0:27) Functions (2:31) Neurons (4:25) Activation Functions (6:36) NNs can learn anything (8:31) NNs can't learn anything (9:35) ...but they can learn a lot MUSIC https://www.youtube.com/watch?v=SmkUY_B9fGg