Best of Machine Learning Resources

A repository for sharing the best resources for learning the current state of the art in machine learning. Suggested by my friend, Tommy Unger.

Soft Introduction to Machine Learning & Neural Networks

---

Video Tutorials and Talks:

Purview of landscape

• The Deep End of Deep Learning | Hugo Larochelle | TEDxBoston
• How we teach computers to understand pictures | Fei Fei Li
• Deep Learning and the Future of AI | Yann LeCun | Talk 1/2
• Deep Learning and the Future of AI | Yann LeCun | Q&A 2/2

Application

• Ian Goodfellow, Google - Practical Methodology for Deploying Machine Learning #AIWTB Oct 2015

Convolutional Neural Networks

• How Convolutional Neural Networks work
• Convolutionlal Visualizer

Recurrent Neural Networks

• Nervana - Recurrent Neural Networks

Playlists

• MIT OCW - 6.034 lecture 12a
• MIT OCW - 6.034 lecture 12b
• Deep Learning SIMPLIFIED
• Neural Networks Demystified
• Deep Learning | Udacity

Visualizations:

• ConvNetJS MNIST demo
• Visual Analysis for Recurrent Neural Networks

Readings:

General Theory

• Understanding the Bias-Variance Tradeoff
• A Tutorial on Deep Learning Part 1: Nonlinear Classifiers and The Backpropagation Algorithm
• A Tutorial on Deep Learning Part 2: Autoencoders, Convolutional Neural Networks and Recurrent Neural Networks
• Sparse Autoencoders
• Neural Network Zoo

Convolutional Neural Networks

• A guide to convolution arithmetic for deep learning:
• Convolutional Neural Networks (CNNs): An Illustrated Explanation

Recurrent Neural Networks

• The Unreasonable Effectiveness of Recurrent Neural Networks

Advanced Machine Learning & Neural Networks

---

Neural Network Theory

Books

• Deep Learning
• Elements of Statistical Learning

Courses

Stanford

CS229 - Machine Learning

• video lectures
• lecture notes

CS231n - Convolutional Neural Networks for Visual Recognition

• video lectures
• lecture notes

CS224d - Deep Learning for Natural Language Processing 2016

• video lectures
• lecture notes

Oxford

Deep learning at Oxford 2015

• video lectures
• lecture notes

University of California, Berkeley

CS294 - Deep Reinforcement Learning 2015

• lecture notes

Math

• Calc on Computational Graphs
• Linear Xforms
• Notes on Minsky's Perceptrons
• Visual Information Theory
• Automatic Differentiation
• AD on Wikipedia
• Non-convex Optimization Blog
• Yes you should understand backprop

Theorems

• No Free Lunch Theorem

There is no one best optimization algorithm.

• Universal Approximation Theorem

A neural network with at least two hidden layers using any activation function can approximate any function to an arbitrary accuracy given appropriate parameters.

• Vitushkin's Theorem

There exist differentiable functions of arbitrarily many variables.

Tutorials

• Gentle Tensorflow Intro
• Hacker's guide to Neural Networks
• A Neural Network in 11 lines of Python
• Stanford's Unsupervised Feature Learning and Deep Learning
• Hugo Larochelle's Neural Network video series
• Neural Networks and Deep Learning: Awesome Ebook
• Rojas Neural Networks: A Systematic Introduction
• Andrej Karpathy's CS231n Convolutional Neural Networks for Visual Recognition
• Wild ML: Lots of deep learning writeups
• Anyone Can Learn To Code an LSTM-RNN in Python
• How to Code and Understand DeepMind's Neural Stack Machine
• The Unreasonable Effectiveness of Recurrent Neural Networks
• Char RNN

Academic Research Lab Publications

• Montréal Institute for Learning Algorithms
• University of Central Florida Evolutionary Complexity Research Group
• University of Wyoming Evolving AI Lab
• Stanford NLP
• Stanford Vision Lab
• NYU Computational Intelligence, Learning, Vision, and Robotics
• Harvard NLP

Industrial Research Lab Publications

• Google
• DeepMind
• Facebook
• IBM DeepQA
• Baidu: Institute of Deep Learning
• Baidu: Big Data Lab
• Baidu: Silicon Valley AI Lab

Conferences

• Open Review
• ICLR (International Conference on Learning Representations)
• NIPS (Neural Information Processing Systems) Conference

Neural Network Application

Deep Learning Framework Comparison

• Comparison: Blessed by LeCun

Frameworks/Toolkits/Libraries

• TensorFlow
• Torch Demos
• Theano
• Keras
• Caffe
• MXNET
• Microsoft Cognitive Toolkit
• Chainer
• Stanford CoreNLP
• Scikit-learn
• DeepMind Lab
• OpenAI Gym
• OpenAI Universe
• DeepLearning4J

Torch Tutorials

• Brief Lua Tutorial + Softmax Classifier Hello World
• Minimal Hello World NN Tutorials
• Torch Overview Slides, Many Topics
• [Torch Documentation Template](http://jucor.github.io/t