From Deep Learning Course Wiki
- Lesson 3 video and associated Lesson 3 Video Timeline
- Lesson Notes
- Lesson 3 notebook
- MNIST walk-through notebook
- Read through the following notebooks carefully:
- Ask at least one question on the forums. Possible types of questions include:
- What is the purpose of <something>?
- Why do we write <something> in python?
- Why is the output of <something> equal to <something>, instead of <something else>?
- How do I fix this error?
- Could I also use <some technique> for <something>?
- Why aren't I getting a better position on the leaderboard for <competition> using <some process>?
- Be sure that you can independently replicate the steps shown in each lesson notebook so far
- We suggest following the guidance: How to use the Provided Notebooks
- Get a result in the top 50% of Dogs v Cats, if you haven't already
- Get a result in the top 50% of State Farm
- Be sure that you have created a validation set that gives similar results to submitting to Kaggle
- Think about which layers of the pre-trained model are likely to be the most useful
Today we reviewed all the key components of a convolutional neural network. Here are some resources you can use to help you if you are unclear on any piece:
Matrix products (dense layers)
- Khan Academy - Good introduction to the basics
- 3 Blue 1 Brown videos - Excellent videos with lots of visualizations and explains the intuition behind matrix products.
Convolutions (and Max-Pooling)
- Image kernels explained visually - fantastic interactive visualizations of convolutions
- Understanding convolutions - a deep dive into convolutions, looking not just at image processing applications, but a more general approach
- Jeremy's convolution notebook motivates and builds convolutions from scratch, as well as introducing max-pooling
- This blog post is a nice introductory overview to CNNs, including convolutions and max-pooling.
- This documentation from the Stanford Course has an overview of commonly used activation functions and their pros vs. cons.
- Jeremy's jupyter notebook walking through using stochastic gradient descent for linear regression, with animation.
- Detailed tutorials explaining and implementing in Python gradient descent and stochastic gradient descent.
- An animation of gradient descent (not stochastic).
Backpropagation (Chain Rule)
- Chris Olah's tutorial on backpropagation, which may resonate with those with CS backgrounds.
- Khan Academy lesson on the Chain Rule.