Lesson 3

From Deep Learning Course Wiki
Jump to: navigation, search

Lesson resources


  • Read through the following notebooks carefully:
    • lesson1.ipynb
    • lesson2.ipynb
    • sgd-intro.ipynb
    • convolution-intro.ipynb
    • lesson3.ipynb
  • Ask at least one question on the forums. Possible types of questions include:
    • What is the purpose of <something>?
    • Why do we write <something> in python?
    • Why is the output of <something> equal to <something>, instead of <something else>?
    • How do I fix this error?
    • Could I also use <some technique> for <something>?
    • Why aren't I getting a better position on the leaderboard for <competition> using <some process>?
  • Be sure that you can independently replicate the steps shown in each lesson notebook so far
  • Get a result in the top 50% of Dogs v Cats, if you haven't already
  • Get a result in the top 50% of State Farm
    • Be sure that you have created a validation set that gives similar results to submitting to Kaggle
    • Think about which layers of the pre-trained model are likely to be the most useful

CNN review

Today we reviewed all the key components of a convolutional neural network. Here are some resources you can use to help you if you are unclear on any piece:

Matrix Product (dense layers)

Convolutions (and Max-Pooling)


  • This documentation from the Stanford Course has an overview of commonly used activation functions and their pros vs. cons.

Stochastic Gradient Descent (SGD)

Backpropagation (Chain Rule)

Putting it all together