Lesson 3 Outline

From Deep Learning Course Wiki
Jump to: navigation, search


  • Teaching is more than 2 hours in class, the idea is to learn outside the class to the place I point them to.
  • Please Ask Questions?
    • For example why is this here, why is this not that way?
    • Even if about previous classes.
    • Remember Vincent Van. quote.
  • The Conv layer can be explained more through this tool:
    • Deep Visualization Toolbox.
    • What's going on in a CNN when the image is scanned around?
      • Edge detector
      • Texure detector
      • Text detector
  • Look at the 5 components of a ConvNet:
    • Convolutions (and Max-Pooling)
    • Activations
    • Stochastic Gradient Descent (SGD)
    • Backpropagation (Chain Rule)

Explaining ConvNets:

  • In Lesson 0, we talked about what a convolution is, using the MNIST dataset.
    • an image is a matrix of numbers. i.e: 224 x 224
    • Check this page, where the image Kernels explained.
    • How are the filters picked? (Sets of them are well-known)
  • The question is, what is the best filter to use?
    • Deep Learning does this through picking random filter, then using SGD to optimize the random values.
  • Looking at the Excel Sheet, it will allow us to see the simplest version of activation functions and Convulsions.
  • VGG in Keras explained:
    • 2d Convulsions
    • Max Pooling (reducing the resolution)
  • Softmax Explained:
    • Used for the last layer to do the classification between the one-hot vectors.

Explaining SGD

Class Part 2:

Fine Tuning VGG Net

Next week will be last class for CNNs, we will be moving on to explaining Recurrent Neural Networks (RNNs).

  • How to avoid over-fitting/underfitting?
    • Linear Model being applied to do a hard task. (Underfitting)
    • Giant model with lots of params to learn one specific image, rather than a pattern to generalize. (Over-fitting)
      • Techniques that avoid overfitting:
        • Dropout Layer.

Data Augmentation Explained:

  • Which allows us adding more data (augmented, alterted in some way) to our dataset.

Batch Normalization Explained:

Building Model For MNIST Step by Step Explained:

Related Papers: