# Lesson 3 Outline

From Deep Learning Course Wiki

## Contents

## Intro:

- Teaching is more than 2 hours in class, the idea is to learn outside the class to the place I point them to.
- Notebooks of the class.
- Try changing the input/outputs, perhaps try rebuilding the model yourself.
- Read the rest of the material other than just the code to be ran.
- For example the paper Visualizing and Understanding Convolutional Networks by Matthew D Zeiler.

- Please Ask Questions?
- For example why is this here, why is this not that way?
- Even if about previous classes.
- Remember Vincent Van. quote.

- The Conv layer can be explained more through this tool:
- Deep Visualization Toolbox.
- What's going on in a CNN when the image is scanned around?
- Edge detector
- Texure detector
- Text detector

- Look at the 5 components of a ConvNet:
- Convolutions (and Max-Pooling)
- Activations
- Stochastic Gradient Descent (SGD)
- Backpropagation (Chain Rule)

## Explaining ConvNets:

- In Lesson 0, we talked about what a convolution is, using the MNIST dataset.
- an image is a matrix of numbers. i.e: 224 x 224
- Check this page, where the image Kernels explained.
- How are the filters picked? (Sets of them are well-known)

- The question is, what is the best filter to use?
- Deep Learning does this through picking random filter, then using SGD to optimize the random values.

- Looking at the Excel Sheet, it will allow us to see the simplest version of activation functions and Convulsions.

- VGG in Keras explained:
- 2d Convulsions
- Max Pooling (reducing the resolution)

- Softmax Explained:
- Used for the last layer to do the classification between the one-hot vectors.

## Explaining SGD

## Class Part 2:

## Fine Tuning VGG Net

Next week will be last class for CNNs, we will be moving on to explaining Recurrent Neural Networks (RNNs).

- How to avoid over-fitting/underfitting?
- Linear Model being applied to do a hard task. (Underfitting)
- Giant model with lots of params to learn one specific image, rather than a pattern to generalize. (Over-fitting)
- Techniques that avoid overfitting:
- Dropout Layer.

- Techniques that avoid overfitting:

## Data Augmentation Explained:

- Which allows us adding more data (augmented, alterted in some way) to our dataset.

## Batch Normalization Explained:

## Building Model For MNIST Step by Step Explained:

## Related Papers:

- Over-fitting and Underfitting and other relevant ML terminologies explained here: A Few Useful Things to Know about Machine Learning
- Matthew D Zeiler's paper: Visualizing and Understanding Convolutional Networks
- The dynamics behind Batch Normalization: Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift