This is a step-by-step course in getting the most out of deep learning models on your own predictive modeling projects.
About This Video
In Detail
Deep learning neural networks …
This is a step-by-step course in getting the most out of deep learning models on your own predictive modeling projects.
About This Video
In Detail
Deep learning neural networks have become easy to create. However, tuning these models for maximum performance remains something of a challenge for most modelers. This course will teach you how to get results as a machine learning practitioner.
The course starts with an introduction to the problem of overfitting and a tour of regularization techniques. Learn through better configured stochastic gradient descent batch size, loss functions, learning rates, and to avoid exploding gradients via gradient clipping. After that, you'll learn regularization techniques and reduce overfitting by updating the loss function using techniques such as weight regularization, weight constraints, and activation regularization. Post that, you'll effectively apply dropout, the addition of noise, and early stopping, and combine the predictions from multiple models.
You'll also look at ensemble learning techniques and diagnose poor model training and problems such as premature convergence and accelerate the model training process. Then, you'll combine the predictions from multiple models saved during a single training run with techniques such as horizontal ensembles and snapshot ensembles.
Finally, you'll diagnose high variance in a final model and improve the average predictive skill.
By the end of this course, you'll learn different techniques for getting better results with deep learning models.
Who this book is for
This course is for developers, machine learning engineers, and data scientists that want to enhance the performance of their deep learning models. This is an intermediate level to advanced level course. It's highly recommended that the learner be proficient in Python, Keras, and machine learning.
A solid foundation in machine learning, deep learning, and Python is required to get better results out of this course. You are also recommended to have the core machine learning libraries in Python.
Chapter 1 : Introduction to the Course
Introduction
Course Overview
Is This Course Right for You?
Course Structure
Neural Network Defined
Framework for Optional Learning
Optimal Generalization Techniques
Optimal Prediction Techniques
Framework Application
Diagnostic Learning Curves
The Fit of the Model
Unrepresentative Dataset
Chapter 2 : Optimal Learning
Neural Networks: Learn a Mapping Function
Error Surface
Features of the Error Surface
Non-Convex Error Surface
Deep Learning Neural Network Components: Part 1
Deep Learning Neural Network Components: Part 2
Neural Network Model Capacity
Anatomy of a Keras Model
Demo: Case Study on Model Capacity: Part 1
Demo: Case Study on Model Capacity: Part 2
Demo: Case Study on Model Capacity: Part 3
Gradient Precision with Batch Size
Demo: Case Study on Batch Size: Part 1
Demo: Case Study on Batch Size: Part 2
Demo: Case Study on Batch Size: Part 3
Loss Function Defined
Choosing a Loss Function
Demo: Case Study on Regression Loss Functions: Part 1
Demo: Case Study on Regression Loss Functions: Part 2
Demo: Case Study on Binary Classification Loss Functions: Part 1
Demo: Case Study on Binary Classification Loss Functions: Part 2
Demo: Case Study on Binary Classification Loss Functions: Part 3
Demo: Case Study on Multiclass Classification Loss Functions: Part 1
Demo: Case Study on Multiclass Classification Loss Functions: Part 2
Learning Rate Defined
Configuring the Learning Rate
Learning Rate Schedules and Adaptive Learning Rates
Defining Learning Rates in Keras
Demo: Case Study on Learning Rates: Part 1
Demo: Case Study on Learning Rates: Part 2
Demo: Case Study on Learning Rates: Part 3
Demo: Case Study on Learning Rates: Part 4
Data Scaling
Scaling the Input and Output Variables
Normalize and Standardize (Rescaling)
Demo: Case Study on Data Scaling: Part 1
Demo: Case Study on Data Scaling: Part 2
Demo: Case Study on Data Scaling: Part 3
Demo: Case Study on Data Scaling: Part 4
Activation Functions and Vanishing Gradients
Rectified Linear Activation Function Defined and Implemented in Python
When ReLU is the Appropriate Choice
Demo: Case Study on Vanishing Gradients: Part 1
Demo: Case Study on Vanishing Gradients: Part 2
Correct Exploding Gradients with Clipping
Gradient Clipping in Keras
Demo: Case Study on Exploding Gradients Part 1
Demo: Case Study on Exploding Gradients Part 2
Batch Normalization
Tips for Applying Batch Normalization
Demo: Case Study on Batch Normalization: Part 1
Demo: Case Study on Batch Normalization: Part 2
Demo: Greedy Layer-Wise Pretraining Case Study: Part 1
Demo: Greedy Layer-Wise Pretraining Case Study: Part 2
Chapter 3 : Optimal Generalization
The Problem of Overfitting
Reduce Overfitting by Constraining Complexity
Regularization Approaches for Neural Networks
Penalize Large Weights via Regularization
How to Penalize Large Weights
Tips for Using Weight Regularization
Demo: Weight Regularization Case Study: Part 1
Demo: Weight Regularization Case Study: Part 2
Activity Regularization
Encouraging Smaller Activations
Tips for Activity Regularization
Activity Regularization in Keras
Demo: Activity Regularization Case Study
Forcing Small Weights
How to Use a Weight Constraint
Tips for Applying Weight Constraints
Weight Constraints in Keras
Demo: Weight Constraint Case Study
Dropout
Dropout Mechanics
Dropout Tips
Dropout in Keras
Demo: Dropout Case Study
Noise Regularization
How to Add Noise
Noise Tips
Adding Noise in Keras
Demo: Noise Regularization Case Study
Chapter 4 : Optimal Predictions
Ensemble Learning
Ensemble Neural Network Models
Varying the Major Elements
Model Averaging Ensembles
Ensembles in Keras
Demo: Model Averaging Ensemble Case Study: Part 1
Demo: Model Averaging Ensemble Case Study: Part 2
Demo: Model Averaging Ensemble Case Study: Part 3
Weighted Average Ensembles
Demo: Weighted Average Ensemble Case Study: Part 1
Demo: Weighted Average Ensemble Case Study: Part 2
Demo: Weighted Average Ensemble Case Study: Part 3
Demo: Weighted Average Ensemble Case Study: Part 4
Resampling Ensembles
Demo: Resampling Ensemble Case Study: Part 1
Demo: Resampling Ensemble Case Study: Part 2
Demo: Resampling Ensemble Case Study: Part 3
Demo: Resampling Ensemble Case Study: Part 4
Horizontal Voting Ensembles
Demo: Horizontal Ensemble Case Study: Part 1
Demo: Horizontal Ensemble Case Study: Part 2