Neural Network Mathematics: Exploring the Math behind Gradient Descent
Math
| Expert
- 12 Videos | 1h 53m 45s
- Includes Assessment
- Earns a Badge
Because neural networks comprise thousands of neurons and interconnections, one can assume training a neural network involves millions of computations. This is where a general-purpose optimization algorithm called gradient descent comes in. Use this course to gain an intuitive and visual understanding of how gradient descent and the gradient vector work. As you advance, examine three neural network activation functions, ReLU, sigmoid, and hyperbolic tangent functions, and two variants of the ReLU function, Leaky ReLU and ELU. In examining variants of the ReLU activation function, learn how to use them to deal with deep neural network training issues. Finally, implement a neural network from scratch using TensorFlow and basic Python. When you're done, you'll be able to illustrate the mathematical intuition behind neural networks and be prepared to tackle more complex machine learning problems.
WHAT YOU WILL LEARN
-
discover the key concepts covered in this courseoutline how gradient descent workssummarize how to compute the gradient vector of partial derivativesrecall the characteristics of activation functionsillustrate step, sigmoid, and tangent activation functionsillustrate ReLU, Leaky ReLU, and ELU activation functions
-
describe how unstable gradients can be mitigated using variants of the ReLU activation functioncreate a simple neural network with one neuron for regressionillustrate the impact of learning rate and number of epochs of trainingillustrate the classification datasetwrite Python code from scratch to represent and train a single neuronsummarize the key concepts covered in this course
IN THIS COURSE
-
1.Course Overview2m 22sUP NEXT
-
2.The Intuition behind Gradient Descent11m 21s
-
3.Computing Gradients12m 24s
-
4.Activation Functions12m 43s
-
5.Visualizing Common Activation Functions11m 33s
-
6.Visualizing the ReLU Function and Its Variants9m 3s
-
7.Mitigating Issues in Neural Network Training10m 8s
-
8.Simple Regression Using TensorFlow12m 7s
-
9.Learning Rate and Number of Epochs8m 10s
-
10.Exploring Datasets and Setting up Utilities10m 31s
-
11.Training a Simple Neural Network from Scratch11m 25s
-
12.Course Summary1m 59s
EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE
Skillsoft is providing you the opportunity to earn a digital badge upon successful completion of this course, which can be shared on any social network or business platform
Digital badges are yours to keep, forever.