# Neural Network Mathematics: Exploring the Math behind Gradient Descent

Math    |    Expert
• 12 videos | 1h 53m 45s
• Includes Assessment
Rating 4.9 of 7 users (7)
Because neural networks comprise thousands of neurons and interconnections, one can assume training a neural network involves millions of computations. This is where a general-purpose optimization algorithm called gradient descent comes in. Use this course to gain an intuitive and visual understanding of how gradient descent and the gradient vector work. As you advance, examine three neural network activation functions, ReLU, sigmoid, and hyperbolic tangent functions, and two variants of the ReLU function, Leaky ReLU and ELU. In examining variants of the ReLU activation function, learn how to use them to deal with deep neural network training issues. Finally, implement a neural network from scratch using TensorFlow and basic Python. When you're done, you'll be able to illustrate the mathematical intuition behind neural networks and be prepared to tackle more complex machine learning problems.

## WHAT YOU WILL LEARN

• Discover the key concepts covered in this course
Summarize how to compute the gradient vector of partial derivatives
Recall the characteristics of activation functions
Illustrate step, sigmoid, and tangent activation functions
Illustrate relu, leaky relu, and elu activation functions
• Describe how unstable gradients can be mitigated using variants of the relu activation function
Create a simple neural network with one neuron for regression
Illustrate the impact of learning rate and number of epochs of training
Illustrate the classification dataset
Write python code from scratch to represent and train a single neuron
Summarize the key concepts covered in this course

## IN THIS COURSE

• 4.  Activation Functions
• 5.  Visualizing Common Activation Functions
• 6.  Visualizing the ReLU Function and Its Variants
• 7.  Mitigating Issues in Neural Network Training
• 8.  Simple Regression Using TensorFlow
• 9.  Learning Rate and Number of Epochs
• 10.  Exploring Datasets and Setting up Utilities
• 11.  Training a Simple Neural Network from Scratch
• 12.  Course Summary

## EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

## YOU MIGHT ALSO LIKE

Rating 3.8 of 4 users (4)

## PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Rating 4.4 of 36 users (36)
Rating 4.5 of 2 users (2)
Rating 4.8 of 4 users (4)