Neural Network Mathematics: Exploring the Math behind Gradient Descent

Math    |    Expert
  • 12 videos | 1h 53m 45s
  • Includes Assessment
  • Earns a Badge
Rating 4.9 of 7 users Rating 4.9 of 7 users (7)
Because neural networks comprise thousands of neurons and interconnections, one can assume training a neural network involves millions of computations. This is where a general-purpose optimization algorithm called gradient descent comes in. Use this course to gain an intuitive and visual understanding of how gradient descent and the gradient vector work. As you advance, examine three neural network activation functions, ReLU, sigmoid, and hyperbolic tangent functions, and two variants of the ReLU function, Leaky ReLU and ELU. In examining variants of the ReLU activation function, learn how to use them to deal with deep neural network training issues. Finally, implement a neural network from scratch using TensorFlow and basic Python. When you're done, you'll be able to illustrate the mathematical intuition behind neural networks and be prepared to tackle more complex machine learning problems.

WHAT YOU WILL LEARN

  • Discover the key concepts covered in this course
    Outline how gradient descent works
    Summarize how to compute the gradient vector of partial derivatives
    Recall the characteristics of activation functions
    Illustrate step, sigmoid, and tangent activation functions
    Illustrate relu, leaky relu, and elu activation functions
  • Describe how unstable gradients can be mitigated using variants of the relu activation function
    Create a simple neural network with one neuron for regression
    Illustrate the impact of learning rate and number of epochs of training
    Illustrate the classification dataset
    Write python code from scratch to represent and train a single neuron
    Summarize the key concepts covered in this course

IN THIS COURSE

  • 2m 22s
  • 11m 21s
  • Locked
    3.  Computing Gradients
    12m 24s
  • Locked
    4.  Activation Functions
    12m 43s
  • Locked
    5.  Visualizing Common Activation Functions
    11m 33s
  • Locked
    6.  Visualizing the ReLU Function and Its Variants
    9m 3s
  • Locked
    7.  Mitigating Issues in Neural Network Training
    10m 8s
  • Locked
    8.  Simple Regression Using TensorFlow
    12m 7s
  • Locked
    9.  Learning Rate and Number of Epochs
    8m 10s
  • Locked
    10.  Exploring Datasets and Setting up Utilities
    10m 31s
  • Locked
    11.  Training a Simple Neural Network from Scratch
    11m 25s
  • Locked
    12.  Course Summary
    1m 59s

EARN A DIGITAL BADGE WHEN YOU COMPLETE THIS COURSE

Skillsoft is providing you the opportunity to earn a digital badge upon successful completion on some of our courses, which can be shared on any social network or business platform.

Digital badges are yours to keep, forever.

YOU MIGHT ALSO LIKE

Course Deep Learning for NLP: Neural Network Architectures
Rating 4.2 of 12 users Rating 4.2 of 12 users (12)
Course Artificial Intelligence and Machine Learning
Rating 4.8 of 23 users Rating 4.8 of 23 users (23)
Audiobook Evolutionary Deep Learning

PEOPLE WHO VIEWED THIS ALSO VIEWED THESE

Course Deep Learning for NLP: Memory-based Networks
Rating 3.8 of 4 users Rating 3.8 of 4 users (4)
Course Training Neural Networks: Advanced Learning Algorithms
Rating 3.8 of 6 users Rating 3.8 of 6 users (6)
Course Building Neural Networks: Development Principles
Rating 4.4 of 29 users Rating 4.4 of 29 users (29)