Weights update process

The process of updating weights is one training iteration involving backpropagation. The latter can be broken down into 4 distinct steps:

  • Forward pass

In this first training step, the intention is to extract weight from the first pass of the training data into the neural network.

  • Loss function

The training loss (precision) is calculated from a loss function, e.g mean square error. On the first iteration, the loss will be high. Then, iteration after iteration, the neural network learns and the training loss (difference) between the prediction label and training label decreases.

  • Backward pass

To achieve a minimum loss, the weights need to be adjusted by taking a derivative of the loss with respect to the weights.

  • Weight update

The weights of each layer are then updated accordingly with the calculated weights from the training iterations.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.