Multiclass Classification Neural Network

Neural Network

Example Neural Network

Creating The Model

Our Neural Network can be modeled as a class with the following attributes and methods:

  • Architecture (specifies the number of nodes in each layer)
  • A list of connections consisting of the weights, bias, and activation function needed to compute the value of each node.
  • A forward propagation and back propagation method.

Forward Propagation

Forward propagation consists of computing the value of each node in the Neural Network. Starting at the input layer and moving towards the output layer. The value of each node can be represented as a linear combination transformed by some activation function (g).


In training a Neural Network the goal is to find the
values of the weights and biases that minimize some cost function J.

Cross Entropy Cost Function
Weights update rule
Biases update rule

Backpropagation Algorithm:

Starting at the output Layer

  • From Deltas Compute update matrix
  • Update Weights and Biases
  • Move left 1 layer

Training The Model

Now that we have implemented the Forward Propagation and Back Propagation methods, we can now create our neural network and train the model via stochastic gradient descent.

Further Reading

Machine Learning: An Applied Mathematics Introduction by Paul Wilmott.



Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store