Neural Networks
One neuron draws one straight line. Stack neurons in layers and they can learn almost any shape. Train a network live, watch it bend its boundary, and write the core calculations yourself in Python.
Start this course →Lessons
- 1Why one neuron is not enoughSee the limit of a single neuron and how a hidden layer removes it.7 min
- 2Activation functionsUnderstand the small function that makes deep networks possible.6 min
- 3The forward passTrace a prediction through a network by hand, then in code.7 min
- 4How a network learnsSee training as a loop of forward pass, error and correction.8 min
What you will learn
- A single neuron can only separate groups that are linearly separable.
- A hidden layer lets the network combine several lines into a shape.
- Running an input through the layers is called the forward pass.
- Without a non-linearity, any stack of layers collapses into one straight line.
- Sigmoid, tanh and ReLU are the common activation functions.
- Smooth or sloped activations give gradient descent a slope to follow.
- A forward pass computes each layer from the one before it.
- Every neuron does the same thing: weighted sum, plus bias, then activation.
- A network is this small calculation repeated many times.
- Training repeats: forward pass, measure loss, backpropagation, update weights.
- Backpropagation shares out the blame for the error among all the weights.
- An epoch is one pass through the data, and the loss curve shows progress.
Terms explained in this course
- layer
- A group of neurons that all receive the same inputs and work side by side.
- input layer
- The first layer of a network. It simply holds the input values, one per feature.
- hidden layer
- A layer between the input and the output. Its neurons find useful intermediate patterns.
- output layer
- The last layer of a network. It produces the prediction.
- forward pass
- Running an input through the network, layer by layer, to get a prediction.
- linearly separable
- Describes two groups of points that a single straight line can separate.
- non-linearity
- Anything that is not a straight line. Activation functions add it, which lets a network learn curved boundaries.
- sigmoid
- An activation function that squeezes any number smoothly into the range 0 to 1.
- ReLU
- An activation function that outputs 0 for negative inputs and passes positive inputs through unchanged. Short for rectified linear unit.
- tanh
- An activation function that squeezes any number smoothly into the range −1 to 1.
- backpropagation
- The method that works out, for every weight in a network, how much it contributed to the error. It passes the error backwards from the output layer to the input layer.
- epoch
- One full pass through all the training examples.
- loss curve
- A chart of the loss as training goes on. A healthy curve falls and then flattens.
- function
- A named, reusable piece of code. You give it inputs and it returns a result.