Bit by Bit
← All courses
Deep Learning · Intermediate

Neural Networks

One neuron draws one straight line. Stack neurons in layers and they can learn almost any shape. Train a network live, watch it bend its boundary, and write the core calculations yourself in Python.

4 lessons28 minutesFree
Start this course →

Lessons

  1. 1Why one neuron is not enoughSee the limit of a single neuron and how a hidden layer removes it.7 min
  2. 2Activation functionsUnderstand the small function that makes deep networks possible.6 min
  3. 3The forward passTrace a prediction through a network by hand, then in code.7 min
  4. 4How a network learnsSee training as a loop of forward pass, error and correction.8 min

What you will learn

Terms explained in this course

layer
A group of neurons that all receive the same inputs and work side by side.
input layer
The first layer of a network. It simply holds the input values, one per feature.
hidden layer
A layer between the input and the output. Its neurons find useful intermediate patterns.
output layer
The last layer of a network. It produces the prediction.
forward pass
Running an input through the network, layer by layer, to get a prediction.
linearly separable
Describes two groups of points that a single straight line can separate.
non-linearity
Anything that is not a straight line. Activation functions add it, which lets a network learn curved boundaries.
sigmoid
An activation function that squeezes any number smoothly into the range 0 to 1.
ReLU
An activation function that outputs 0 for negative inputs and passes positive inputs through unchanged. Short for rectified linear unit.
tanh
An activation function that squeezes any number smoothly into the range −1 to 1.
backpropagation
The method that works out, for every weight in a network, how much it contributed to the error. It passes the error backwards from the output layer to the input layer.
epoch
One full pass through all the training examples.
loss curve
A chart of the loss as training goes on. A healthy curve falls and then flattens.
function
A named, reusable piece of code. You give it inputs and it returns a result.