Bit by Bit
← All courses
Machine Learning · Beginner

How Machines Learn

Data, models, error and the downhill walk that trains almost every AI system. You will fit a line, train by gradient descent, tune a neuron and catch a model that memorised its homework.

5 lessons33 minutesFree
Start this course →

Lessons

  1. 1Data: the raw materialRead a dataset the way a learning program does.5 min
  2. 2A model is a machine with knobsFit a line to data by hand and measure how wrong it is.7 min
  3. 3Gradient descent: walking downhillWatch a computer train itself, and learn why step size matters.7 min
  4. 4One artificial neuronBuild the unit that every neural network is made of.7 min
  5. 5Overfitting: memorising is not learningSpot a model that looks perfect in training and fails in real use.7 min

What you will learn

Terms explained in this course

dataset
A collection of examples, usually arranged as a table with one row per example.
supervised learning
Machine learning from examples that come with the correct answer (a label) attached.
regression
Predicting a number, such as a price or a temperature.
classification
Predicting a category, such as "spam" or "not spam".
parameter
An adjustable number inside a model. Training means finding good values for the parameters.
linear model
A model that predicts with a straight line: prediction = slope × input + intercept.
slope
How much the prediction rises when the input goes up by one. It sets how steep the line is.
intercept
The prediction when the input is zero. It sets where the line starts.
error
The gap between a model's prediction and the correct answer.
loss
A single number that says how wrong a model is across all its examples. Lower is better.
mean squared error
A common loss. Square each error, then take the average. Often shortened to MSE.
gradient
The slope of the loss at the current parameter values. It points in the direction that makes the loss grow fastest.
gradient descent
A training method: repeatedly nudge each parameter a little in the direction that lowers the loss.
learning rate
A number that sets how big each gradient descent step is.
iteration
One repeat of a process. In training, one update of the parameters.
convergence
The point where more training steps no longer change the loss in any meaningful way.
neuron
The basic unit of a neural network. It multiplies each input by a weight, adds the results and a bias term, then applies an activation function.
weight
A parameter that sets how strongly one input influences a neuron.
bias term
A parameter added to a neuron's total. It shifts the point at which the neuron switches on. It has nothing to do with unfairness, which is a different meaning of the word bias.
activation function
The final step in a neuron, which turns its total into an output. The simplest one gives 1 if the total is above zero and 0 otherwise.
decision boundary
The line that separates the inputs a model puts in one category from those it puts in another.
training set
The examples a model is allowed to learn from.
test set
Examples kept hidden during training and used afterwards to check the model honestly.
noise
Random variation in data that carries no real pattern, such as measurement mistakes.
overfitting
When a model learns the training examples too closely, noise included, and so does badly on new examples.
underfitting
When a model is too simple to capture the real pattern, so it does badly even on its training examples.
generalisation
How well a model performs on examples it has never seen. This is the real goal of training.