How Machines Learn
Data, models, error and the downhill walk that trains almost every AI system. You will fit a line, train by gradient descent, tune a neuron and catch a model that memorised its homework.
Start this course →Lessons
- 1Data: the raw materialRead a dataset the way a learning program does.5 min
- 2A model is a machine with knobsFit a line to data by hand and measure how wrong it is.7 min
- 3Gradient descent: walking downhillWatch a computer train itself, and learn why step size matters.7 min
- 4One artificial neuronBuild the unit that every neural network is made of.7 min
- 5Overfitting: memorising is not learningSpot a model that looks perfect in training and fails in real use.7 min
What you will learn
- A dataset is a table: each row is an example, feature columns describe it, and the label is the answer.
- Learning from labelled examples is supervised learning.
- Predicting a number is regression. Predicting a category is classification.
- A model contains adjustable numbers called parameters.
- The loss is one number that measures how wrong the model is.
- Training means searching for the parameter values that make the loss small.
- The gradient is the slope of the loss. Stepping against it lowers the loss.
- Gradient descent repeats that step again and again.
- The learning rate sets the step size: too small is slow, too large overshoots.
- A neuron multiplies inputs by weights, adds a bias term, and applies an activation function.
- A single neuron draws one straight decision boundary.
- Weights rotate the boundary and the bias term slides it.
- Judge a model on a test set it never saw during training.
- Underfitting: too simple, wrong everywhere. Overfitting: too flexible, it memorises noise.
- The goal is generalisation: doing well on new examples.
Terms explained in this course
- dataset
- A collection of examples, usually arranged as a table with one row per example.
- supervised learning
- Machine learning from examples that come with the correct answer (a label) attached.
- regression
- Predicting a number, such as a price or a temperature.
- classification
- Predicting a category, such as "spam" or "not spam".
- parameter
- An adjustable number inside a model. Training means finding good values for the parameters.
- linear model
- A model that predicts with a straight line: prediction = slope × input + intercept.
- slope
- How much the prediction rises when the input goes up by one. It sets how steep the line is.
- intercept
- The prediction when the input is zero. It sets where the line starts.
- error
- The gap between a model's prediction and the correct answer.
- loss
- A single number that says how wrong a model is across all its examples. Lower is better.
- mean squared error
- A common loss. Square each error, then take the average. Often shortened to MSE.
- gradient
- The slope of the loss at the current parameter values. It points in the direction that makes the loss grow fastest.
- gradient descent
- A training method: repeatedly nudge each parameter a little in the direction that lowers the loss.
- learning rate
- A number that sets how big each gradient descent step is.
- iteration
- One repeat of a process. In training, one update of the parameters.
- convergence
- The point where more training steps no longer change the loss in any meaningful way.
- neuron
- The basic unit of a neural network. It multiplies each input by a weight, adds the results and a bias term, then applies an activation function.
- weight
- A parameter that sets how strongly one input influences a neuron.
- bias term
- A parameter added to a neuron's total. It shifts the point at which the neuron switches on. It has nothing to do with unfairness, which is a different meaning of the word bias.
- activation function
- The final step in a neuron, which turns its total into an output. The simplest one gives 1 if the total is above zero and 0 otherwise.
- decision boundary
- The line that separates the inputs a model puts in one category from those it puts in another.
- training set
- The examples a model is allowed to learn from.
- test set
- Examples kept hidden during training and used afterwards to check the model honestly.
- noise
- Random variation in data that carries no real pattern, such as measurement mistakes.
- overfitting
- When a model learns the training examples too closely, noise included, and so does badly on new examples.
- underfitting
- When a model is too simple to capture the real pattern, so it does badly even on its training examples.
- generalisation
- How well a model performs on examples it has never seen. This is the real goal of training.