Classification
Sort things into categories: spam or not, disease or healthy, cat, dog or bird. Turn scores into probabilities, write a nearest-neighbour classifier yourself, and see how decision trees ask a chain of yes-or-no questions.
Start this course →Lessons
- 1Predicting categoriesTell binary from multiclass problems, and see a classifier as a map.6 min
- 2From scores to probabilitiesTurn a weighted sum into a probability and a decision.7 min
- 3Nearest neighbours, in codeWrite a classifier from scratch and see why k matters.8 min
- 4Decision trees and forestsRead a decision tree and know why forests of trees work better.7 min
What you will learn
- A classifier picks one class from a fixed set.
- Binary classification has two classes; multiclass classification has more.
- A classifier divides the feature space into regions, separated by decision boundaries.
- Logistic regression = weighted sum, then the sigmoid function.
- The output is a probability between 0 and 1.
- A threshold, often 0.5, turns the probability into a class.
- Euclidean distance measures how far apart two examples are.
- k-nearest neighbours lets the k closest examples vote.
- Small k follows noise; large k smooths too much. k is a choice you tune.
- A decision tree asks yes-or-no questions until it reaches a leaf with a prediction.
- Deep trees overfit; limiting depth helps.
- A random forest averages many different trees and is usually more accurate.
Terms explained in this course
- class
- One of the categories a classifier chooses between, such as "spam" or "not spam".
- binary classification
- Classification with exactly two classes, such as yes or no.
- multiclass classification
- Classification with three or more classes, such as recognising which of ten digits is in an image.
- probability
- A number from 0 to 1 saying how likely something is. 0 means impossible, 1 means certain, 0.5 means even odds.
- logistic regression
- A classification method that computes a weighted sum, like linear regression, then squeezes it with the sigmoid function into a probability.
- distance
- How far apart two examples are, worked out from their features.
- euclidean distance
- Straight-line distance: the square root of the sum of squared differences between the features.
- k-nearest neighbours
- A classifier that looks at the k most similar training examples and takes a vote among their labels. Often shortened to k-NN.
- decision tree
- A model that makes a prediction by asking a chain of yes-or-no questions about the features, like a flowchart.
- leaf
- An end point of a decision tree, where a prediction is given.
- random forest
- Many decision trees, each trained on a slightly different sample of the data, voting together. Usually more accurate than one tree.