Regression
Predict numbers: prices, temperatures, exam scores. Fit lines by hand and in one line of NumPy, use many features at once, and measure honestly how far off your predictions are.
Start this course →Lessons
- 1Predicting numbers with linesRead a regression model and its residuals.7 min
- 2Least squares in one lineFit a regression line exactly with NumPy.7 min
- 3Many features at oncePredict from several features, and keep them on a fair scale.8 min
- 4How wrong is it?Measure a regression model with MAE, RMSE and R², and spot overfitting.8 min
What you will learn
- Linear regression predicts a number as weight × feature + intercept.
- A residual is the actual value minus the prediction.
- Training chooses the weights that make the residuals small.
- Least squares finds the line with the smallest sum of squared residuals.
- For a straight line there is an exact formula; np.polyfit applies it.
- Gradient descent reaches the same answer step by step, and also works for models with no exact formula.
- Multiple linear regression gives each feature its own coefficient.
- A prediction is a weighted sum: np.dot(weights, features) + intercept.
- Feature scaling stops large-numbered features from dominating.
- MAE is the average size of the errors; RMSE punishes big errors more.
- R² is the share of variation the model explains.
- Polynomial regression can bend, which also makes overfitting possible: always check the test set.
Terms explained in this course
- linear regression
- Predicting a number with a straight-line formula: each feature times a weight, added up, plus an intercept.
- residual
- For one example, the actual value minus the predicted value. The part of the answer the model missed.
- least squares
- Choosing the line that makes the sum of squared residuals as small as possible. It has an exact formula, so no training loop is needed.
- multiple linear regression
- Linear regression with more than one feature, such as predicting a house price from its size, rooms and age.
- coefficient
- The weight a regression model gives one feature. It says how much the prediction changes when that feature goes up by one.
- feature scaling
- Putting features on similar ranges, such as 0 to 1, so that no feature dominates just because its numbers are bigger.
- standardisation
- A common way to scale a feature: subtract its mean and divide by its standard deviation.
- standard deviation
- A measure of how spread out values are around their mean. Small means tightly clustered.
- mean absolute error
- The average size of the errors, ignoring their sign. Easy to explain: "on average we are off by 3 points". Often shortened to MAE.
- root mean squared error
- The square root of the mean squared error. It is in the same units as the target, and punishes big errors more than MAE does. Often shortened to RMSE.
- r-squared
- The share of the variation in the target that the model explains, usually between 0 and 1. Higher is better. Written R².
- polynomial regression
- Regression that allows curves, by adding powers of a feature such as x² and x³ as extra features.