Working with Data
Every AI system is built on data. Learn to read a table, summarise it honestly, clean up the mess real data always contains, and spot a misleading chart. You will write Python for each step.
Start this course →Lessons
- 1Data is a tableRead a dataset as rows, columns and kinds of values.6 min
- 2Summing up: mean, median and modeChoose the summary that tells the truth about your data.7 min
- 3Real data is messyFind and fix the problems that real data always has.7 min
- 4Charts that tell the truthPick the right chart, and spot one that misleads.7 min
What you will learn
- A dataset is usually a table: each row is an example and each column is a property.
- Tables are often stored as CSV text.
- Numerical data can be averaged; categorical data names groups.
- The mean adds and divides; the median is the middle value; the mode is the most common.
- One outlier can drag the mean far away, while the median barely moves.
- The range gives a quick sense of spread.
- Real data has missing values, duplicates, typing mistakes and outliers.
- Data cleaning fixes them, and each fix is a decision that should be written down.
- A model trained on dirty data learns the dirt.
- Bar charts compare categories, histograms show shape, scatter plots show relationships.
- A cut-off axis can make a small difference look huge.
- Correlation is not causation: two things moving together does not prove one causes the other.
Terms explained in this course
- row
- One horizontal line in a table. In a dataset, usually one example, such as one person or one sale.
- column
- One vertical line in a table. In a dataset, usually one property measured for every example, such as age or price.
- csv
- Comma-separated values: a plain text file where each line is a row and commas separate the columns. A common way to store tables.
- numerical data
- Data that is a number you can do arithmetic with, such as height or price.
- categorical data
- Data that names a group, such as a country or a colour. Averaging it makes no sense.
- mode
- The value that appears most often.
- range
- The largest value minus the smallest value. A quick measure of how spread out data is.
- outlier
- A value far away from the others. It may be a mistake or a real but unusual case.
- missing value
- An empty spot in a dataset where a value should be.
- duplicate
- The same record appearing more than once in a dataset.
- data cleaning
- Finding and fixing problems in data, such as missing values, duplicates and typing mistakes, before using it.
- bar chart
- A chart that compares amounts across categories using bars of different lengths.
- histogram
- A chart showing how often values fall into each range, which reveals the shape of the data.
- scatter plot
- A chart with one dot per example, placed by two numerical values, to show whether they are related.
- correlation
- A pattern in which two quantities tend to rise and fall together, or in opposite directions.
- causation
- When one thing actually makes another happen. Correlation alone does not prove causation.