NumPy and Pandas
The two Python libraries behind almost all data science and machine learning. Do maths on whole tables at once, pick out exactly the rows you need, and summarise a dataset in one line.
Start this course →Lessons
- 1Arrays: maths on everything at onceDo arithmetic on a whole collection of numbers without a loop.7 min
- 2Picking out what you needSelect items by position and by condition.7 min
- 3Tables with pandasLoad a table, look at it, and summarise its columns.8 min
- 4Filter, group and cleanAnswer real questions about a table in one or two lines.8 min
What you will learn
- NumPy stores numbers in arrays and does maths on all of them at once.
- An element-wise operation like prices * 0.9 changes every item.
- Vectorisation replaces slow Python loops with one fast operation.
- An index picks one item; slicing picks a range, such as a[1:4].
- A boolean mask like a > 30 selects every item that meets a condition.
- Masks replace a loop with an if statement inside it.
- Pandas gives you the DataFrame: a table with named columns.
- Each column is a Series; df["age"].mean() summarises it.
- df.describe() summarises every numerical column at once.
- Filter rows with a boolean mask: df[df["age"] > 20].
- groupby splits a table into groups and summarises each one.
- dropna() removes rows with missing values; fillna() fills them in.
Terms explained in this course
- library
- A collection of ready-made code that you load into your program with import, so you do not have to write everything yourself.
- numpy
- The standard Python library for fast maths on large collections of numbers. Short for Numerical Python.
- array
- NumPy's main object: a grid of numbers of the same type, such as a row of prices or a table of pixels.
- element-wise operation
- An operation applied to every item of an array at once, such as adding 1 to every number.
- vectorisation
- Writing a calculation as one operation on a whole array instead of a Python loop over its items. It is much faster.
- shape
- The size of an array in each direction, such as (3, 4) for 3 rows and 4 columns.
- index
- The position of an item in a list or array. Python starts counting at 0.
- slicing
- Taking a section of a list or array by giving a start and an end position, such as a[2:5].
- boolean mask
- An array of True and False values used to pick out only the items where the value is True.
- pandas
- The standard Python library for working with tables of data, with named columns and mixed types.
- dataframe
- A pandas table: rows of examples and named columns of values, like a spreadsheet inside Python.
- series
- A single column of a pandas DataFrame.
- groupby
- Splitting a table into groups that share a value, such as a city, then summarising each group.