Foundations of Machine Learning

• Tentative Syllabus: Syllabus_ML_Oxford_2026
• Introduction: The Means of Prediction - Introduction

Slides

• Review of decision theory Slides

Foundations of supervised learning

• Supervised learning using Python Jupyter Notebook, Html version
• Probably approximately correct learning theory Slides
• Shrinkage in the normal means model Slides

Supervised learning methods

• Ridge regression and Gaussian process priors Slides
• Regression trees and random forests Slides
• Deep neural nets Slides
• Conformal inference for prediction Slides

Online learning and active learning

• Overview of online learning and active learning Slides
• Online learning Slides
• Online convex optimization Slides
• Bandit problems Slides
• Reinforcement learning Slides

Ethics and machine learning

• Fairness Slides
• Differential privacy Slides
• Explainability Slides

Bonus material

• Limiting experiments and the normal means model Slides
• Reproducing Kernel Hilbert Spaces and Splines Slides
• Applications of Gaussian process priors in economics Slides
• Transformer models for natural language processing Slides
• Double/debiased machine learning Slides
• Variational auto-encoders and diffusion models for image generation Slides
• Worst-case sequence for Thompson sampling Slides
• Experiments for policy choice Slides

Homework problems

1. Supervised learning: foundations_ml_ps1.pdf
2. Shrinkage estimation: foundations_ml_ps2.pdf
3. Conformal inference: foundations_ml_ps3.pdf
4. Multiarmed bandits: foundations_ml_ps4.pdf

Class readers

• Foundations.zip
• Supervised.zip
• Active.zip
• Ethics.zip

Useful links

Machine learning books

• The Means of Prediction
• The Elements of Statistical Learning
• Understanding machine learning: From theory to algorithms
• Gaussian Processes for Machine Learning
• Introduction to Online Convex Optimization
• Deep Learning
• Speech and Language Processing
• Reinforcement learning - An introduction
• The Ethical Algorithm

Programming in Python

• A Whirlwind Tour of Python
• Python Data Science Handbook
• scikit-learn - Machine Learning in Python
• Vega-Altair: Declarative Visualization in Python

Programming in R

• An Introduction to R
• R for Data Science
• Advanced R
• Hands-On Machine Learning with R

Data visualization

• Data Visualization - A Practical Introduction
• An Economist’s Guide to Visualizing Data
• ggplot2 - Elegant Graphics for Data Analysis