Foundations of Machine Learning

• Tentative Syllabus: Syllabus_ML_Oxford_2027
• Introduction: The Means of Prediction - Introduction

Slides

• Review of decision theory Slides

Foundations of supervised learning

• Supervised learning using Python Jupyter Notebook, Html version
• Probably approximately correct learning theory Slides
• Shrinkage in the normal means model Slides

Supervised learning methods

• Ridge regression and Gaussian process priors Slides
• Deep neural nets Slides
• Conformal inference for prediction Slides

Online learning and reinforcement learning

• Overview of online learning and reinforcement learning Slides
• Online convex optimization Slides
• Bandit problems Slides
• Markov decision problems and reinforcement learning Slides

Language models

• Large language models: Training and architecture Slides

Ethics and machine learning

• Fairness Slides
• Differential privacy Slides

Bonus material

• Limiting experiments and the normal means model Slides
• Reproducing Kernel Hilbert Spaces and Splines Slides
• Applications of Gaussian process priors in economics Slides
• Regression trees and random forests Slides
• Double/debiased machine learning Slides
• Variational auto-encoders and diffusion models for image generation Slides
• Adversarial online learning Slides
• Worst-case sequence for Thompson sampling Slides
• Experiments for policy choice Slides

Homework problems

1. Approximation, estimation, and shrinkage: foundations_ml_ps1_2027.pdf
2. Conformal prediction: foundations_ml_ps2_2027.pdf
3. Multi-armed bandits: foundations_ml_ps3_2027.pdf

Class readers

• Foundations.zip
• Supervised.zip
• Active.zip
• Ethics.zip

Useful links

Machine learning books

• The Means of Prediction
• The Elements of Statistical Learning
• Understanding machine learning: From theory to algorithms
• Gaussian Processes for Machine Learning
• Introduction to Online Convex Optimization
• Deep Learning
• Reinforcement learning - An introduction
• Reinforcement Learning: Theory and Algorithms
• Speech and Language Processing
  • microgpt
• Reinforcement Learning from Human Feedback
• The Ethical Algorithm

Programming in Python

• A Whirlwind Tour of Python
• Python Data Science Handbook
• scikit-learn - Machine Learning in Python
• Vega-Altair: Declarative Visualization in Python

Programming in R

• An Introduction to R
• R for Data Science
• Advanced R
• Hands-On Machine Learning with R

Data visualization

• Data Visualization - A Practical Introduction
• An Economist’s Guide to Visualizing Data
• ggplot2 - Elegant Graphics for Data Analysis