Foundations of Machine Learning

Tentative Syllabus: Syllabus_ML_Oxford_2024.pdf

Foundations

• Review of decision theory Slides
• Probably approximately correct learning theory Slides
• Shrinkage in the normal means model Slides

Supervised learning

• Gaussian process priors and reproducing kernel Hilbert spaces Slides
• Regression trees and random forests Slides
• Deep neural nets Slides
• Transformer models for natural language processing Slides
• Variational auto-encoders and diffusion models for image generation Slides
• Double/debiased machine learning Slides

Online learning and active learning

• Overview of online learning and active learning Slides
• Online learning Slides
• Online convex optimization Slides
• Bandit problems Slides
• Reinforcement learning Slides

Ethics and machine learning

• Fairness Slides
• Differential privacy Slides

Homework problems

1. foundations_ml_ps1.pdf
2. foundations_ml_ps2.pdf
3. foundations_ml_ps3.pdf

Class readers

• Foundations.zip
• Supervised.zip
• Active.zip
• Ethics.zip

Useful links

Machine learning books

• The Elements of Statistical Learning
• Understanding machine learning: From theory to algorithms
• Gaussian Processes for Machine Learning
• Deep Learning
• Speech and Language Processing
• Reinforcement learning - An introduction
• The Ethical Algorithm

Programming in Python

• A Whirlwind Tour of Python
• Python Data Science Handbook

Programming in R

• An Introduction to R
• R for Data Science
• Advanced R
• Hands-On Machine Learning with R

Data visualization

• Data Visualization - A Practical Introduction
• An Economist’s Guide to Visualizing Data
• ggplot2 - Elegant Graphics for Data Analysis