Introduction to Machine Learning

Course goals

Zie onder vakinhoud.

Content

The goal of this course is to give an introduction to machine learning. We will introduce several successful methods for regression, classification, clustering, and data dimension reduction and develop the mathematical theory behind these methods, with a focus on continuous optimization theory. In the exercise sessions, the students will work on proof-style exercises, as well as implementation exercises in Python.

The course builds on knowledge acquired in the first year, in particular Calculus and Linear Algebra 1 and 2, Analysis, Probability (or its predecessor Introduction to Probability and Statistics) and Computer Programming for Mathematics.

This course is one of the modelling courses in the bachelor programme with major 'mathematics' or 'mathematics and applications'. Please find more information on the students website.

In the course we will treat the following topics:

• Basic machine learning terminology: different types of tasks, loss functions, training-validation-test split, cross-validation, generalization error, under- and overfitting, interpolation
• Linear regression and regularization (ridge regression and the Lasso)
• Logistic regression
• Support vector machines
• Kernel methods
• Decision trees and random forests
• Introduction to neural networks
• Data dimension reduction methods (e.g., principal component analysis and autoencoders)
• Clustering methods (e.g., k-means and spectral clustering)
• Gradient descent and its variants, e.g., stochastic gradient descent and accelerated gradient descent
• Convergence analysis of descent methods
• Theory for constrained convex optimization problems, in particular the Karush-Kuhn-Tucker theorem and Lagrangian duality. 

Course format
Lecture (2 times per week), exercise class (2 times per week)

Examination
There will be 4 homework assignments and a final exam. Let A₁, A₂, A₃, A₄ be your grades for the four homework assignments and let A=(A₁+A₂+A₃+A₄)/4 be the average. Let E be your grade for the final exam. Your final grade is max{0,2 * A + 0,8 * E, E}.

The final exam will be a written exam. The final exam will be partially open book exams: you are allowed to use the lecture notes of the course during the exam.

Retake and participation obligation
You are allowed to participate in the re-take exam if you handed in at least two assignments and participated in the final exam. In case of a re-take, your grade is determined as above, where E is your grade for the re-take exam.

The re-take exam will be either a written or an oral exam, depending on the number of participants.