Statistical learning and stochastic processes

Course goals

After completing this course, the student will be able to:

• specify a stochastic process appropriate to model a given dynamic system
• conduct analytical derivations on the behavior of the stochastic process
• conduct simulations of the stochastic process
• estimate the parameters of a stochastic process from data 
• perform hypothesis tests to test a given stochastic model, or to compare different models
• select an appropriate model from a number of candidates using scoring functions (AIC/BIC)

Assessment
The final grade consists of:

• written exam (60% of the final mark)
• 2 practical assignments (20% each)

The minimum grade to pass the exam is 5.5.
For each of the assignments, the minimum grade to pass is 6.
If an assignment is failed, then it can be redone. The maximum grade for a retake is 7. 

To qualify for a repair of the final result the mark needs to be at least a 4, or “AANV”.

Content

This course introduces stochastic processes to model complex, dynamic systems. 
First we discuss a number of important stochastic processes such as Markov chains and Poisson processes. We study their properties, and discuss how to determine the probabilities of different events for a given stochastic process
both analytically and through simulation.
Then we turn to the problem of estimating the parameters of stochastic processes, e.g. the transition probabilities of a Markov chain, from data. Here we enter the area of statistical inference.

As a possible area of application, think of an airport where data is available on the number of passengers present, the duration of their stay at the airport premises, the facilities used, etc. All these data can be used to specify a stochastic process, 
which supports the analysis of the system (e.g., how busy is the airport expected to be tomorrow) as well as decision-making (e.g., the increase/decrease of the number of check-in desks). 

The course provides students with the skills to use data for model specification (stochastic processes), model analysis and decision-making.

List of topics:
1. Monte Carlo simulation
2. Markov Chains, Poisson processes
3. Gaussian processes
4. Markov decision processes
5. statistical inference for Markov chains and Poisson processes
6. linear regression, autoregressive models

Course form
2 lectures every week, and 1 tutorial session every week. The lectures and the tutorials are on campus.