Sampling, inference and estimation

Course goals

The objective of this course is to introduce basic and more advanced methods for analysing survey data. At the end of course, the students will be able to:

• Describe and explain the principles of modern survey statistics, inference and multi-source data (Knowledge and Understanding)
• Critically evaluate case studies in terms of design and quality, and propose appropriate methodological solutions (Judgement - Learning skills)
• Assess the limitations of different forms of data and apply design- or model-based inference to obtain valid population or analytical estimates (Judgment - Learning skills)
• Analyse surveys and multi-source data using statistical software (Applying)
• Interpret and communicate results effectively, both in writing and through oral presentations (Communication skills)
• Demonstrate professional and ethical responsibility in research, ensuring transparency, data integrity, and adherence to research ethics (Learning skills)

Students are tested with an exam and an assignment. 

Content

Population inference is a key goal of almost any scientific project: How can we use data to produce knowledge about the world around us? This course guides students through the complete statistical process, from sampling to modern estimation methods for drawing inferences from complex surveys and multi-source data.

In today’s digital world, data come from many different sources: traditional face-to-face surveys, online surveys, and digital traces (e.g. social media posts, sensor data). Each of these sources raises specific challenges for how we can generalize from observed data to the population of interest. The main focus of the course is on surveys and throughout the weeks we will discuss different data sources that can be combined to produce multi-source statistics.

In the first part of the course (Weeks 1-5), we focus on sampling and introduce the distinction between probability and nonprobability sampling techniques. You will learn how to draw different types of probability samples (simple random sampling, stratified, cluster, and multistage sampling) and the statistical theory of design-based inference. We will study complex sampling designs and how to use sampling weights to produce unbiased population estimates, as well as how to conduct analytic inference (for example, regression analysis under complex survey designs) using real data such as the European Social Survey.
A challenge of drawing a sample is to obtain a good sampling frame (list of units in the population). If the sampling frame does not fully cover the target population, coverage error arises. We will discuss how to address coverage problems by applying statistical adjustments, such as weighting and calibration.
In practice, almost every study also faces missing data. Respondents may not participate at all (unit nonresponse) or may answer only some questions (item nonresponse). Both processes, if non-random, introduce bias. We will examine in detail two major approaches to deal with missing data: weighting adjustments that re-balance the sample to account for nonresponse, and imputation methods that fill in missing values under explicit statistical models.
Failure to account for sampling design characteristics, coverage and nonresponse problems, has major consequences for estimation and inference across different analyses, from descriptive statistics to hypothesis testing and estimation of parameters in multivariate models. Throughout the first part of the module, we will develop both the theoretical foundations and applied skills to address these challenges.

In the second part of the course (Weeks 6-10), building on the foundations of part 1, we turn to advanced topics: inference with nonprobability samples, data integration, and small area estimation.
Many studies rely on convenience, quota, opt-in panel samples, where individuals voluntarily take part in the study. In such cases, design-based inference is not applicable because the sample is not selected randomly and is subject to selection bias. We will study how to address selection bias using weighting, quasi-randomization, mass imputation, and doubly robust estimation.  These methods often rely on auxiliary information from high-quality probability samples or administrative registers.
Given the increasing availability of multiple data sources, data integration is another important area of research. It not only helps to improve inference with nonprobability samples but also enables richer analyses by combining sources that measure different aspects of the same phenomenon. We will explore record linkage, and matching techniques.
Finally, in many research projects, the objective is to obtain reliable estimates for small domains (for example, regions, municipalities, or specific subpopulations defined by cross-tabulating age, education, or income). Traditional probability samples provide unbiased estimates for small groups, but often with very high variance due to limited sample sizes. As a solution, you will learn how to apply small area estimation methods that borrow strength by auxiliary data, such as other surveys, administrative, and digital trace data.

By the end of the course, students will be able to design, evaluate, and analyse surveys and multi-source data with an understanding of the key methodological challenges in modern data analysis.

Throughout the course, practical exercises are conducted using the software package R. This course considers the nature of various general methods, the supporting statistical theory, but also practical applications. The R-packages survey, sampling and mice will be used for statistical computations and are part of the course material.