Experimentation in psychology, linguistics, and AI

Course goals

At the end of this course, you can

• design experiments on the basis of a given model and research question in the realm of linguistics and/or psychology with relevance to artificial intelligence
• implement experiments using various computational techniques
• extract crucial data from experimental responses
• interpret experimental results and report on these in an appropriate way

Students participating in this course will learn

1. how to design experiments given an existing model
2. how to implement experiments using various tools and, finally
3. how to extract data from the recorded responses for analysis purposes.

Assessment
The learning goals will be examined in three ways, and all three will be graded: 

1. students will work in groups and individually on lab assignments in which they read and critically reflect on selected articles from the experimental literature, and develop data analysis for experimental data and build statistical models. 
2. students will give a presentation on a selected paper. The presentation should include a summary, a critical reflection and plans for own experimental work.
3. students will design and implement an experiment on a topic of their own choice and write a report on the experiment. he report will be graded.

To qualify for a repair of the final result the mark needs to be at least a 4, or “AANV”.

Content

Both science and industry are interested in creating precise formal models of human behavior and cognition. To help build, test and optimize such models, one needs to create and run experiments.

Most theoretical claims in linguistics and psychology are made by positing a formal model. The aim of such models is to make precise predictions.
Moreover, the predictions of a model need to tested with formal experiments.
The results of the experiment may or may not lead to changes in the model and thus lead to a new set of testable predictions.  
Essential in the modelling-experimenting cycle is careful experimental design.
The course covers the practical and theoretical considerations for experimental research, from posing the research question to interpreting and reporting experimental results.
 
In industry, experiments are also used frequently. For example, to assess how people use interfaces (e.g., where do they look or click, or how particular text influences their subsequent choices?), to test what the best design of a product is, or to test the appropriateness of a user model (e.g., do people learn what the model predicts them to learn, do they have a more immersive experience when a model guides adaptation of the software?).
 
In this course you will get an overview of various experimenting techniques that are used world-wide and also by researchers in Utrecht (esp. in the department of psychology and the department of linguistics).
You will learn how to use such techniques for testing specific models, as well as where the limits of these technique lie.
In the tutorials you will also gain hands-on experience with the implementation, data manipulation and data analysis steps of experimenting.

Course form
Lectures, tutorials.