Natural language generation

Course goals

Upon completion of this course, the student will have

• knowledge of the architecture of Natural Language Generation systems and the main tasks they perform
• knowledge of the linguistic and psycholinguistic underpinnings of different NLG tasks
• understanding of how neural network-based NLG models, including large language models, are designed and trained
• understanding of the decoding algorithms for generation with neural network-based NLG models
• knowledge of evaluation methods and metrics
• the ability to summarize and present a research paper on NLG

Assessment

• exam (70% of the final mark)
• individual coursework (20%)
• group presentation (10%).

To qualify for a repair of the final result the mark needs to be at least a 4, or “AANV”.

Content

This course studies the task of generating language automatically, from different types of input.
Examples include generating a summary of numerical data (for instance physiological parameters of a patient in a medical setting, or weather parameters for a weather report), or generating a description of an image.
Another example might be generating a response to a query, an increasingly common use case with large language models.

The generation of language involves many choices at different levels.
We therefore begin with an overview of “classical” NLG architectures and the various tasks they subsume, with a discussion of relevant linguistic and psycholinguistic insights.
We will then focus on more recent neural network architectures, from encoder-decoder to decoder-only models for text generation.
Here, we also pay attention to how such models are trained and finetuned, as well as the decoding algorithms for generating text stochastically.
Finally, we consider the challenges involved in evaluating NLG models using a variety of methods, from automatic metrics to task-oriented studies.

Course form
Lectures, presentations.

Literature
Most literature will be handed out during the course.