Multi-agent learning

Course goals

Learning objective of the course: to acquire an adequate impression of state-of-the-art research in multi-agent learning, including topics such as learning and teaching, fictitious play, rational learning, no-regret learning, targeted learning, multi-agent reinforcement learning, evolutionary learning.

Assessment
Two exams.

To qualify for a repair of the final result the mark needs to be at least a 4, or “AANV”.

Content

This course discusses algorithms for machine learning that typically occur in multi-agent systems, such as: reinforcement learning, no-regret learning, fictitious play, satisficing play, Bayesian learning, learning and teaching, and evolutionary learning (replicator dynamic).

Multi-agent learning (MAL) studies agents that learn and adapt to the behavior of other agents that themselves learn and adapt.
The presence of other learning agents complicates learning, which makes the environment non-stationary (a situation of learning a moving target) and non-Markovian (a situation where not only experiences from the immediate past but also earlier experiences are relevant).
It becomes less beneficent to only adapt to the behavior of other agents, on the pain of being exploited by more steadfast agents that do not follow but instead impose their strategy on others.
Important topics of MAL include (evolutionary) game theory, fictitious play, gradient dynamics, no-regret learning, multi-agent reinforcement learning (MinMax-Q, Nash-Q), leader (teacher) vs. follower (learner) adaptation, and the emergence of social conventions.
Examples of domains that need robust MAL algorithms include manufacturing systems (managers of a factory coordinate to maximize their profit), distributed sensor networks (multiple sensors collaborate to perform a large-scale sensing task under strict power constraints), robo-soccer, disaster rescue (robots must safely find victims as fast as possible after an earthquake) and recreational games of imperfect information such as poker. Indeed, poker and simplified forms of poker are an important topics of research in multi-agent learning.

Course form
Lectures.

Literature
Available on the course site.