Multi-Agent Transfer Learning

Reinforcement learning is a mature technique for solving complex problems. Significant advances have been developed to speed up the learning process through the incorporation of domain knowledge, the decomposition of problems in subtasks, using a generalised state space representation or learning higher level actions, composed of multiple one step actions. The main drawback of all these techniques is however that each time the RL algorithm has to start the learning process from the beginning. Recently the idea of transfer learning has been applied to reinforcement learning tasks. Transfer learning leverages the experience an agent acquires in a source task in order to improve its performance in a related target task. In multi-agent environments, where learning is even slower due to the size of the state space, transfer learning is a interesting approach to explore. Only two attempts at using transferred experience in multi-agent RL are cited in [Taylor & Stone (2009)]. These approaches, [Kuhlmann & Stone (2007)] and [Banerjee & Stone (2007)] both deal with extensive form games. The only work reported on transfer learning in stochastic games (aka Markov games) is [De Hauwere et al. (2009)] and [Vrancx et al. (2011)].

The goal of this dissertation is to port the framework of transfer learning to a multi-agent setting. This can be done both in simulation as on actual robot hardware (quadcopters, wireless sensor nodes, khepera robots, mindstorms,…). Multiple thesis students can work on this project.

• Yann-Michael De Hauwere
• Peter Vrancx

• Reinforcement Learning: an Introduction
• [Banerjee & Stone (2007)] General game learning using knowledge transfer
• [Kuhlmann & Stone (2007)] Graph-based domain mapping for transfer learning in general games.
• [Taylor & Stone (2009)] Transfer learning for reinforcement learning domains: A survey.
• [De Hauwere et al. (2009)] Multi layer learning and knowledge transfer in MAS
• [Vrancx et al. (2011)] Transfer Learning for Multi-Agent Coordination