Learning Control for Quadrotor Helicopters.

This thesis investigates the use of reinforcement learning for autonomous flight of 4-rotor helicopters (quadcopters). CoMo has recently acquired 2 Parrot AR Quadcopter Drones. These quadcopters can be controlled by users over WiFi, using iPhone or Android applications or custom developed software. Each helicopter has 2 camera’s in addition to a number of sensors (accelerometer, gyroscope, ultrasound) which can be read. During this thesis you will develop control software for the drones which uses reinforcement learning to learn how to perform maneuvers.

Reinforcement learning (RL) is one of the most promising AI paradigms for the future development of autonomous robots. RL allows a robot to learn from trial-and error interactions with its environment. By observing the results of its actions, a robot can determine the optimal sequence of actions to take in order to reach some goal. For most applications, however, starting the learning from scratch is not feasible, as the problem space is too large and learning without any prior information takes too long. Therefore, a number of techniques have been developed to provide the learner with initial information about the problem it has to solve. One of the possibilities is to give the learner demonstrations of possible strategies.

The idea of this thesis is to let the helicopters learn maneuvers from demonstrations given by a user. A maneuver is demonstrated by the user using remote control of the drone. These demonstrations can then be used to create initial policies from which the controller software can learn how to perform the maneuvers autonomously under different circumstances (e.g. varying heights, speeds, winds).

Students should have some knowledge of reinforcement learning (eg. from the courses Machine Learning or Multi-agent Learning) or be willing to acquire this knowledge during the thesis.

This thesis includes a research component, which investigates the use of RL for helicopter control. It should also lead to development of working control software for the AR drones.

• Stanford research on learning control for autonomous helicopter flight
• The Parrot AR drone Quadcopter
• Introduction to reinforcement learning
• Robot learning from demonstration
• A survey of robot learning by demonstration
• CoMo Robotics Wiki

• Peter Vrancx: pvrancx@vub.ac.be
• Ann Nowe (promotor): ann.nowe@vub.ac.be