Video based motion tracking and gesture recognition

Within a broader framework of learning systems to behave in an “intelligent” manner, the recognition of specific patterns such as gestures, objects, activities etc. from sensors’ output is needed. In this proposal students are asked to develop a gesture recognition method capable to identify a set of predefined gestures from video signals. The correct recognition of gestures is needed so that the system could take the right action and to adapt in a dynamic environment.

One typical application is be the recognition of traffic policeman or airplane controller gestures from video images. The main tasks of this project include:

• object recognition and object tracking from video images
• extraction of motion information associated to each object
• recognition of gestures from motion tracking time series associated to objects of interest

Students are provided with 2D and 3D cameras in order to build a database of video images which serving as knowledge for extracting meaningful features characterizing each individual gesture. Within this interdisciplinary project the students have the opportunity to apply their knowledge in o real application. Gesture recognition is performed through time series segmentation. This implies the use of data mining methods for time series as well as feature extraction. Various methods for gesture recognitions will be investigated, particularly hidden markov models. The students are guided towards the most relevant publications in this area.

Basic image processing notions are required for the first task to be completed.

• Supervisor: Cosmin Lazar
• Promoter: Ann Nowé