Machine Vision - site visit 2

Machine Vision for Sensing and Understanding Human Actions, a project of the Machine Vision Group at the University of Oulu

Site visit in Oulu on 17 May 2004

The project studies how machine vision can be used in proactive computing. The capability to understand human actions in proactive applications is mandatory. Actions such as sitting, standing, walking, using one's hands, talking or facing in a certain direction will most likely matter in a proactive solution. Based on this knowledge a proactive application may then adapt to the activities and respond in a desired way.

The group has an extensive research record and has developed methods for skin detection, detection and recognition of faces, and tracking humans. For example, a face recognition system preprocesses the image, detects the face, extracts features and then tries to classify the image (i.e. recognise the person). State-of-the-art systems detect faces well when the test person looks straight into the camera, and are quite good at recognising faces under laboratory conditions. Challenges are posed by varying illuminating conditions, different facing directions, aging, eye glasses and beards, as well as by working "in the field". In addition, existing detection and recogntion algorithms tend to be slow.

A technique that has proved simple and efficient in recognising faces is Local Binary Patterns (LBPs). The image of a face is divided into small regions that are encoded by local LBP histograms. These in turn are concatenated into a single feature histogram that represents the whole face. The original LBP operator labels pixels by considering a 3*3 pixel area and taking the centre pixel value as a threshold. Each of the eight surrounding pixels is denoted with either a one or a zero depending on if their value exceeds or falls below the centre value. The result (label of the centre pixel) is considered as a binary number and the histogram of the labels can be used as a texture descriptor. In more recent studies, neighbourhoods of different sizes have been used. Important regions, such as the eye area, may also be weighted to achieve better results. Recognition is performed by using a nearest-neighbour classifier with Chi square as a dissimilarity measure. Experimental results show that the LBP method clearly outperforms other methods such as PCA, Bayesian Intra/extrapersonal Classifier and Elastic Bunch Graph Matching.

Skin detection is challenged by the fact that skin colour might change when lightning changes. Therefore a skin detection model must be able to cope with varying illumination conditions. The group has tested several methods for detecting skin, which as long as they are used for images of good calibration, work quite well. By taking into account the range of possible skin colours, or skin locus, the group has been able to build a model that increases the detection capability when the illumination conditions vary. This method, on the other hand, may discriminate poorly and return false positives, i.e., classify non-skin as skin. But the results can be improved, e.g,  by using additional information such as consecutive frames (of video images).

Some further research of the group include a new stabilization method for improving video quality when using a handheld camera and a real-time head tracker that estimates the pose of the head of a human person.  The group has also developed methods for person identification based on colour histograms of clothes and started research on subtracting the background of moving objects.

Some publications related to the project are listed below. More publications can be found on the web page of the research group http://www.ee.oulu.fi/mvg/ . 

• J. Heikkilä. A statistical method for object alignment under affine transformation. Proc. 12th International Conference on Image Analysis and Processing (ICIAP 2003), September 17-19, Mantova, Italy.
• J. Heikkilä, O. Silvén. A real-time system for monitoring of cyclists and pedestrians. Image and Vision Computing, in press.
• A. Hadid. M. Pietikäinen. Efficient locally linear embeddings of imperfect manifolds. Prod. 3rd International Conference on Machine Learning and Data Mining (MLMD 2003), July 5-7, Leipzig, Germany.
• B. Martinkauppi, M. Soriano, M. Pietikäinen. Detection of skin color under changing illumination: a comparative study. Proc. 12th International Conference on Image Analysis and Processing (ICIAP 2003), September 17-19, Mantova, Italy.
• M. Soriano, B. Martinkauppi, S. Huovinen, M. Laaksonen. Adaptive skin color modeling using the skin locus for selecting training pixels. Pattern Recognition 36(3):681-690.
• T. Ahonen, A. Hadid, M. Pietikäinen. Face recognition with local binary patterns. Computer Vision, ECCV 2004 Proceedings, Lecture Notes in Computer Science 3021, Springer, 469-481.

Some recent theses in the project are

• T. Ahonen. Face recognition with local binary patterns. M.Sc. thesis, Department of Electrical and Information Engineering, University of Oulu, Finland, 2004, 55 p.
• J. Hannuksela. Facial feature based head tracking and pose estimation. M.Sc. thesis, Department of Electrical and Information Engineering, University of Oulu, Finland, 2003, 60 p + App.
• M. Heikkilä. Liikkuvien objektien ilmaiseminen tekstuuriin perustuvalla menetelmällä (Moving object detection with a texture-based method). M.Sc. thesis, Department of Electrical and Information Engineering, University of Oulu, Finland, 2004, 55 p + App.
• R. Niemelä. Henkilön tunnistaminen värihistogrammien avulla (Person identification using color histograms). M.Sc. thesis, Department of Electrical and Information Engineering, University of Oulu, Finland, 2003, 61 p + App.
• P. Saarinen. Proaktiivisen konenäköjärjestelmän arkkitehtuuri (Architecture of a proactive machine vision system). M.Sc. thesis, Department of Electrical and Information Engineering, University of Oulu, Finland, 2003, 58 p + App.

More information

For more information, please contact Professors Pietikäinen and Silvén at the University of Oulu or see the home page of the group at http://www.ee.oulu.fi/mvg/mvg.php.

Research Group

University of Oulu, Department of Electrical and Information Engineering, Information Processing Laboratory, Machine Vision Group:

• Professors Matti Pietikäinen, Olli Silvén, and Janne Heikkilä, Dr. Birgitta Martinkauppi
• Researchers Abdenour Hadid, Jari Hannuksela, Marko Heikkilä, Sami Huttunen, Vili Kellokumpu, Pekka Sangi