Face and hands localization and tracking for sign language recognition

Abstract

We develop the face and hand detection and tracking for sign language recognition system. We first perform a preliminary evaluation on several color spaces to find the most suitable one using a nonparametric model approach. Then, we propose to use the elliptical model in the CbCr color space to lower the complexity of the detection algorithm and to better model the skin color. After the skin regions from the input video have been segmented, the interesting facial features and hands are detected using luminance differences and skeleton features respectively. In the tracking stage, each blob determines the search region and finds the MMSE (minimum mean square error) to match its own blob. The block matching method between previous and current frame is used. Experimental results show that our proposed system is able to detect and track the face and hands in sign language video sequences.