Analysis on skin colour model using adaptive threshold values for hand segmentation

Hand gesture segmentation is the hottest topic in manmachine interaction and computer vision. Segmentation is the process of separating the background and foreground. The objective of hand segmentation is to find out the hand region from the images region or no-hand region. There are many hands segmentation techniques such as threshold-based, edge-based, region-based, pixel-based, model-based, ANN-based, watershed-based, and clustering-based and many other segmentation methods. In all of these techniques, the skin colour threshold based method is simple and also common used method in state-of-art research. But, skin colour thresholds based method remains challenging when background contains the other objects like the human skin colour. Firstly, in this paper,

YCbCr colour threshol ds based approach is implemented. YCbCr colour space has good characteristics such as colour independent each colour components, good colour clustering and simple to calculate. The main advantages of skin colour based segmentation are (i) it is easy and fast to implement in training and testing. (ii) It is not dependent on the distribution of neighbour colour pixels. The strength of hand segmentation for hand gesture recognition can be reducing the amount of data or the dimension of data values. In this paper, the authors are experimented with various threshold values on YCbCr colour space to detect the human skin.

The main challenges that need to solve as research issue include:

• •    Illumination condition is the most sensitivity for hand gesture recognition of vision based system.

• •    Complex backgrounds, dynamic backgrounds are also main difficult for hand gesture recognition.

• •    The different size of the user’s hand is another issue.

• •    The multiple gestures in the same background and different viewpoints.

• •    The several persons contains in the sense other than the real subjects.

• •    The occlusion problem can also occur for the hand gesture recognition problem.

• •    Devices are also important to capture the images because different cameras have the different output for the same images.

• •    In dynamic gestures, the movement of hand can degrade the quality of images due to the blurring conditions.

The motivation of our research is to find out the best threshold values of YCbCr colour space for human skin detection. YCbCr color space is the best colour model than HSV and RGB in colour clustering range and colour independent for skin segmentation [9,17,18,21]. According to the previous literature, YCbCr colour space is the best suitable colour space for human skin detection. But, the colour spaces and threshold values are not fixing because different taking image devices, skin tone, illumination changing, blurring condition due to the movement object, shadow and many other conditions.

The general steps of hand gesture recognition system describe the following Fig 1. In Pre-Processing step performs image resizing, image debluring, image enhancement and so on. In hand region detection, extract the hand from the background region by using human skin colour, background subtraction method, foreground extraction and many other methods. In feature extraction step, extract the hand gesture features by using various deep learning pre-train models; supervise learning and unsupervised learning methods. The most common features are colour features, edge features, shape features, orientation features, location features and many other specific features.

The rest of the paper is described as following: Section II describes the detail explanation of the literature survey and Section III presents the ways of data collection for hand gestures. Section IV describes about the datasets. Section V explains the YCbCr colour model. The experimental results are described in section VI. The last section describes the conclusion and future work of the research.

H and Region

Detection

Feature Extraction

Hand Gesture Recognition

Class of Hand

Gesture

Fig.1. General Hand Gesture Recognition Steps

• II.    Related Wroks

Nguyen et al (2018) introduce the new hand segmentation by combining the Mask R-CNN and the tracking. The original Mask R-CNN can’t detect the hand when the blur condition due to the motion. The author’s proposed method produced a better result than the original Mask R-CNN [3].

Zhang et al (2018) implemented novel hand segmentation by combining saliency detection and skin colour detection. The implemented method is more robust than state-of-art research in complex background [4]. Rahmat and his partners (2016) proposed skin colour segmentation by combining three different colour spaces such as Normalized RGB, HSV and YCbCr colour space. The skin colour detection accuracy is more robust than the single colour spaces. The recognition accuracy was obtained 91.05% on two different datasets (ECU and HGR) [5].

Thakur and his friends (2011) study that novel skin colour detection model by combining RGB, HS and CbCr and named as RGB-HS-CbCr to detect the human face. The proposed model can successfully detect the human skin regions and human non-skin regions [6]. Yingxin et al (2016) presents a robust hand gesture recognition method via convolutional neural network. In this paper, the authors firstly pre-process on the hand gesture input by using canny edge detection and then extracted features by using CNN. The accuracy of the experimental results on original the input image is lower than the accuracy of the edge extracted images. Canny edge detection can be used to eliminate the illumination variation on the original input image [15].

Phaung et al (2003) implemented skin segmentation algorithm by combining colour and edge information. In this paper, the authors tested five different colour spaces such as YCbCr, HSV, RGB, CIE XYZ and CIE Lab [8]. Phaung et al (2005) also have written the next paper for human skin colour segmentation on four different colour spaces such as CIE-Lab, YCbCr, HSV and RGB colour spaces. [7].

Rungruangbaiyok et al (2015) proposed skin colour threshold-based segmentation methods with the optimized threshold values on various colour models. In this paper, the ensemble Threshold Segmentation method (ETS) with the HSV colour space model is the most accurate method than other colour space threshold based methods. The threshold values of all colour spaces are between 0 and 1. The accuracy was obtained 98.51 % on NUS Hand Pose Dataset I. [14].

Wang et al (2012) implemented hand segmentation methods by combining skin colour and background information. The performance of the proposed system is better than the single skin colour model [12]. Xu et al (2017) proposed the hand region identification algorithm by using YCgCr colour space model. YCgCr colour space is based on YCbCr colour space and also similar [11]. Patidar et al [2013] proposed that hand segmentation and tracking by using mix colour model such as (YCbCr+HSV) and (YCbCr+RGB) [13].

Tabassum et al (2010) implemented statistical skin detection algorithms for sub-Continental Human Images. In this paper, the authors implemented with three different colour models (HSV, YUV, YUV and YIQ). According to the experimental result, the HSV colour space model is the best performance models than other two colour space models to Indian sub-continental skin colour recognition [16]. Deng et al (2011) proposed real time hand gesture recognition by using cost matrix and context based matching approach. In this paper, the authors used YCbCr skin colour model to detecting the human hand. In the experimental results, the precision rates are between 70% and 90 % [17].

Ahlvers et al (2005) implemented face detection and head tracking with morphological hole mapping without using model. In this paper, the authors used three different colour modes (RGB skin cluster, YCbCr skin cluster and HSV skin cluster model) [18]. Lee et al. (2018) implemented static hand gesture recognition using wristband-based contour features. Wristband detection and skin color detection used to detect the hand and watershed segmentation and also used region merging techniques to remove the overlap condition. Finally, Minimum cosine distance used to recognize the static hand gestures. In this paper, the classification accuracy is up to 99.31 % for 29 gestures. The general issue of this paper is difficult to detect the wristband when the background contains dark color and dark shadows within the gesturing hand.[19]

Sahoo et al. (2018) used grey world algorithm to remove illumination variation from the input hand gesture images and discrete wavelet transform and fisher ration (F-ration) used to extract features. Finally, the hand gestures recognized based on Support Vector Machine Algorithm. This paper is experimented on three different datasets such as MUDB dataset, Jochen-Triesch dataset and Self-Constructed dataset. The mean accuracy was obtained 98.64 %, 95.42 % and 99.08 % respectively. The issues of this paper can be misclassifying gestures vocabulary when rotation noise above 15 degree. [20].

• III.    Data Collation For Hand Gestures

There are basically three ways to collect the raw data for input of hand gesture recognition system.

• •    The first one is applied colour glove or data glove to collect the raw hand data called glove based approach. In data glove approach, the main drawbacks are expensive because sensor node, heavily, naturalness by using data glove. The weakness of glove user has to wear the glove every time. But, the colour gloves are inexpensive; no sensors are embedded in or outside the gloves and robust method for hand gesture recognition system.

• •    The second way used one or more camera to collect raw input hand gesture called vision based approach. The strength of this approach is natural and more convenient for communication. But, easily affected by complex background.

• •    The last approach is hybrid approach to collect raw hand data by combining the above two methods.

• IV.    Datasets

NUS Hand Pose Dataset I consists of 10 classes of posture, 24 samples per class with uniform simplest background condition.

NUS Hand Post Dataset II [4] is a dataset for hand gesture recognition with complex background condition. NUS hand pose dataset II are defined 10 different static gestures of 750 hand images with human noise, the hand with 2000 hand images, and 2000 background images. In this dataset, each gesture has 5 times with 40 different subjects. The age ranges of 40 subjects are 22 to 56 years. The class types of gestures are alphabet character a to j. The posture images take form 40 subjects and the size of images (160*120). All the images are RGB colour space and the format of the images are .jpeg.

The sample frames of these two datasets as shows in Fig. 2.

(a) NUS Hand Post Dataset I

Y

Cb

Cr

■ 16"	1	" 65.738   129.057   25.064 "
128	+	- 37.945 - 74.494  112.439
128	256	112.439   - 94.154 - 18.285

R

G

B

(b) NUS Hand Post Dataset II

Fig.2. Some Sample Hand Gesture Images form two different datasets

• V.    YCbCr Colour Space

There are many colour spaces in digital image processing such as RGB, HSV, YCbCr, Lab, YIQ, CMYK and many others. We should note the definition about these colour space models which are mostly used in image processing field:

According to the state of art research, YCbCr is the most suitable for human skin colour detection. YCbCr is more accurate than RGB in clustering characteristic. In YCbCr colour space, the channel Y can eliminate to improve the performance in various illumination conditions and also reduce the dimension than RGB. HSI and other colour space also eliminate the luminance component but YCbCr colour space has better advantages. Y describes the colour luminance or colour brightness and more sensitive for the human eyes. Cb and Cr is the chrominance component of the colour and less sensitive for the human eyes. The calculation process of YCbCr is very simple. YCbCr skin colour based segmentation is explained in [1,2,5,9,10].The YCbCr colour model is shown in Fig 2. In our experiment, firstly convert RGB to YCbCr by using the following equation (1).

Fig.3. The YCbCr Colour Space Model

• VI.    Experimental Section

In the previous literature, the researchers used in different colour spaces and various threshold values for human skin colour detection. Skin colour thresholding based method is very popular, easy to implement, and most often used to detect/segment the human skin. The threshold value for skin colour is difficult to set uniformly because of different types of skin colour tone, various illumination conditions. According to the start-of-art research, the different threshold values of YCbCr, RGB, Normalized RGB, HSV colour spaces have been used for human skin segmentation. In this paper, only YCbCr colour space has been used to segment the human hand as shown in the following Table 1.

| 1 Cb (77,127) && Cr (133,173)  ^

I          0; otherwise

Where Y, Cb, Cr =[0;255] and 0=black and 1=white;

Table 1. Different Threshold Values for Human Skin Colour Detection