Driver's Face Tracking Based on Improved CAMShift

• I.    Introduction

  With the increasing popularity of automobiles, the traffic accidents have become severe social problems. Recent statistic shows that the number of fatalities due to road accidents in Malaysia increased 1.9% to 6,872 deaths in 2010 from 6,745 deaths in 2009. In a separate online survey by the community project Malaysians Unite for Road Safety (MUFORS), 61.6% of the respondents believed that human error is the biggest cause for road carnages, while 15.6% of the respondents blamed road conditions [1]. Malaysia is paying a heavy price due to road accidents, and the cost to the economy.

Successfully addressing the issue of driver drowsiness in the commercial motor vehicle industry is a formidable and multi-faceted challenge. A lot of research and experiments have been done in the last decade. Driver drowsiness detection techniques can be broadly classified into three categories: physiological measurement (such as brain waves (EEG), eye movements (EOG) and heart rate (ECG)), visual cues (such as eyelid movement, facial orientation, as well as yawning) and driving performance (monitor how the driver handles the vehicle) [2]. The first two categories monitor the driver’s bio-signals directly, whereas the third category monitors the driver indirectly. The first category have to be attached to driver, making this approach both intrusive and impractical, the last two categories have non-intrusive nature. By reason of its non-intrusive nature, considerable accuracy and fully portability, the visual cues are the most widely used technique to detect the driver fatigue.

Object tracking is a fundamental and an important problem of dynamically extracting two-dimensional (2D) information in most visual applications including image processing, computer vision, video surveillance, humancomputer interaction (HCI). With the proliferation of high-powered personal computers and portable and lowcost video cameras has brought forth a large numbers of algorithms in object tracking. There are many different approaches for tracking an object proposed to overcome the difficulties arising from noise, similar color distribution, and complex background environment.

The CAMShift tracking algorithms stands out as its simplicity and efficiency. CAMShift stands for Continuously Adaptive Mean Shift and it was first proposed by G. Bradski et al [3], aiming at efficient head and face tracking from a stationary camera in a perceptual user interface but has since been modified for a variety of other tracking situations[4][5].

The CAMShift algorithm was derived from the earlier Mean Shift algorithm [6] and is a simple, yet very effective, color-based tracking technique. It is proved by lot of proposed work. Bogdan kwolek [6] track human head and body respectively in order to tracking people. In the paper [7], color-histogram was presented for tracking; it is insensitive to small object pose change due to discard spatial information. Zhaowen Wang et al [8] used particle filter and CAMShift to Increase the robustness, but the calculation time of particle filter reduce the real-time performance. In the paper [9], David Exner, Erich Bruns et al had accomplished CAMShift efficiently on the GPU; all of the excellent performance makes full use of a GPU’s high parallelization capabilities. That means need special hardware support.

This paper proposed an improved CamShift algorithm based on segmentation of skin color in H channel of VSH space to increase the robustness performance. Local back projection was proposed to saving the calculation times and increases the weight of the skin color pixel. Due to captures a richer description of target, the improved CamShift algorithm increase robustness in complex background circumstance.

• II.    Algorithm ANALYSIS

From the side of defects in CAMShift algorithm, it can help us to know how to make use of this algorithm properly. The defects of CAMShift algorithm are described as follows:

• >    The CAMShift algorithm is semiautomatic tracking algorithm. It needs be given the target object position information by other object detection algorithm in the first frame to do object tracking.

• >    The CAMShift algorithm is sensitive to color, so if the similar color distribution exist in background of target object. In the end, the CAMShift will lost the target object or locate wrong object.

• >    The CAMShift algorithm is suitable for uniformcolored object tracking, so it will fail in high probability when using to track multi-colored object.

• >    Although the CAMShift algorithm uses HSV color space, it has resistance against changing illumination. But when illumination changes intensively, this algorithm wills underperformance

When detecting driver’s fatigue in the vehicle, it is usually to install a camera in front of the driver. In the captured video frame, the vehicle driver’s hand or face is the biggest object comparing other object in the video frame. In other words there have seldom other objects to disturb the CAMSfit algorithm. Although driving car on the road, the background behind vehicle driver is almost stationary. Human being’s face is uniform-colored distribution object. Light changing in vehicle can affect the performance of CAMShift, but the result can acceptable. Using only hue from the HSV color space gives CAMSHIFT wide lighting tolerance [16]. All the analysis shows that CAMShift is suitable to tracking deriver’s face in vehicle

In this paper, firstly, main image segmentation algorithms are classified and reviewed; then evaluation and comparison of image segmentation algorithms are discussed in depth based on the reason that evaluation of image segmentation is essential in the aspect of comparing the segmentation algorithm and providing advice for improvement.; at last, evaluation results of typical segmentation algorithms in MATLAB environment are summarized and presented.

• III.    Methodology

When using CAMShift algorithm to track object, the first step is that convert color video frame to HSV. After this step, the other steps can named as back projection, Mean Shift and CAMShift Procedure. All the steps will be present in detail in this paper.

A. HSV COLOR SPACE

HSV are the two most common cylindrical-coordinate representations of points in an RGB color model, which rearrange the geometry of RGB in an attempt to be more perceptually relevant than the cartesian representation. HSV stands for hue, saturation, and value, and is also often called HSB (B for brightness) [13]. Hue is the attribute of a visual sensation according to which an area appears to similar to one of the perceived colors: red, yellow, green and blue, or to a combination of two of them. It can be described in terms of an angle on the above circle. Although a circle contains 360 degrees of rotation, the hue value is normalized to a range from 0 to 255, starting from red. Saturation measures the departure of a hue from achromatic, i.e., from white or gray. In other words, saturation represents the vibrancy of the color. Its value ranges from 0 to 255. The lower the saturation value, the more gray is present in the color, causing it to appear faded. Value measures the departure of a hue from black, the color of zero energy. It ranges from 0 to 255, with 0 being completely dark and 255 being fully bright. From figure 1, it can be explained more clearly.

Figure 1. HSV color cylinder [13]

RGB to HSV Algorithm [14]:

Given: R, G and B, each on domain [0, 1]

Desired: The equivalent H, S, and V, each on range [0, 1]

• 1.    V: = max(R,G,B);

• 2.    Let X: = min(R,G,B);

• ^3.    S := ------;if S = 0 return;

,      V - X___

V

V - R _ V - G .V - B

Let r := -------; g := -------; b := ;

V - X    V - XV - X

If R=V then H: =(if G=X then 5+b else 1-g);

If G=V then H :=( if B=X then 1+r else 3-b);

Else H = (if R=X then 3+g else 5-r);

corresponding to the histogram of the pixel, which can be showed with high brightness. This probability distribution of the color can be showed in Fig. 3. Figures (a) are the RGB picture of target object. Figures (b) are the histogram of target object in V channel of HSV color space. Figures(c) are original image. Figures (d) are back projection of whole image according target histogram in H channel.

H := H

The image of H, S and V channels is as follows:

(a)                (b)                      (c)                   (d)

(a) RGB image               (b) H channel image

(c) S channel image              (d) V channel image

Figure 2. The image of H,S and V channels

(a)                 (b)

(c)                   (d)

(a)              (b)                 (b)                      (d)

Figure 3. Histogram and back projection

B. BACK PROJECTIONS

Back projection is a primitive operation that associates the probability of being part of the tracked object in the image of each pixel with the value of the calculated color histogram [10]. The histogram is calculated as

n

qu= a d[p(xi)- u], u = L..m.                 (i)

i= 1

Here δ is unit impulse function. The p ( ) associating the pixel at with the index of its color bin in the quantized feature space is the image histogram value of the elements in H channel. The formula of histogram normalization is as

C. MEAN SHIFT

The Mean Shift algorithm is used as a way to converge from an initial area for location and scale to the best match based on the color-histogram similarity. Mean shift procedure is applied to find the mass center of the current tracked region, it described by Figure 4.. And the calculated mass center will be used as the initial center of the next frame.

Let a kernel function is given as following:

K(x, - x) = ec x-x-2

qu = min{

255 max(q)

*qu,255}, foru = 1...m.

The range of values in [0, max(q)] can be normalized to the interval of [0, 255] based on formula (2) . Here the function max (q) denotes the maximum value of the histogram. Then the image element value of image corresponding to the value of the histogram for each color can be linked up by back-projection. That is the greatest probability of Neurosurgery which is

This Gaussian kernel function determines the weight of nearby points for re-estimation of the mean. The weighted mean of the density in the window determined by K is

m(x) =

a xiiN(x)xi* K(x- x)

K(xi - x)

x i I N(x) ^v i       ⁷

where N(x) is the neighborhood of x, a set of points for which K(x) ^ 0 . The mean shift algorithm sets x ^ m(X) , repeats the estimation until m(x) converges.

Figure 4. Find the densest region

• D.    CAMSHIFT PEOCEDURES

After convergence of Mean Shift accomplished in previous frame, the mass center will be updated .after that the new mass center be treated as central point of search window in new video frame. In this way, the target object can be found in each frame sequences, no matter the position of the target object.

• E.    SUMMARY OF CLASSIC CAMSHIFT ALGORITHM

The CAMShift algorithm has been used to adaptively meet the size and location of the search window. Current frame’s localization results of the tracked object are reported and used to set the size and location of the search window in the next frame image. Firstly, it is usually converts video frame to HSV colorful space. The CAMShift algorithm is shown as follows [10, 11, 12]:

• (a)    Set the calculation region of the probability distribution to the whole image.

• (b)    Choose the initial location of the 2D mean shift search window.

• (c)    Calculate the color probability distribution in the 2D region centered at the search window location in an ROI slightly larger than the mean shift window size

• (d)    Run Mean Shift algorithm, to get a new location and size of the search window. The centroid of search window can be obtained by calculating the moment of search window as follows :

The zero-order moment: M_oo = ЕЖ у)      ⁽⁵⁾

xy

The first-order moments of x and y:

^M00 = ZZ ^xlc^(x,y) xy

^M00 = ZZ VIc^V) xy

The centroid of search window:

x = Mt , ⁰ M 00

Vo = M 01

⁰   M 00

The size of search window:

s = 2 *

^M00 256

• (e)    In the next frame of video images, use the value obtained by Step c to re-initialize the search window’s location and size, and then jump to Step C to continue to run until convergence.

• F.    IMPROVED CAMSHIFT ALGORITHM

In the back projection step of classic CAMShift, looking for exist color pixel in H channel histogram and changing value of each pixel are time-consuming works. Unfortunately, the back projection step will map object H channel histogram into whole image through the whole period of tracking. In paper [17], skin color segmentation using H threshold is presented, if pixels belong to range of skin color then the pixel’s value in V channel equal one, otherwise the value is zero in H channel. But the value of skin in V channel is affected by illumination.

In different light condition, the central point of the skin distribution is different. As shown in the Figure 3.b, the uniform-colored object has single peak in its histogram chart in regular light condition. Therefore, a self-adaptive threshold method was presented in there. In the frame of video, the histogram of target object was calculated in H channel of HSV. Then, the peak of histogram will be selected; the value of bin (P) in peak is the center-point of skin-color distribution. In the paper, the values of H channel are divided into 256 levels (bins) from 0 to 255.after that According experiments, the range [P-13, P+13] can cover the color of skin, this threshold will be used to segment sequent frames. The segmentation is shown in the Figure 5.

In general, they have the same function for find target object in video frame. So, this step will be substituted for the back projections step. By using proposed method, there are two aspects contribute to tracking algorithm. Firstly, driver’s face will be found very quickly using proposed algorithm and it will improve real-time performance. Secondly, self-adaptive threshold is going to be used that can enhance the robust performance of light changing for tracking algorithm.

Figure 5. Self-adaptive threshold segmentation

The CAMShift algorithm can cooperate with face detection smoothly. One of the most classic and power face detection methods is Viola-Jones detector proposed by Paul Viola and Michael Jones in Second International Workshop On Statistical and computational Theories of Vision[15].The Viola-Jones detector is a strong binary classifier build of several weak detectors as shown in Figure 6. Each weak detector is an extremely simple binary classifier, those weak classifier trained by Harr-like features as Figure 7 shown. Casusing Viola-Jones face detector insensitive to color distribution; it can provide accurate position information to CAMShift tracking system. Other face detection methods, for example skin-color classifier, also can work smoothly with CAMShift.

Figure 6. Schematic depiction of a the detection cascade

Figure 7. The feature prototype of simple Harr-like

The tracking process of improved CAMShift can be denoted on figure 8.

Figure 8. Flow chart of improved CAMShift

• IV.    EXPERIMENTS

In this phase, two video will be used to verify the validity the algorithm proposed in this paper. The hand tracking video (red cup) is downloaded from internet.

Figure 9 is the result of improved CAMShift algorithm and classic CAMShift algorithm.to track hand. The above images are result of the improved CAMShift algorithm; the below ones are the classic CAMShift algorithm result. Figure 10 is the motion trajectory of tracked hand. The left image is outcome of the improved CAMShift algorithm the right one is outcome of classic CAMShift algorithm.

a. Using improved CAMShift

b.Using Classic CAMShift

Figure 9. Tracking result

Figure 10. Motion trajectory

Reinier Coetzer, Driver fatigue detection based on eye tracking, in Southern Africa Telecommunication Networks and Applications Conference (SATNAC), 2010.

Bradski G R. Computer V ision F ace T racking F or Use i n a Perceptual User In terface[M].Intel T echnology Journal,1998.

J. G. Allen, R. Y. D. Xu and J. S. Jin, Object tracking using camshift algorithm and multiple quantized feature spaces, in Proceedings of the PanSydney area workshop on Visual information processing, ACM International Conference Proceeding Series Vol. 100 (Australian Computer Society, Inc., Darlinghurst, Australia, 2004).

N. Liu and B. C. Lovell, Mmx-accelerated real-time hand tracking system, in IVCNZ 2001, (Dunedin, New Zealand, 2001).

Bogdan Kwolek. CamShift - based tracking in joint color-spatial spaces [J] . Computer Analysis of Images and Pat2 terns, 2005 (3691): 693 - 700.

Z. Zivkovic and B. Krose, An em-like algorithm for color-histogram based object tracking, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2004.

Figure 11. Results of face tracking

• [8]    Zhaowen Wang, XiaoKang Yang , Yi Xu, CamShift guided particle filter for visual tracking, Signal Processing Systems. China:  Shanghai:  IEEE

  Workshop, 2007: 301- 306..

• [9]    David Exner, Erich Bruns, Daniel Kurz, and Anselm Grundh ὅ fer, Fast and Robust CAMShift Tracking, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2010.

• [10]    Xia Liu, Hongxia Chu, Pingjun Li. Research of the Improved Camshift Tracking Algorithm. IEEE International Conference on Mechatronics and Automation, PP 5-8, August, 2007.

• V. CONCLUSIONS

In this paper, an improved CAMShift algorithm according the driver’s head motion was proposed. Experimental results indicate that the overall performance of the proposed algorithm is rapid and reasonable compared with the classic algorithms. The performance of improved algorithm has considerable robustness when illumination changing. However this algorithm still has space to improve such as if an object similar to face locate in sub-window, the algorithm will miss of target object. These will be addressed in future works.