Real Time Implementation of Driver Drowsiness Monitoring System Using SVM Classifier

In our day today’s life transportation has become an important part especially for those who work on daily basis. For this reason driving is becoming fatigue when human beings travel for long distances every day. Long distance driving leads to Drowsyness it is also known as driver fatigue or tired driving. Due to lack of rest the driver who is operating the vehicle feels fatigue or sleepy or tired. This may lead to severe damage to human life. The main effects of drowsy driving are the inability to focus, delay in reactions, improper judgment towards distances and speeds. Every year, drowsy driving accounts for about 71,000 injuries, 1,00,000 crashes and 1,550 fatalities, according to the National Safety Council (NSC). Taxi, bus, truck and car drivers are the people who travel long-distances and suffer from lack of sleep. Due to which it becomes very dangerous to drive when feeling sleepy. So to prevent these accidents we have designed a system using Raspberry Pi, OpenCV, Dlib and SVM algorithm which takes video as input for facial detection and facial landmark detection to obtain EAR and MOR parameters helping the driver by predicting the drowsiness and alerting when falling sleepy. Later longitude and latitude co-ordinates that is the location intimation are sent to owner of the vehicle.

2.    Related Work

In [1] Belal Alshaqaqi; Abdullah Salem Baquhaizel, “Driver Drowsiness Detection System”.

An AI based Advanced Driver Assistance System (ADAS) was introduced for driver safety for drowsiness. It tracks visual information of the driver face and eyes to calculate PERCLOS (percentage of eye closure). IR camera used here operates under dark and night conditions.

In [2] Alireza Rahmani Azar, Farhad Khalilzadeh, “Real Time Eye Detection Using Edge Detection and Euclidean Distance”.

In this Euclidian distance based eye edge detection method is used. Here skin area is analysed specifically using a skin detection method which employs few formulas in two color spaces differently. In the skin area, horizontal edges are tracked by combining the two known prewitt, sobel and masks values. By using dilation and erosion morphological operations minute edges are removed and close edges are connected. The face area is divided into three sections: the upper left and right quarters and the lower half. In each section, the very large binary object is distinguished as the right eye, the left eye and the mouth. To verify the procedure, the distance between mouth center and left eye center is calculated. Similarly mouth center to the right eye center is calculated. The proposed method is tested on PICS, it includes a database of face images which gives 93% accuracy in an acceptable time.

In [3] Mandi Sushmanth Reddy; R.Ezhilarasie; S.Kanimozhi Suguna, “Fatigue Detection Using Raspberry Pi 3”.

A Raspberry pi3 model with 5-megapixel camera is used here to detect driver’s fatigue. The Camera captures driver’s eyes and face and processes the images to detect driver’s drowsiness. When the driver is found sleepy he will be alerted by an alarm. They have used HAAR Cascade Classifier to detect the Eye blink duration of the driver and Eye Aspect Ratio (EAR) calculations. Finally, the alert message through Ubidots cloud service and Twilio API is sent to the concerned person mobile along with car plate number. For the testing purpose around 10 set of peoples data is collected to capture eye-opening and closing and is evaluated for the given threshold. If the person closes his eyes for more than fixed threshold range the alert signal is generated to wake-up the driver.

3.    Methodology

The block diagram explains the working mechanism of the designed system. Where Raspberry Pi camera is placed in front of driver so that driver’s frontal face can be captured in the form of video. From the video the frames are extracted to form 2D images. System uses Dlib and a linear support vector machine (SVM) for face detection and facial landmarks detection, after landmarks detection Eye Aspect Ratio (EAR) and Mouth Opening Ratio (MOR) are being calculated which provides the status of driver about the drowsiness. If driver is found to be drowsy an alarm will alert the driver The flowchart of the proposed system is depicted in Fig.1.

The proposed model can be divided into two sections tracking and face detection.

• 3.1    Tracking

• 3.2    Face Detection

Initially, GPS (Global Positioning System) is connected to raspberry pi using USB serial port with the help of USB to TTL converter, we continuously read locations from GPS receiver and wait for GPRMC (General Purpose Remote Machining Center) message which contains information related to date, time and position. Once GPRMC message is received from GPS, we parse GPRMC message using PYNMEA2 which is python library to access latitude and longitude which helps to identify the location of driver.

For face detection, we have used a raspberry pi camera. With OpenCV, we capture a video from the camera. We load the SVM classifier and create two objects that are detector and predictor. The detector is to detect the frontal face and the predictor uses an SVM classifier to identify facial landmarks which is the right eye, left eye and mouth. Frames from the video are extracted and converted to grayscale and later these 2D frames are given to SVM detector which looks for the face and if the face is present it is given to SVM predictor which sets the co-ordinates of the eyes and mouth. Later Eye aspect ratio (EAR) and Mouth opening ratio (MOR) is calculated based on Euclidean distance. Table1shows the Facial landmark points for EAR and MOR calculations and Fig. 2 provides the landmarks with points located around eye and mouth on extracted facial area.

Table 1. Facial Landmark Points

Parts	Landmark Points
Right eye	[P0-P5]
Left eye	[P0-P5]
Mouth	[P0-P11]

Fig. 1. Flowchart of driver drowsiness detection system

Fig. 2. Facial Landmark Points

Eye Aspect Ratio (EAR): It is the ratio that helps to determine the status of an eye. Its an Euclidean distance between the eyes, where horizontal and vertical distance is calculated. As fig2 shows (P1, P5) and (P2, P4) represent vertical distance and, (P0, P3) represents horizontal distance which gives threshold value of 0.25 when tested manually. When eyes are open and stable, the threshold value is normal, and if the driver is found to be sleepy, the value decreases towards the zero. The EAR is calculated using eq.1 with the vertical and horizontal distances obtained from facial landmark points.

EAR =

Sum of Verticle distance 2*Horizontal Distance of Eye

Mouth Opening Ratio (MOR): It is the parameter which is used to identify yawning of a person during driving the vehicle, and is measured in similar way as EAR. As we know mouth opens wide during yawning, creating a big hole on the face which is nothing but no skin area on a face which gives a mouth opening ratio (MOR). MOR is calculated based on vertical and horizontal points of mouth. From fig.2 (P2, P10) and (P4, P11) represents vertical distance, and (P0, P6) represents horizontal distance which gives an approximate threshold of 0.75 when tested manually. When a person yawns he/she crosses the threshold that is the value becomes greater than 0.75 indicating that the person is yawning. The MOR is calculated using eq.2 with the vertical and horizontal distances obtained from facial landmark points.

MOR

Sum of Verticle distance 2*Horizontal Distance of Mouth

Fig. 3. Hardware Block Diagram

The Fig. 3 shows the hardware block diagram of the Drowsiness Detection system, the system consists of raspberry pi, pi camera, GPS, and GSM module. Initially, the camera is positioned in front of the driver and the video is captured to create 2D images. Further face detection is performed on frames with the help of Dlib which is a library for implementing ML algorithms and a linear support vector machine (SVM) classifier, to detect the facial expression of a driver. Raspberry pi works as a CPU of the system and helps to co-ordinates with other modules to perform the operation. It consists of GPS which is a radio navigation system and can be used to determine the exact location of a vehicle. It has GSM for mobile communication used to send and receive a messages to and from the owner of the vehicle when drowsiness is detected. We have added buzzer to the system which produces sound when driver is found sleepy.

4.    Results and Discussion

We have successfully developed a real time prototype for driver drowsiness detection, which successfully calculates EAR and MOR which has threshold value of 0.25 and 0.75 respectively. The system is designed in a way that when eyes are in close state the threshold value decreases towards zero and similarly for detection of yawning the threshold value exceeds 0.75 that is the value gets increased. We experimented the designed system in real time for various iterations where we found a delay of 2 seconds so we can say that driver gets a buffer of 2 seconds. The below Table 2 and 3 shows various parameters of EAR and MOR which represents drowsiness detected and not detected respectively, based on the above threshold values. In all the iterations the proposed prototype successfully detects the drowsiness leading to accuracy 95%.

Similarly Fig.4 and 5 depicts the graphical representations of the EAR and MOR parameters for variation interactions indicating the status of drowsiness as detected or not detected based on the threshold values during day and night time tested in real time. The evaluation and detection of drowsiness of 2D image with eye and mouth status detection tracking for the calculation of EAR and MOR is depicted in Fig.7 and 8 respectively.

Table 2. Parameters of EAR

Sl no	Readings	Drowsiness Detected and Not Detected
1	0.18	✓
2	0.29	X
3	0.24	✓
4	0.28	X
5	0.19	✓
6	0.31	X
7	0.21	✓
8	0.33	X
9	0.17	✓
10	0.27	X

Table 3. Parameters of MOR

Sl no	Readings	Drowsiness Detected and Not Detected
1	0.77	✓
2	0.27	X
3	0.97	✓
4	0.72	X
5	1.09	✓
6	0.33	X
7	1.03	✓
8	0.56	X
9	0.79	✓
10	0.65	X

• Fig. 4.    Graphical representation of EAR

  

• Fig. 5.    Graphical representation of MOR

  

• Fig. 6.    EAR detected

  

• Fig. 7.    MOR detected

5.    Conclusion

The presented driver drowsiness detection system keeps the driver awake and alert if he founds sleepy by monitoring the driver's facial status and determines that the driver is feeling drowsy or not. We have implemented it using raspberry pi, pi camera, GSM, and GPS modules. The system makes use of dlib and SVM algorithm for facial landmark detection by calculating EAR and MOR. If the driver is found to be drowsy an alarm sound will alert the driver and a message is sent to the owner of the vehicle. We were successfully able to develop a model that detects the drowsiness of the driver and alerts him for some iterations leading to overall 95% accuracy.