Face Mask Recognition by the Viola-Jones Method Using Fuzzy Logic

Currently, there is a need for the development of reliable, fast and easy-to-use face mask recognition systems. Such recognition systems will allow monitoring the presence of people without masks in vehicles, in the premises of educational institutions, shopping centers, etc. Detecting people with and without masks is important for security purposes, in particular, in quarantine conditions. In addition, the use of a face mask recognition system in educational institutions will allow determining the number of people in the room and their spatial location, taking into account the results of recognition in the educational process.

Obtaining images of faces using digital video cameras is relatively simple. However, the process of recognizing faces in images with or without masks is much more complicated. This is due to different lighting conditions, different distances of people from the video cameras, people's turns relative to the video cameras, as well as due to the presence of additional objects in the frame: glasses, hats, etc. Therefore, various methods using Artificial Intelligence (AI) technologies are currently used to recognize face masks. The process of recognizing a mask on a face image is usually divided into two stages: 1) localization of the face area; 2) detection or non-detection of the mask on the face.

One of the common methods of recognizing images of faces and masks on faces is the use of Artificial Neural Networks (ANN), in particular, Convolutional Neural Networks (СNN), which are effective specifically for image processing [1]. The effectiveness of CNN for image recognition is demonstrated in [2] and training of CNN with the ImageNet architecture was carried out. It is shown that even for a large training sample (more than 1 million images), CNN training can be performed in an acceptable time. The possibility of effective recognition of a mask on a face using CNN with VGG-16, AlexNet and ResNet-50 architectures has been shown by researchers [3]. The detection of a mask on a face by CNN with the FMDNet architecture was performed by the authors [4]; for the used dataset of images, a recognition accuracy of 99% was obtained. In the work [5] considered the possibilities of face detection in images using CNN, where a person's face was highlighted in the image as a Region of Interest (ROI). A Different Scales Face Detector (DSFD) based on Faster R-CNN was used, which allow to detect faces with fairly high accuracy even at small scales. In the process of training CNN, the dataset can be divided into minibatches, which ensures high accuracy of face recognition (96.6% for Yale Datasets) [6]. When training a CNN, the input data (images) are divided into a training dataset and a validation dataset, which is designed to prevent overtraining.

Images of faces can be recognized by CNN not only in the visible range, but also in the infrared. The researchers in [7] showed that analysis of Thermal Images allows accurate Facial Expression Recognition (FER). However, infrared imaging requires special video cameras. Analysis of a person's emotional state using FER is also performed using Scale Invariant Feature Transform (SIFT) [8]. Such an analysis of a person's emotional state is especially useful when organizing the educational process, for example, for assessing students' concentration and attention. By CNN it is possible to recognize not only faces with high accuracy [9, 10], but also masks on faces. However, this requires extensive training using appropriate datasets.

Face images in most cases differ in texture from other areas of the images, which was used for face recognition in [12] for building a biometric system. Face recognition based on their texture was performed using Local Binary Pattern (LBP) and Gray Level Co-occurrence Matrix (GLCM), while three classifiers were used: Euclidean distance, Manhattan distance and Cosine distance. In the paper [13] the Discrete Complex Fuzzy Transform (DCFT) method was used to recognize face images, which consists of the stages of histogram extraction, peak points of histogram calculation and image construction. To create images, polynomial and smith fuzzy sets are used. In the classification phase, support vector machine (SVM) and k Nearest Neighborhood (KNN) classifiers are used. Due to the descriptor SURF (Speed Up Robust Features) [14] and the descriptors Histogram of Directional Gradient (HDG) and Histogram of Directional Gradient Generalized (HDGG) [15] high accuracy of image recognition is ensured. However, the proposed methods using textures, special descriptors (SURF, HDG, HDGG and DCFT) [12, 13, 14] are designed for recognizing faces, not masks on faces.

If the experimental images contain a significant level of noise (for example, Gaussian or impulse), then before face recognition, pre-processing of the images is performed by filtering noise in the spatial or frequency domains [16, 17]. Filter parameters are determined taking into account the noise level in the image [19, 20]. Image pre-processing also consists of segmentation [21], contour selection [22, 23] and contrast enhancement [24, 25, 26], which makes it possible to increase the accuracy of face recognition using CNN [27].

Detection of face regions in images is often performed by the Viola-Jones method [28], which uses Haar cascades and is one of the best in terms of accuracy and processing speed. Also, this detector has a low probability of false identification of a person. There are Haar cascades designed for detecting frontal face images and profile face images, as well as for detecting facial features (eyes, mouth). Complex use of the results of different Haar cascades potentially allows to increase the accuracy of face recognition. For example, if a face, eyes and mouth are detected simultaneously in a certain area of the image, then a face without a mask is also recognized in such an area with high confidence. An important advantage of the Viola-Jones method is the use of trained Haar cascades, which avoids a complex and lengthy training procedure (as in the case of ANN). Therefore, in the paper proposes to use the Viola-Jones method to detect (localize) faces and their features, due to which it is possible to solve the urgent task of building a system for recognizing images of masks in faces. However, due to the difficulty of detecting faces and their features in real images, the detection confidence value can take not only the minimum or maximum, but also an arbitrary intermediate value. For this reason, the results of detecting faces and their features in images are described by means of fuzzy logic [29, 30].

2.    Theoretical Foundations of Face Mask Recognition Using the Viola-Jones Method and Fuzzy Logic

The face mask recognition system [31, 32, 33] uses the Viola-Jones method to detect (localize) the face and its features (eyes, mouth), and artificial intelligence tools [34, 35] are used for analysis detection results. Fuzzy logic is used as a means of artificial intelligence [29, 30], which allows to quantitatively assess the confidence of face detection and its features.

The Viola-Jones method is based on the use of Haar cascades (detectors) [36]. Haar detectors consist of contiguous rectangular areas of the image. They are positioned on the image, then the pixel intensities in different areas are summed, after which the difference between the sums is calculated. Such a difference is the value of a certain Haar feature (signature). At the stage of object detection using the Viola-Jones method, a window of a fixed size moves across the image, and for each area of the image over which the window passes, the Haar feature is calculated. The presence or absence of an object in the window is determined by the difference between the value of the feature and the set threshold. A significant number of Haar features are required for accurate object detection, so in the Viola-Jones method, the features are organized into a cascade classifier. The simplest rectangular Haar feature can be defined as the difference of the pixel sums of two adjacent areas inside a rectangle, which can occupy different positions and scales in the image. This type of features is called 2-rectangular. Similarly, 3-rectangular and 4-rectangular features are defined. In the original version of the Viola-Jones algorithm, only detectors (primitives) without turns were used. Later, 45-degree tilt detectors and asymmetrical configurations were proposed.

The result of the Viola-Jones method is the position of the objects in the image, which are described by the following parameters: x and y are the coordinates of the upper right corner of the rectangle (which limits the desired object, for example, a face), w is the width, h is the height of the rectangle. To localize objects in images, the Viola-Jones method uses a set of discrete scale values for the scanning window.

When detecting objects (for example, faces) using the Viola-Jones method, a certain value of the minNeighbors parameter is pre-set (in many cases, the value of the parameter is set by default). The minNeighbors parameter, which specifies how many neighbors each candidate rectangle (detected object area) must have in order to save it (the value of minNeighbors varies from 1 to 6). For each face region, the face detection confidence value D f (which ranges from 1 to 6) is calculated as the maximum value of minNeighbors for which the face is still detected (larger D f means more confident face recognition). Similarly, for each eye region, the eye detection confidence value D e (which ranges from 0 to 5) is calculated as the maximum value of minNeighbors at which an eye is still detected (a larger D_e means an eye is detected with greater confidence). For each mouth region, the mouth detection confidence value D_s (which ranges from 0 to 5) is calculated as the maximum value of minNeighbors at which a mouth is still detected (a higher D_s means a more confident mouth is detected).

In the source images, the Viola-Jones method detects the face, eye and mouth regions, which allows to recognize faces without a mask and with a mask on the face with high reliability. A face without a mask is recognized with high confidence if the face and mouth are detected with high confidence. A mask is recognized with high confidence if the face and eyes are detected with high confidence and the mouth with low confidence.

Face recognition based on detected face and eye regions is performed by a fuzzy system whose input parameters are face detection confidence D f and eye detection confidence D e , and the output parameter is face recognition confidence D fr (which lies in the range of 1 to 6).

Mamdani fuzzy knowledge base # 1 [30] has been developed for face recognition, which contains 36 rules, for example:

Rule # 1; If DfL = Low and DeL = Low, then DfrL = Low.

Rule # 6; If DfL = Below Average and DeL = Below Average, then DfrL = Below Average.

Rule # 14; If DfL = Above Average and DeL = Below Average, then DfrL = Average.

Rule # 20; If DfL = Average and DeL = Average, then DfrL = Above Average.

Rule # 30; If DfL = Below High and DeL = Above Average, then DfrL = Below High.

Rule # 33; If DfL = High and DeL = High, then DfrL = High.

In the rules of the knowledge base, the notation DfL describes the linguistic variable "Confidence of face detection", DeL – "Confidence of eye detection", DfrL – "Confidence of face recognition taking into account the eyes".

The terms of the linguistic variable DfL «Confidence of face detection» are Low, Below Average, Average, Above Average, Below High, High (which are numbered from 1 to 6), and the numerical linguistic variable is described by the face detection confidence D f (carrier of fuzzy set). Analogous terms describe the linguistic variables DeL "Confidence of eye detection" (with numeric input variable D_e ) and DfrL "Confidence of face recognition taking into account the eyes" (with numeric output variable D f _r ).

The terms of the linguistic variable DfL "Confidence of face detection" are described by fuzzy sets with triangular membership functions μ f1 ( D f ), μ f2 ( D f )… μ f6 ( D f ) and the carrier D f (Fig. 1). Similarly, the terms of the linguistic variable

DeL "Confidence of eye detection" are created, which are described by fuzzy sets with triangular membership functions: μ_e1( D_e ),..., μ_e6( D_e ). The terms of the linguistic variable DfrL "Confidence of face recognition taking into account the eyes" are described by fuzzy sets with triangular membership functions: μ fr1 ( D fr ),..., μ fr6 ( D fr ).

Fig. 1. Fuzzy membership functions of the linguistic variable DfL "Confidence of face detection": μf1 (red graph) – the confidence of face detection is Low; μf2 – Below Average, μf3 – Average; μf4 – Above Average; μf5 – Below High; μf6 – High

An initial image is fed to the inputs of the face recognition system, on which faces are detected with D fC confidence and eyes are detected with D eC confidence using the Viola-Jones method.

Based on the obtained values of D fС and D eС , the values of the membership functions μ f(m) ( D fС ), where m = 1,..., Q m ( Q_m = 6) and μ_e ( _n ) ( D_eС ), where n = 1,..., Q_n ( Q_n = 6) were calculated (for the corresponding terms). Based on the fuzzy knowledge base, the values of membership functions μ_fr ( _k ) ( D f _rC ), where k = 1,..., Q_k ( Q_k = 6), were calculated according to the formula

µ fr(k)(DfrC) =min(µ f(m)(DfC),µe(n)(DeC)) , where m is number of the membership function μf, n is number of the membership function μe, k is number of the membership function μfr.

Membership function numbers are determined according to the rules of the fuzzy knowledge base. For example, for rule # 14: DfL = Above average ( m = 4), DeL = Below average ( n = 2), DfrL = Average ( k = 3), the values of membership functions μ fr3 ( D frC ) were calculated

µ f_{r 3} ( D f_rC ) = min( µ f ₄ ( D f_C ), µ _{e 2} ( D _eC )) .

After that, the composition of the fuzzy membership functions μ f _r ( _k ) ( D f _r ), where k = 1,..., Q_k , is performed, as a result of which the fuzzy set μ f _Ua ( D f _r ) is calculated according by formula

µ _fUa ( D _fr ) = max( µ _{fr 1} ( D _fr ),.., µ _{fr ( k )} ( D _fr ),..., µ _{fr 6} ( D _fr )) .                             (3)

On the basis of the fuzzy set μ fUa ( D fr ), its defuzzified value D frС is calculated by the method of the gravity center according to the formula

Qk

∑ D fr ( k ) ⋅µ fUa ( D fr ( k )) k = 1

.

^DfrC ⁼     Qk

∑ µ fUa ( D fr ( k )) k = 1

where D f _r ( k ) are discrete values of the D f _r parameter (from 1 to 6).

Thus, based on the parameters of the detected face, eyes and mathematical model (1-4), the face recognition (with confidence D f _rC ) is performed by fuzzy logic. During such fuzzy inference, it is taken into account that if eyes are detected within a face region with high confidence, then such a face is recognized with high confidence. Conversely, if eyes are detected within a face region with low confidence, then such a face is recognized with low confidence.

Face mask recognition based on recognized face regions (using fuzzy inference) and detected mouth regions (obtained by the Viola-Jones method) is performed by a fuzzy system whose input parameters are face recognition confidence D fr and mouth detection confidence D s , and the output parameter is mask recognition confidence D m (which lies in the range from 0 to 5).

Mamdani fuzzy knowledge base # 2 has been developed for mask recognition, which contains 36 rules, for example:

Rule # 1; If DfrL = Low and DsL = Medium, then DmL = Low.

Rule # 6; If DfrL = High and DsL = Below Average, then DmL = Below Average.

Rule # 14; If DfrL = Low and DsL = Low, then DmL = Medium.

Rule # 20; If DfrL = Medium and DsL = Low, then DmL = Above Medium.

Rule # 30; If DfrL = Above Medium and DsL = Low, then DmL = Below High.

Rule # 36; If DfrL = Lower High and DsL = Low, then DmL = High.

In the rules of the knowledge base, the notation DfrL describes the linguistic variable "Confidence of face recognition taking into account the eyes", DsL – "Confidence of mouth detection", DmL – "Confidence of mask recognition".

The linguistic variable terms are the values Low, Below Average, Medium, Above Average, Below High, High (which are numbered from 1 to 6). For the linguistic variable DsL "Confidence of mouth detection" the input variable is D s . For the linguistic variable DmL "Confidence of mask recognition" the output variable is D m .

The terms of the linguistic variable DsL "Confidence of mouth detection" are described by fuzzy sets with triangular membership functions μ_s1( D_s ), μ_s2( D_s )… μ_s6( D_s ) and the carrier D_s . Similarly, the terms of the linguistic variable DmL "Confidence of mask recognition" are created, which are described by fuzzy sets with triangular membership functions μ_m1( D_m ),..., μ_m6( D_m ).

Mouth regions are detected on the input image using the Viola-Jones method with D_sC confidence. Based on the predefined face recognition confidence D fr and mouth detection confidence D sС , the values of membership functions μ fr ( m ) ( D frС ), where m = 1,..., Q m ( Q m = 6) and μ s ( n ) ( D sС ), where n = 1,..., Q n ( Q n = 6) were calculated (for the corresponding terms). Based on the rules of the fuzzy knowledge base, the values of the membership functions μ m(k) ( D mC ), where k = 1,..., Q k ( Q k = 6), are calculated according to the formula

µ m ( k ) ( D mC ) = min( µ fr ( m ) ( D frC ), µ s ( n ) ( D sC )) ,                                (5)

where m is number of the membership function μ_fr, n is number of the membership function μ _s , k is number of the membership function μ _m .

Membership function numbers are determined according to the rules of the fuzzy knowledge base. For example, for rule #20: DfrL= Medium ( m = 3), DsL = Low ( n = 1), DmL = Above Medium ( k = 4), the values of membership functions μ m4 ( D mC ) were calculated

µ _{m 4} ( D _mC ) = min( µ _{fr 3} ( D _frC ), µ _{s 1} ( D _sC )) .

After that, the composition of the fuzzy membership functions μ m(k) ( D m ), where k = 1,..., Q k , is performed, as a result of which the fuzzy set μ mUa ( D m ) is calculated according by formula

µ _mUa ( D f_r ) = max( µ _{m 1} ( D _m ),.., µ _m ( _k ) ( D _m ),..., µ _{m 6} ( D _m )) .                          (7)

On the basis of the fuzzy set μ _mUa ( D_m ), its defuzzified value D_mС is calculated by the method of the gravity center according to the formula

Qk

^DmC ⁼

∑ D m ( k ) ⋅µ mUa ( D m ( k )) k = 1

Qk

∑ µ _mUa ( D_m ( k ))

k = 1

where D m ( k ) are discrete values of the D m parameter.

Thus, on the basis of the parameters of the recognized face, the detected mouth and the mathematical model (5-8), the mask recognition (with confidence D_mС ) is performed by fuzzy logic. During such fuzzy inference, it is taken into account that if a mouth is detected within a face area with high confidence, then a mask is recognized with low confidence on such a face. Conversely, if a mouth is detected within a face area with low confidence, then a mask on such a face is recognized with high confidence.

3.    Software Implementation of Face Mask Recognition

The fuzzy face and face mask image recognition system was software implemented in Python using the Google Colab cloud service (in the Jupyter Notebook) [31, 32, 33] based on mathematical models (1-4) and (5-8). Fuzzy triangular membership functions for linguistic variables are generated by the ''trimf'' method of the scikit-fuzzy library. The operation of the recognition system is described by the following algorithm (Fig. 2).

Fig. 2. Activity diagram of face and face mask image recognition system

The initial (input) data for the program are digital images which read from a video camera or from graphic files. Such images may contain masked or unmasked faces. The initial color image is read as a three-dimensional array f_RGB ( i, k, c ), where i = 0, ..., M -1; k = 0, ..., N -1; M – image height in pixels, N – image width in pixels, c = 0, ..., 2 – color channel number (Red, Green, Blue). Сolor image f_RGB are converted to grey image f , which are processed by software as rectangular matrices f = ( f ( i, k )), where i = 0,..., M -1, k = 0,..., N -1.

On the obtained images f , the Viola-Jones method detects Q_n f of rectangular areas of faces. The following Haar cascades [36] were used for detection in the image of the face, eyes, and mouth:

• 1.    haarcascade_frontalface_default.xml (Detection of frontally turned faces).

• 2.    haarcascade_profileface.xml (Detection of faces in profile).

• 3.    haarcascade_eye.xml (Eye detection).

• 4.    haarcascade_smile.xml (Mouth or smile detection).

In the loop, all faces with numbers n f from 1 to Q_n f are processed. To detect frontally rotated faces, the face_cascade classifier is created based on the Haar cascade (using the opencv library imported into the program as cv2) by the command:

Detection of areas of faces is performed by the command:

face_coord = face_cascade.detectMultiScale(f, scaleFactor, minNeighbors), where f is the initial grey image;

scaleFactor is the parameter that determines how much the size of the scanning window is reduced for each image scale (for example, scaleFactor = 1.1);

minNeighbors is the parameter that specifies how many neighbors each candidate rectangle must have in order to save it (varies from 1 to 6).

For a detected rectangular area, 4 parameters are recorded in face_coord: x – coordinates of the upper left corner of the area by width, y – coordinates of the upper left corner of the area by height, w – width of the area, h – height of the area. The face detection confidence D f is calculated as the maximum value of minNeighbors at which a face is still detected.

Within the rectangular area of each face ( h f × w f pixels), eyes and mouth are detected by the Viola-Jones method using the corresponding Haar cascades. For each eye region, the value of eye detection confidence D e is calculated, and for each mouth region, the value of mouth detection confidence D s is calculated (similarly to face detection).

On the basis of the developed mathematical model (1-4), face recognition is performed taking into account the eyes using Mamdani fuzzy knowledge base #1. Similarly, on the basis of the mathematical model (5-8), a fuzzy inference is performed for mask recognition using Mamdani fuzzy knowledge base # 2.

The output data (results) of the software are the coordinates of the rectangles that correspond to the faces in the image, as well as the face recognition confidence D f _rС (4) and mask recognition confidence D_mС (8) for each such rectangle.

4.    Results and Discussion

The developed software reads initial images from a video camera or from graphic files. Let's consider example # 1 of facial mask recognition using the developed program.

First, a face is detected in the image with high confidence D f _C = 6 (Fig. 3), and then eyes are detected with also high confidence D_eC = 5 (Fig. 4).

Therefore, the initial data of the face recognition system are as follows: D f _C = 6, the linguistic variable DfL "Confidence of face detection" takes the value "High" (Fig. 1); D_eC = 5, the linguistic variable DeL "Confidence of eye detection" takes the value "High". On the basis of the initial data, fuzzy inference is performed using the rules of the developed Mamdani knowledge base #1. At the same time, rule # 33 is fulfilled:

If DfL= High and DeL = High, then DfrL = High.

By defuzzifying the "High" value of the linguistic variable DfrL "Confidence of face recognition taking into account the eyes", a defuzzified value of face recognition confidence D frС = 5.7 (maximum value of 6) was obtained (Fig. 5, Fig. 6a). Therefore, the initial data of the mask recognition system is as follows: D frC = 5.7, the linguistic variable DfrL "Confidence of face recognition taking into account the eyes" takes the values "Below High" and "High". The mouth is not detected in the image, so D_sC = 0, the linguistic variable DsL "Confidence of mouth detection" takes the value "Low". On the basis of the initial data, a fuzzy inference is performed using the rules of the developed knowledge base Mamdani # 2. At the same time, rule # 36 is fulfilled:

If DfrL= Lower High and DsL = Low, then DmL = High.

By defuzzifying the "High" value of the linguistic variable DmL "Confidence of mask recognition", a defuzzified value of mask recognition confidence D_mС = 4.7 (with a maximum value of 5) (Fig. 6b) was obtained, which means the presence of a mask.

Fig. 3. Detection of faces in the image, face detection confidence D fC = 6 (maximum value is 6)

Fig. 4. Detection of eyes in the image, eye detection confidence D eС = 5 (maximum value is 5)

Fig. 5. Fuzzy inference for recognizing a face with number nf=1; calculation of DfrС by aggregation of membership function μfUa using the center of gravity method (DfrС – face recognition confidence)

(a)

(b)

Fig. 6. Recognition of face with confidence DfrС = 5.7 (a) and recognition of mask with confidence DmС = 4.7 (b)

Let's consider example # 2 of facial mask recognition using the developed system.

First, the face is detected in the image with high confidence D f _C = 6 (Fig. 7), and then eyes are detected with confidence D_eC = 5 (Fig. 8a) and the mouth is detected with confidence D_sC = 1 (Fig. 8b).

Therefore, the initial data of the face recognition systems are as follows: DfC= 6, the linguistic variable DfL "Confidence of face detection" takes the value "High"; DeC = 5, the linguistic variable DeL "Confidence of eye detection" takes the value "High". On the basis of the initial data, fuzzy inference is performed using the rules of the developed Mamdani knowledge base #1. At the same time, rule # 33 is fulfilled:

If DfL= High and DeL = High, then DfrL = High.

By defuzzifying the "High" value of the linguistic variable DfrL "Confidence of face recognition taking into account the eyes", a defuzzified value of face recognition confidence D f _rС = 5.7 (maximum value of 6) was obtained (Fig. 9a).

Therefore, the initial data of the mask recognition system are as follows: D f _rC = 5.7, the linguistic variable DfrL "Confidence of face recognition taking into account the eyes" takes the values "Below High" and "High"; D_sC = 1, the linguistic variable DsL "Confidence of mouth detection" takes the value "Below Average". On the basis of the initial data, a fuzzy inference is performed using the rules of the developed knowledge base of Mamdani # 2. At the same time, rule # 6 is fulfilled:

If DfrL= High and DsL = Below Average, then DmL = Below Average.

By defuzzifying the "Below Average" value of the linguistic variable DmL "Confidence of mask recognition", a defuzzified value of mask recognition confidence D mС = 1.0 (with a maximum value of 5) (Fig. 9b) was obtained, which means the absence of a mask.

Fig. 7. Detection of faces in the image, face detection confidence D fC = 6 (maximum value is 6)

(b)

(a)

Fig. 8. Eye detection with confidence D eС = 5 (a) and mouth detection with confidence D sС = 1 (b)

(a)                                                                        (b)

Fig. 9. Recognition of face with confidence DfrС = 5.7 (a) and recognition of mask with confidence DmС = 1.0 (b)

Consider example # 3 of detecting faces (Fig. 10) and eyes (Fig. 11) of participants in the educational process [37] using the developed system. Detection is performed with a fixed value of the minNeighbors parameter of the Viola-Jones method, so some areas are mistakenly detected as faces (area # 4 in Fig. 10). However, eyes are not detected in such area # 4 (Fig. 11), so only face areas #1-#3 (Fig. 10) can be recognized with high confidence. On the areas of faces #1-#3 (Fig. 10), the mouth is detected only for face #1. Therefore, the mask is not recognized on face #1, and the mask is recognized on faces #2, #3.

Fig. 10. Detection of faces in the image with fixed parameters of the Viola-Jones method

Fig. 11. Detection of eyes in the image with fixed parameters of the Viola-Jones method

The time of face mask recognition in the developed software is an order of magnitude higher than when using the Viola-Jones method with fixed parameters (since the detection is performed for 6 values of the minNeighbors parameter). Therefore, such recognition can be applied to individual images or frames of a video stream. This is acceptable, in particular, when recognizing images of participants in the educational process [38, 39], when recognition should be carried out with a period of several seconds or more.

6.    Conclusion and Future Research Directions

The software implementation of the face mask recognition system using the Viola-Jones method was performed. To improve the accuracy of face and mask recognition, Haar cascades are used for face, eye and mouth detection. Object detection was performed at different values of the minNeighbors parameter for the Viola-Jones method, due to which the confidence of object detection was determined.

A mathematical model of fuzzy inference (1-4) and Mamdani fuzzy knowledge base #1 are developed, which is designed for face recognition based on face detection confidence and eye detection confidence. In the process of fuzzy inference, fuzzy sets are used, which are described by fuzzy trianglular membership functions. Mathematical model of fuzzy inference (5-8) and Mamdani fuzzy knowledge base #2, which is designed for mask recognition based on face recognition confidence and mouth detection confidence, are also developed.

The fuzzy face and face mask image recognition system was software implemented in Python using the Google Colab cloud service. The initial data for the program are digital images read from a video camera or from graphic files. The results of the program are the coordinates of the rectangles that correspond to the faces in the image, as well as the face recognition confidence D f _rС (4) and mask recognition confidence D_mС (8) for each such rectangle. Processing of test images showed generally correct recognition of faces and masks on faces.

Due to the detection of not only faces, but also eyes and mouths, as well as due to the implementation of fuzzy inference, the time of face mask recognition in the developed software is an order of magnitude greater than when using the Viola-Jones method with fixed parameters. The decrease in the speed of image recognition is compensated by higher accuracy, since the complex use of the results of various Haar cascades and fuzzy inference allowed to increase the accuracy of recognition of faces and face masks. The speed of the developed software is sufficient, in particular, for recognizing images of participants in the educational process.

The developed system for recognizing images of faces and masks on faces can be used not only for applied purposes, but also in STEM education when studying the principles of image recognition [40].

Acknowledgment

We would like to express our gratitude to the reviewers for their precise and succinct recommendations that improved the presentation of the results obtained.

Conflict of Interest

The authors declare no conflict of interest.