Video-based Flame Detection using LBP-based Descriptor: Influences of Classifiers Variety on Detection Efficiency

Published Online February 2017 in MECS

Fire detection from the video stream is considered to be difficult and not fully resolved the problem in computer vision areas research. The majority of fire detection methods in images are based on such features as color, texture or some another temporal characteristic. However, the flame as object of classification has a difficult structure with unstable parameters of movement, wide range of colors (depending on temperature and burning substances), variable contours and other dynamic effects. All these factors greatly increase the number of incorrect classification.

This article describes the process of fire detecting with usage of local binary patterns (LBP) texture descriptor and its modifications, aimed at improving the flame detector efficiency in video stream dissemination. Further, for received features, support vector machine classifier (SVM) applied with different variations of kernels and k-nearest neighbors (k-NN) algorithm. As a result of using this approach, the time required to process the image which is analyzed, average parameters of detection accuracy and recall were calculated.

• II.    Research Background

To solve the problem of fire detecting was used a variety of approaches. Conditionally, they can be divided into the following groups: color and texture segmentation, motion detection in images, additional flame attributes (flickering, sharp corners, etc.).

There are a lot of research works devoted to color segmentation, which can be divided by type of model used: RGB [1], HSV [2], YCbCr [3], CIE L*a*b [4]. In this paper, we used color model YCbCr, which shows the best quality detection rate - 99% [3]. However, despite the good results, presented by researchers, this method characterizes by the high rate of false positives alarms, that arise from changes in lighting and required parameters setting for each facility separately. Obtained results, in our view, were largely represented by small representative dataset on which the test was performed, which, in particular, taken into account during our experimental studies.

To lower the false alarm rate, in addition to different color models, some researchers also incorporated motion and fire flickering into their work. Rinsurongkawong et al. [4] have proposed to use Lucas Kanade optical flow algorithm for fire moving detect. Toreyin et al. [5] used hybrid background estimation method to detect moving objects. For checks flicker in flames he using 1-D temporal wavelet transform and color variation in fire colored moving regions using 2 –D spatial wavelet transform. Although powerful, these techniques are computationally intensive.

Problems of using various descriptors to identify fire in images/video stream are lighted in works [6] (SIFT descriptor), [7] (HOG descriptor), [8] (SURF descriptor).

Also, there are some studies of using LBP descriptors for the task of flames identify [9, 10]. However, these works are not sufficiently disclosed potential choice classifier, which, according to experimental results can positively effect on developed detection system. In particular, we proposed a number of modifications LBP algorithm, which allow improving the quality of flame detection.

• III.    Theoretical Basis of Lbp Descriptor

As noted above, there are various feature extraction techniques: HOG, Haar Wavelets, LBP etc. For developed system, we have decided to use LBP descriptor. The reason for this choice was caused by number of advantages for fire detection: speed calculation, invariability to brightness change which preserve order ( a > b → f ( a )> f ( b )) , resistance to noise and textures variations.

Classical LBP operator, which was proposed in work [11], forms labels for the image pixels by thresholding the 3 x 3 neighborhood of each pixel with the center value and considering the result as a binary number. Using a circular neighborhood and bilinearly interpolating values at non-integer pixel coordinates allow any radius and number of pixels in the neighborhood. Pixels that have a value greater than the central pixel (or equal), take the value "1", those that are less central, taking the value "0". The results of the comparisons are stored in an 8-bit binary code, which describes the neighborhood pixels. Neighborhood examples used to define a texture and calculate a local binary pattern shown in Fig.1.

Fig.1. Example of computing LBP

Mathematically, this process can be defined as follows:

LBPp , R =∑p=0 ^ (  - g c )2^p          (1)

where g _c is a center pixel value and ( p =0,…, P -1) – value of pixels P, which are along a circle of radius R (R > 0), s(x) - sign function, which can be defined as follows:

1 if x ≥0

{ ₀ if x <0

A number of pixels (P) along a circle with radius R can be chosen arbitrarily. To calculate values in these points for different radius R and points number P, bilinear interpolation was used. Fig. 2 shows pixels set for different neighborhood P and R.

local binary pattern with different value P and R

It has also been experimentally established, that for improving descriptor performance and efficiency, following set of pixels along a circle: P = 8, R = 1.0 is the most optimal choice for our tasks.

In work [12] was proposed to analyze the images using not all binary patterns, but only those who may contain at most two bitwise transitions from "1" to "0" or vice versa. The number of bitwise transitions, when the binary string is circular, gives a uniformity measure U of the pattern as follows:

,

={

P-1

∑ S (  - 9c ), ifU ( LBPp , „)≤2

p=0

p +1, ot ℎ erwise

U ( LBPp , R )=| S ⁽ Qp-i ^- 9c )- s ( 9o ^- 9c ) |+ p-i

+∑|S( - 9 c )- S ( 9p-i ^- 9c ) |

P=1

These templates are called uniform pattern and characterized by two advantages. First, we save a memory because the number of uniform LBP patterns is given by p ( p -1)+3 , leading to a much shorter histogram representation. Secondly, they define only important local textures, such as the end of lines, edges, corners, spots. Further LBP operator with uniform patterns we will defined as LBPp^

In the last step, after the LBP labeled image has been obtained, the LBP histogram ( H[ ) is computed, which can be defined as:

H[ =∑ в (    _,( x , У )= i ), i =0,․․,n-1,

,                  1,

^{B (} к )={ _0, ot ℎ erwise

In many cases, the LBP histogram may have excessive information, and much of it is often irrelevant to the problem to be solved. Which is why it was decided to use the method proposed in [13]. So, the histogram bins are sorted by descending order. After that, there is only part of the histogram that includes a percentage and patterns which are then submitted for classification.

Classifiers Variety on Detection Efficiency

• IV. Proposed Method

Developed flame/non-flame detector must meet the requirements of implicitly high-level object detection and good performance even on slow machines. In this paper, to reduce the time required for verification area, which may cause a fire, we proposed the additional use of color segmentation and some spatial-temporal object characteristics.

The process of detecting area with fire in images based on the five basic principles described below:

R₄ ( x ,у)={^1, ifCb ( ,^y)- Cr ^{( 1 ,y})≥ T ^ℎ (10) _0, ot ℎ erwise

The mean value for Y, Cb and Cr are calculated as equation:

I   M N

Ymean ( X , У )=  ¹ ∑∑ Y ( x , У )

Cbmean ( x , У )

Crmean ( x , У )

M ∙ N

M ∙ N

M N

∑∑ Cb ( x , У )

X=1y=l

M N

∑∑ Cr ( X , У )

X=1y=l

"Y" [ Cb ] .Cr.

0․2568   0․5041   0․0979

-0․1482  - 0․2910 - 0․4392 ∗

0․4392 ∗

- 0․3678

[ SG]+[11268]

Lb 128

- 0․0714

where Y - luminance component; Cb - the difference between blue component and brightness Y; Cr - the difference between red component and brightness Y.

Segmentation area with flames was based on rules:

1, if Y ( x , У )≥ Cb ( x , У )∩

Ri ( x , У )={   Cr ( x , У )≥ Cb ( x , У )        (7)

0, ot ℎ erwise

1, if ( Cb ( x , У )≤120)∩

R₂ ( X , У )={   ( Cr ( X , У ) ≥150)         (8)

0, ot ℎ erwise

R₃ ( x , У )={

1, if Y ( x , У )≥ Ymean ∩ Cb ( x , У )≤ Cbmean ∩ Cr ( x , У )≥ Crmean 0, ot ℎ erwise

B. Moving objects detection

Getting frame difference is due to calculation difference D ( x , У )=| I_t ( x , У )- B_t ( x , У )| current video frame from the updated binary mask and calculating two thresholds cyclical buffer expressions:

T ^ℎ1⁽i^, j )=(|^P^ ^{- P}^ |); (14)

Tℎ2(i,j)= |Pi - Pk | where I,j - number of row and column number which is at the intersection pixel frame, к - the number of elements cyclical buffer, Pi - element circular buffer that is on position i.

In the resulting binary mask M ( x , У ), a pixel difference D ( x , У ) is classified as belonging a moving object, if he presents on mask Th 1 and located into three pixels’ mask Th 2 , otherwise the pixel is considered as background.

For pixels segmentation that can be attributed to micromotions generated by background, moving objects in the resulting binary mask M ( x , У ) allocated by grouping them all connected pixels and calculates their

modules gradient by the expression:

• G. =√‖    ⁽ ; ^{, 5} ) _‖ +‖ ^ ⁽ ; ^{, 5} ) _‖              (15)

where      ‖ - ⁽ J ^, - ) _{‖ =} It- ∆ t ( i -1, j )- It ( i +1, j ),

‖ ”. ( , - )‖  = It-∆t(i,j-1)-It (i,j+1) - gradients in horizontal and vertical directions, ‖‖ - vector norm.

• C.    LBP descriptor using

Taking into account the combustion process, we have chosen some modifications of LBP descriptor which will improve the efficiency of our flame detector on the one hand and increase computation speed on the other.

• D.    NRLBP

Classic LBP descriptor characterized by two disadvantages that are described in [16, 23]. First of all, it is sensitive to changes between background and foreground. As can be observed in fig. 3, the binary code of LBP descriptor in two images with flames are different from each other. However, they describe the same spatial characteristics of fire. Another disadvantage of classic LBP descriptor it is high demands on the bins calculation and storage. Thus, the original implementation      , required 256 histogram bins, in     ,  – only 59 bins, which is also quite costly. To develop flame detection system which can work in real time it is necessary to minimize requirements for their calculation and storage.

Taking into consideration this disadvantages we propose to use Non-Redundant Local Binary Pattern (NRLBP) [17] to obtain visual information about flame presence. From Fig. 3 can be seen that LBP codes of two images complement each other (the sum will amount 2p-1 ). Mathematically NRLBP can be defined as follows:

NRLBPp , R ( X_C , Ус)= min {LBP , R ( X_C , Ус),2^P-1-

LBP , R ( X_c , ^c )}               (16)

Fig.3. LBP codes for the flame with same structure, but different inversion in background and foreground

Based on the above, binary codes of upper and lower images in fig.3 represent the same texture. This, in turn, will reduce feature vectors dimension at half.

• E.    VLBP

Despite the fact that discriminatory ability of NRLBP characterized by improved capacity and much lower bins storage requirements, in comparison with classical LBP, it includes only local visual information. To build a quality vision-based fire monitoring system, primarily important information about its temporal characteristics. For this, we decide to use Volume Local Binary Pattern (VLBP), which is calculated on the difference between frames.

In this method, arbitrary pixel frame of video is described as _, =( X ,у, t ) and its coordinates determined in accordance ( X , У ) and times moment t . Target (central) pixels has described by equation (17):

• 9 i,    =( x , У , t + i ∙∆ t ),    i = -1,0,1,        (17)

where ∆ t – time between sequence.

Neighboring pixels are selected according to equation (18):

• 9 i,    p =( x + x_p , У + У_Р , t + i ∙∆ t ),    P = 0,…, P -

• 1,(6)                   (18)

where P - the number of neighboring pixels.

Dynamic LBP operator, which will depend on the time interval between successive frames ∆ t , number of neighboring pixels P and the distance between the target and adjacent pixels R.

3P + 1

VLBP = ∑ ^2⁴

q=0

where ^rQ – determines function (2), arguments of which is the difference (   , -    , ), whose number is ((3 P +2)

• [18] .

As a result, after operators using we obtained a histogram, which, in turn, is fed to the input classifier for further processing.

• F.    Classification

Selection of qualitative features descriptor undoubtedly plays a key role in developing vision-based fire detection system. However, after receiving features that describe the desired image, the equally important step is the rejection of so-called "extra" features around the set. Therefore, classifier selection is important task during image classification and requires more detailed study.

• G.    Support Vector Machine

Classification using support vector method is widely used in image recognition thanks to its empirically good performance and moderate calculation complexity. General principles of SVM described in [19, 20]. SVM becomes famous when using pixel maps as input; it gives accuracy comparable to sophisticated neural networks with elaborated features in a handwriting recognition task. It is also being used for many applications, such as hand writing analysis, face analysis and so forth, especially for pattern classification and regression based applications [19].

It should also be noted that the classification function (F) becomes:

F (%) = s ign((w,ф(x)') + b )          (20)

and expression k(x,x) = ( ф(х), ф(X')‘) called core classifier. Classification accuracy can be improved by proper kernels selection.

The most common kernels are [24]:

• •   polynomial (homogeneous): k(x, x) = (x • X)^d;

• •   polynomial (inhomogeneous):      k(x,x) =

(x • X + 1)d;

• •    radial basis function (RBF):   k(x,x) =

exp(-ylx -x||)², для у > 0;

• •    Gaussian radial basis function (RBF-G): k(x,x) = exp(-^<)

• •    sigmoid function: к(x,x) = tanh(kx • x + c), almost for all k>0 і c<0

To analyze the performance of the flame classifier we analyzed this kernels in work, results of which are described in Section V.

• H.    k-nearest neighbors algorithm

k-nearest neighbors algorithm is a non-parametric method based on a similarity measure (e.g., distance functions). If K = 1, then the case is simply assigned to the class of its nearest neighbor.

A main advantage of the k-NN algorithm is that it performs well with multi-modal classes because the basis of its decision is based on a small neighborhood of similar objects. Therefore, even if the target class is multi-modal, the algorithm can still lead to good accuracy. However, a major disadvantage of the k-NN algorithm is that it uses all the features equally in computing for similarities [21, 25].

It should be noted about some disadvantages of k-NN: model can not be interpreted, it is computationally expensive to find the k nearest neighbors when the dataset is very large, performance depends on the number of dimensions that we have.

• V.    Results

• A.    Dataset

For test classifications quality we formed the own dataset of video, which is divided into two categories: the flame and without flame. A number of copies used for the category "flame" - 22, for categories in which there are no flames - 34. In particular, the last category includes objects which may look like a flame (bright clothing, lights, garlands, etc.).

To train the classifier we used image dataset that was formed during the previous study [22]. It includes 1876 images with fire and the 4634 images without fire.

• B.    Evaluation method

Results of fire classification evaluated based on two well-known techniques - precision and recall. High precision relates to a low false positive rate, and(2h0i)gh recall relates to a low false negative rate. High scores for both show that the classifier is returning accurate results (high precision), as well as returning a majority of all positive results (high recall).

Precision =  TP(21)

TP+FP

Recall =(22)

TP+FN where TP - the number of true positives (correct system inference about area where were flames); FP - falsepositive operation (system result agrees with the presence of fire whereas ground truth agrees with the absence of fire); FN - false-negative operation (system result agree of absence of fire whereas ground truth agree of the presence of fire).

• C.    Experimental results

The results figures of above methods are presented in Table I. As we can see, using SVM classifier with a kernel based on radial basis function showed the best precision result. SVM classifier with Gaussian radial basis function showed the best recall result in fire detection task. SVM with polynomial kernel showed the best time information processing. k-nearest neighbors algorithm shows the worse performance of time and classification efficiency.

Table 1. Results of using different classification methods

Method	Precision	Recall	Time
SVM- polynomial kernel	81,3 %	93,5 %	0,009
SVM-RBF kernel	81,9 %	94,2 %	0,02
SVM-RBFG kernel	81,6 %	94,3 %	0,03
SVM- sigmoid kernel	81,3 %	93,7 %	0,03
k-NN	80,7 %	92, 6 %	0.01

• VI.    Conclusions

For developed video-based flame detector, the most important parameter is the recall of classification. Using SVM classifier with Gaussian RBF kernel shows the best recall and relatively high performance of accuracy and performance time. So, it’s the most better choice for solving the problem of flame detection in video.

More future research should be done to verify and improve the results of this paper. In future work, we will endeavor to incorporate fire reflection detection algorithms into our current algorithm.

Some results of false positive and true positive/negative detection shown in Fig. 4, 5, 6.