A Novel Approach for Detecting Number Plate Based on Overlapping Window and Region Clustering for Indian Conditions

Published Online April 2015 in MECS DOI: 10.5815/ijigsp.2015.05.07

In India past few years, traffic control has become a challenging issue in day-to-day life. Due to insensible driving and lack of self-discipline in driving has caused many problems in recent years. Therefore, it is the high time to use automatic surveillance to identify the vehicle owner from the photo capture of vehicle number plate. It is necessary to locate the vehicle number plate from the vehicle image. The focus of this paper is to segment the vehicle number plate from the vehicle image in Automatic Number Plate Recognition (ANPR) System. The image segmentation is a very vital technique for number plate identification. The novel contributions are:

• 1)    An Overlapping Windows based approach for detection of edges and removal of noise.

• 2)    A region clustering based approach for locating NP regions.

The paper is organized as follows. The review of similar techniques is presented in section II. In Section III the proposed method is explained. Experiment Results are discussed in section IV. Problems, restrictions and experiment set up are discussed in section V. The paper is concluded in section VI.

• II.    Review Of Existing Techniques

Image segmentation is a very vast research topic and it can be useful in various research areas. The edge detection method like canny edge detector [1] is very useful for detecting the different edges of the object. A canny edge detector based vehicle plate detection method is also used by [2]. A histogram based method such as [3] is also used to improve the edge detection.

A very good approach for detecting license plate is mentioned in [4]. The authors used Sliding Concentric Window (SCW) to describe irregularities in the image by using standard deviation and mean values. The authors also used sauvola’s method to convert image into binary. Similar approach is presented in [5].

A feature-based approach to locate the NP region is discussed in [6]. The algorithm is well suited for Indian NPs. In this method, a mask of inverted ‘L’ used to isolate NP characters and then after completion of six steps the NP with only characters is segmented. The authors suggest that feature based approach is well suited for the Indian NPs. For converting gray scale image to binary image Otsu’s method is used.

Another Otsu’s method based algorithm is proposed in [7]. The authors used improved Bernsen method for conditions like uneven illumination and particularly for shadow removal. For good accuracy local Otsu, global Otsu, and differential local threshold binary methods are

Table 1 ． Performance comparison of techniques presented in the literature review

Ref	Platform and Processor	NP detection rate	Processing Time (In Sec)
[4]	Visual C++	96.5%	0.111
[6]	Not reported	87%	Not Reported
[9]	Pentium IV 3.0 GHz Processor	98.77% For 320 X 240 99.807% For 640 X 320 99.857% For 640 X 480	Not Reported
[10]	Pentium IV 3.0 GHz Processor	95.9 (Multinationals) 96.9 (Multi Style)	Not Reported
[8]	Pentium-IV 2.26-GHz, 1 GB RAM, MATLAB	97.3%
[7]	CPU 1.8 GHz VC++	97.16%	0.158
[11]	MATLAB	Not Reported	Not Reported
[12]	MATLAB	Not Reported	Not Reported
[14]	C Programming, 2.3 GHz processor	Not Reported	0.025
[2]	Not Reported	98.45%	Not Reported
[15]	Not Reported	Not Reported	Not Reported
[16]	Pentium 2.8 GHz Processor	96.4%	0.204
[5]	Pentium IV 2.4GHz, 1GB RAM, MATLAB	89%	Not Reported
[20]	Pentium IV 3.0 GHz processor	81.2%	0.4
[22]	Intel Core-Duo 3.0GHz CPU	97%	0.5
[24]	Pentium - IV 1.6 GHz	97.9%	0.4
[25]	Not Reported	93.78%	Not Reported
[26]	Not Reported	92.31%	Not Reported
[27]	Pentium 2.66 GHz processor, MB RAM, Visual Studio 2005	98.3%	.015
[31]	Pentium IV 3.0GHz processor, MATLAB	~75.17%	1.1

used. In this algorithm, shadow removal is possible, which was not possible in the traditional Bernsen method.

It is also possible to do the segmentation based on feature of image like image shape, image color, texture etc. By considering these features, a feature salient method proposed in [8]. To detect vertical and horizontal lines Hough transform (HT) is used. The number plate is further processed by converting red, green, blue (RGB) to hue-intensity-saturation (HIS) to segment it. This algorithm is executed on Pentium-IV 2.26-GHz PC with 1 GB RAM using MATLAB.

To achieve high accuracy rate of plate detection, [9] proposed cascading framework based approach. In this approach, a cascade framework is developed successively based on the term called Rejecter. The main constraint of Rejecter is to have high True Positive Rate (TPR) and low False Positive Rate (FPR). The framework is known as cascade because each rejecter accepts output from the previous one. By using these rejecters in the shortest possible time, the average computational speed of the system would be faster than that obtained by adopting the more complex processes for all input candidates.

A novel configurable method is proposed in [10], to detect the multinational and multiple license plate. A user can configure the algorithm by changing parameter values such as plate rotation angle, character line number, recognition models, character formats. The plate rotation angel should be set to correct skewed images. The authors mention that the license plate characters span across row or column so character line number parameter is considered for this purpose. The algorithm works for maximum 3 rows. Most of the NP contains either alphabets or numbers or symbols or combinations of these characters. To identify this recognition model be used and lastly to label and classify these characters in each category characters format parameter is used. For example, a symbol is labeled as S, alphabet is represented as A and digit is represented as D. So if vehicle number is GJ 23 AS 890, it can be represented as AADAADDDD, by using these parameters. The algorithm was carried out on Pentium IV 3.0 GHz processor.

To find exact rectangle with vehicle number a system is proposed by [11]. In this method, horizontal and vertical difference is calculated to locate exact rectangle with vehicle number. The further technical details regarding the algorithm and experiment set up is not mentioned in this paper.

Morphological operation is also useful in image segmentation. In [12], a novel approach based on texture characteristics and wavelets is proposed. The authors applied morphological operations for better performance in complicated background. To detect vertical edges Sobel mask is used. Another system based on Sobel operator is mentioned in [13].

In [14], the authors proposed a novel approach for efficient localization of license plates from the CCTV footage. In this method, revised version of an existing technique for tracking and recognition is proposed. The system is intelligent enough to automatically adjust varying camera distance and diverse lighting conditions. The NP is detected based on the preprocessing steps such as background learning and Median Filtering-Morphological operations. Then after the NP detection procedure is started. First step is to find contours and connected component analysis (CCA) for detection of Region Of Interest (ROI). The rectangular region is selected based on the size and aspect ratio in the step 2. In the third step, initial learning is used for adaptive camera distance/height. Finally, the NP is detected based on Histogram Oriented Gradients (HOG) feature extraction and nearest mean classifier.

Sometimes it is necessary to remove “salt” and “peeper” noise to avoid unwanted parts in the NP. To overcome this problem, the authors in [15] presented four stepbased approach for NP detection. Initially median filtering is applied to remove non-candidate regions of NP. Then the image is converted to binary Otsu’s method. After that Component labeling and region growing is used to find candidate regions. Finally, the NP is segmented based on the bounding box method. The further detail regarding this method is not mentioned in this paper.

As per [16], NP detection is a crucial step in an LPR system. The authors used global edge features and local Haar-like features for real-time traffic video. A scanning windows is moved around the vehicle image. The scanning window is categorized as license plate region and non license plate region based on the pre-defined classifier. In the training phase, six cascade classifier layers are constructed for future processing. In the testing phase, local Haar-like features and global features are extracted. These features are used to find the number of rectangles covering adjacent image regions. Global features include edge density and edge density variable are calculated by using fixed size of sample image i.e. 48 X 16, which is scaled in the training phase.

In [17], weighted statistical method is applied. A 24-bit color image is converted into gray scale image and then the weighted statistical method is applied on it. In this method, a 2D image matrix of N rows and M columns is prepared. Then after the modified image matrix is formed after adding weights. Further technical details are not available.

To detect license plate in different varying illumination conditions, authors proposed different approach in[18]. In this method, binarization is applied as a pre-processing step for NP segmentation. The image is divided in small window regions and then dynamic thresholding method is applied to each region. The authors claim that this method is very robust for a local change in brightness of an image. Finally, labeling and segmentation is applied to the binarized image to detect candidate regions.

In [19], an approach for finding text in the images is discussed. The authors assume that NP has light background and dark foreground for NP characters. To extract NP character spatial variance method is applied to identify and find text regions and non-text regions. The high variance indicates the region is text region and a low variance indicates that the region is non-text region.

Some of the NP recognition algorithms are specific to certain vehicle like car, bus, two wheelers etc. In [20], a system for multinational car license plate recognition is proposed. The system is composed of mainly three steps Pre-processing, segmentation and verification. First, the pre-processing is used to apply global Thresholding for mapping color intensity into gray scale. Robert’s edge operator is used to detect vertical boundary of NP. Skew position is eliminated by using Randon Transform (RT)

function in conjunction with Dirac’s delta function. Horizontal boundaries are detected by using a series of morphological erosions with horizontally oriented structured elements are applied. In the second step, the authors compared binarization threshold with the plate intensity median. The authors admit that the probability of detecting a number plate is higher when the intensity median of the plate zone is greater than the threshold of the image. After passing the entire test sequence, the number plate is approved successfully. If any of the tests fails, then the current plate region is rejected and the search for another NP region is started. After reaching the maximum number of iterations and no NP found then algorithms stops and appropriate error message is generated.

In [21], a dynamic programming based NP segmentation algorithm is discussed. The authors discussed different approach to detect NP.Similar approach is reported in [22]. Multiple threshold-based method is used to extract candidate regions. The segmented blobs are used to provide geometric constraints for numeric characters of a number plate. So it is not required use any image features likes edges, colors, or lines. The authors used Adaboost [23] algorithm with OpenCV for training.

In [24], a fuzzy discipline based approach is discussed. In NP locating module, having colors, white, black, red and green are considered. The edge detector algorithm is sensitive to only black-white, red-white and green-white edges. Then transformation from RGB (Red, Green, and Blue) to HIS (Hue, Intensity, and Saturation) is performed. Then after that different edge maps are formed and by using a two stage fuzzy aggregator, these maps are integrated. Finally, by using color edge detector, fuzzy maps and fuzzy aggregation candidate region is detected. It took around 0.4s to locate all the possible license plate regions.

A probabilistic based approach is discussed in [25]. The authors propose a novel future fusion. First, the color image is transformed into gray scale by using multiple thresholds and Otsu’s threshold. The authors used different approaches such as deterministic approach: pixel voting, probabilistic approach: global binarization, probabilistic approach: local binarization and combination among these approaches to locate the license plate.

In [26], to detect NP candidates, the edges of each detecting line are examined from the bottom to the top of an input image as the authors mention that generally license plate is located at the bottom of the vehicle. They suggest that the vertical edges offer more information about the vehicle compared to the horizontal edges so they used vertical edges to find candidate region. The threshold calculation and image binarization methods are applied using the equations mentioned in this paper. Finally, candidate regions are identified by using verification process. The algorithm is sensitive to noise in the head-light area or license plate area.

A cognitive and video-based approach is proposed in [27]. The authors classify license plate detection approach in two groups: appearance based and gradient methods. The authors implemented gradient-based approach in which, each frame is processed to localize areas with a high vertical gradient density. A vertical Sobel mark filter is applied after contour enhancement. After final labeling, pixels are identified as text and nontext area.

Apart from above systems other systems such as inductive learning [28], Region based [29], Fuzzy based algorithm [30], iterative threshold based method [33] and edge-based color aided method [31] are also useful for the NP detection. As these methods are similar to the methods discussed in this section, further detail of these methods is not presented here to avoid duplication. Our previous work [32] produces good result of NP segmentation but it works with fixed threshold and under restricted conditions. The performance comparison of different non-commercial systems is presented in Table 1.

• III.    Proposed Work

The work is divided in two parts, which is shown in fig.1. The First vehicle image is given as input to the overlapping window based method which converts the image into binary image based on the algorithm presented in table 2. Finally the NP is segmented by using region based row and column clustering methods. The further details about these methods are presented in fig.2 and table 2. Indian number plates are generally classified in three groups:

Fig. 1. Proposed system architecture

• 1)    NP with white background and Black Fonts for the private vehicles.

• 2)    NP with yellow background and black font for commercial vehicles.

• 3)    NP for government vehicles.

Our algorithm works well for all these kind of Indian NPs. The details about individual steps of fig.1 are discussed in the sub sections A and B of section III.

• A.    An Overlapping Windows Based Method

Generally, license plate image can have different size for different vehicle as discussed in the previous section. As there are many components in the vehicle image, it is quite difficult to isolate NP regions from it. It is also subject to different illumination and lightening conditions during the day and night. By considering these factors, a novel overlapping windows based approach is proposed in this paper. The method is discussed as follows:

• 1)    Two scanning windows W1 and W2 are sliding and overlapping each other from first row to nth row and based on four neighbour connectivity (N₄)

• 2)    The standard deviation is calculated based on the following equations (1), (2) and (3).

о1 = о(Р1,Р2,Р3,Р4)

с2 = с( Р,Р1,Р2,Р3,Р4)

Here P = Current pixel and P1, P2, P3 and P4 are four connected neighbours of pixel P. The new image Img2 is obtained as per the following formula:

• -ж». IV) {  0 т

1 о teerwise

Here T is the threshold that can be selected based on trial and error. In this method, the value of T is 2 for a sample set of images as discussed in section IV. The windows slide until the entire image is scanned. Also, there is no evidence to choose the best possible way to choose the optimized value of threshold T. The algorithm is presented in the Table 2.

• B.    Region Clustering

In the column clustering method, a cluster is formed from first column (n) of image to next k columns of image. Again next row cluster is formed from (n+ k)th column to next k column. In each cluster, number of pixels having percentage of white color and percentage of pixels having black color is calculated separately. This process is repeated until the entire image is scanned. Based on out observation and experiments, in this algorithm we have taken the value of k as 10. During this process, the clusters satisfying the criteria are considered as candidate region and stored in the array candidate region. Meanwhile, numbers of clusters having candidates regions are also calculated. If this count is less than or equal to 2 then it is considered that the image is not having number plate or algorithm fails to detect the numbers, otherwise number plate is displayed to the user. Our experiments reveal that in Indian number plate the percentage of white pixel is at least 6% and the percentage of black color is at least 6%. So 6% is considered as value of parameter N1 and N2 for identifying candidate region. The entire process is depicted in Fig 2.

Fig. 2. Flow chart of the system

Table 2 ． NP Segmentation Algorithm/ Pseudo code

Step 1: Input Image Img1

Step 2: For each pixel in Img1

Perform overlapping window method for each pixels

{ a 1= standard deviation of pixels P,P1, P2, P3 and P4

^ 2= stadard deviation of pixels P1, P2, P3 and P4

P= pixel (x,y), P1=pixel (x,y-1), P2= pixel (x,y+1), P3= pixel (x-1, y) , P4 = pixel (x+1, y) }

Set the pixel value of current pixel based on the following condition and obtain image Img2

Img2(x,y) = {  . .Black if ^{( a} 1 . ^²⁾ > ^        //In this algorithm T is considered as 2

^a         (  W ^ ite ot h erwise

Step 3: Perform region clustering based on row cluster=0

N=0 {N=number of clusters} while(cluster

for i=cluster to cluster +10

for j=0 to width of Img2

color_black=number of black pixels in the cluster color_white=number of white pixels in the cluster end for per_black=percentage of black pixels in the cluster per_white=percentage of white pixels in the cluster end for if (per_black >N1 and per_white > N2)

store the cluster in the candidate_region(m, n) array

N=N +1

cluster = cluster +10;

end While

{ N1 and N2 can be calculated based on trial and error. In our experiment N1 =6% and N2=6%}

Step 4: Perform region clustering based on column {Same as step 3}

Cluster=0

while(cluster

for j=cluster to cluster +10

for i=0 to height of Img2

color_black=number of black pixels in the cluster color_white=number of white pixels in the cluster end for per_black=percentage of black pixels in the cluster per_white=percentage of white pixels in the cluster end for if (per_black >N1 and per_white > N2)

store the cluster in the candidate_region(m, n) array

N=N +1

cluster = cluster +10;

end While

{ N1 and N2 can be calculated based on trial and error. In our experiment N1 =1% and N2=1%}

Step 5: if N <=2 then Display “Number plate not found”

else Display contents of candidate_region array

	Table 3．NP Segmentation experiment set up details
Sample Set	Number of          Image size         Accuracy       Threshold T images

Vehicle images captured during 9:00 am to 4:30 pm          50

800 X 600

100%

Vehicle images captured after 4:30 pm

800 X 600

97.50%

Average Accuracy =98.88%

• IV.    Experimental Results

As it is presented in Table 3, our system works in 24 hours manner with the accuracy of 98.88%, which is not possible in any of the existing systems. The average processing time is ~2s for processing 800 X 600. Our experiments reveal that if image size is reduced, then processing time reduced to less than 1s. In table 3, details about sample set is mentioned. One of the images from this sample is shown in the Fig 3. The image shown in Fig 3(a) is given as input to the algorithm and NP of vehicle is segmented as shown in Fig 3(b).

Fig. 3. (a) Original Image (b) Image with Segmented NP

• V.    Problems And Restrictions

As vehicle NP is a complex entity, in certain conditions the algorithm fails to detect number plate. It is observed that if the image is captured at the distance of more than 5m then it is difficult to segment the NP. Some of the NPs were not as per the standard defined by Indian Road and Transport Office (RTO) and because of that NPs were not detected correctly. The algorithm works well in the 24 hours manner in different lighting conditions. So mainly in two restrictions the system does not work: distance of more than 5m and non-uniform NP. The system is also dependent on a high resolution camera so if the image is captured with high resolution camera of more than 5 mega pixels then the algorithm might produce better accuracy.

• VI.    Conclusion

A novel NP segmentation technique has been discussed in this paper. The offline image of the vehicle is processed by our algorithm. The system can be further exploited by attaching the camera with it and taking the real time image of the vehicle. Also the average processing time of 2s can be improved by code optimization as present code is not optimized.

Acknowledgment

The authors would like to thank Mr. Dharmendra Patel for providing his valuable inputs to improve this paper. The authors also thank Charotar University of Science and Technology (CHARUSAT) for providing necessary resources to accomplish this research.