Video Analytic Algorithm for License Plate Recognition System

The significant advancement in technology and equipment motivates researchers now days to go for system automation in every area of life. Traffic management has always been a serious issue for most of the countries especially in developing countries it always remains a challenging task. Developed countries like USA, England, Germany, Australia etc. have automated their traffic monitoring systems to manage the traffic effectively, but in developing countries like Pakistan, India, Bangladesh, SriLankaand others still have problems regarding traffic management and very few areas implemented some automated systems for traffic monitoring. Security is also a key aspect and reason to use automated systems to verify vehicles and traffic surveillance. License Plate Recognition is an image processing technology which is used to identify the vehicles by their number plates. Some License plate recognition systems are generic which covers every form of license plate while some systems are specific like the system of Wangunyu P.W, Opiyo E.T.O and Rodrigues [1] which discusses a system for license plate recognition for Kenyan license plate. Felix Arya, IpingSuprianaSuwardi in their work[2] covers license plate for Indonesian plates. There also exist many systems which presents generic algorithms to discover all kind of plates [3,4].VISEC Surveillance Software[5] organization has developed world’s first IP based License plate recognition system. The VISEC LPR has been constructed using Advanced Mathematical Modeling, employs complex Algorithms, mapped and tested using supercomputer technology, developed using logic derived from Chaos Theory and Fibonacci Sequencing laws. Visec is capable of reading license plates at close to 99.9% accuracy at speeds of over 100 kilometers per hour. PIPS Technology has developed one of the most advanced License Plate Recognition system [6].

License plate recognition system has been a key research area in computer vision and image processing field now a days. License plate recognition systems have a very vital role in defense systems and can be employed very effectively there. The Proposed system also have several other applications including the toll station system, vehicle identification system in cases of theft and robbery, Parking systems as well as in systems based on access control.

There has been a lot of research work done previously in this field. Many country specific license plate recognition systemsexist in the market but no such professional developed system exists for Pakistani standard car plates. The proposed system, in this research work presents an efficient and accurate method for the identification of license plate and recognition of numbers in license plate for both the Pakistani standard and non-standard license plate. Organization of the proposed system is as follows: Section I includes the introduction whereas the second section systems which have been developed previously are discussed and reviewed. The architecture of the proposed system has been discussed in section III. Proposed methodology of the system has been included in section IV of the paper. The setup of the experiment and the results obtained are discussed in section V. Section VI includes the conclusion of the proposed system.

• II.    LITERATURE REVIEW

Two complementary methods have been used by HalinaKwaśnicka and BartoszWawrzyniak [7] for the localization of license plate. Connected components analysis method is used in the first approach whereas the signature of the license plate has been used in the other method. This method is identical to labeling of connected component but it operates on a diverse data set repository resulting in unique license plate candidates. A set of license plate candidate is produced by the simultaneous working of these processes which is then used by the character segmentation module for the purpose of character identification.

Two elements are involved in character segmentation module. “Peak to valley” is the first element, which is calculated by the candidate image’s vertical projection. Probable segmentation points are obtained from the values of these functions which show deep “Valleys” wherevertical projection contains high peaks. Set of regions in the form of spots are identified in the candidate image during labeling, this is the second element of character segmentation. These two elements are collectively taken into consideration while deciding which part belongs to the number plate. A three layer neural network based recognition module is then passed these segmented characters from segmentation module. A chain of character is produced as a result of character recognition; syntax analysis technique is then used to compare this chain of character with the pattern of correct vehicle numbers. An algorithm based on the combination of modifiedHough    transform techniques and morphological processes have been proposed by Velappa GANAPATHY and Wen Lik Dennis LUI [8] to enable the system act robustly. The morphological techniques applied on the image serve to remove all the unrelated portion of the image and make it sure that only the characters of the number plate remain well preserved. Three major morphological processes are applied to the image. The width, height and number of holes present in each image object are calculated. Those images exceeding the upper and lower boundaries are removed from the image and remaining is retained for the further processing. After morphological process, modified Hough transform technique is applied for further processing. Auto skew correction and the candidate evolution process are used for the completion of license plate localization module. Prior to recognition the characters are normalized into the size of 25 x 15 pixels. Bounding box calculation is used for the character segmentation at this stage. Another approach proposed by velappa and denis includes feed-forward back propagation artificial neural network.

Another approach for accurate license plate localization based on region based algorithms has been proposed by WenjingJia, Huaifeng Zhang, Xiangjian [9] in which color vehicle images are filtered and segmented using mean shift. The decision of detecting license plate region is based on the extraction of three attributes. It includes rectangularity, edge density and aspect ratio. Candidate regions are classified on the basis ofMahalanobis classifier with the help of which license plate regions are accurately identified.

M J. Ahmed, M Sarfaz, A. Zidouri, and K G. AI-Khatib [10] in their work presents three distinct approaches for image acquisition, one by using analogue camera, second by using a digital camera and the third one is to use a video camera. The step of candidate region extraction is divided into a chain of interrelated steps including the size and shape filtering, vertical edge detection, black/white ratio and plate extraction,vertical edge matching, sobel edge detection operator is used for vertical edge detection, seed filling algorithm which plays a major role in filtering the undesirable objects is used in size and shape filtering stages. The pair of regions that constitute the license plate are calculated using the vertical edge matching. The ratio between the black and white is calculated, and if the ratio is between the previously defined thresholds, it is perceived as a license plate. Template matching technique is used for character recognition.

A system based on Hausdorff distance algorithm has been proposed by Tang Shuang-tong, LI Wen-ju [11]. The technique used in this system is two step template matching. First step of template matching is matching depending upon background and foreground match. The next step is the whole mismatch measure. Template matching based on the edge hausdorff distance is the second technique.

• III.    PROPOSED METHODOLOGY

• A.    System architecture

The proposed system flow chart is depicted in Fig. 1.

Figure1. System Flow Chart

A static digital camera has been used in the proposed system for image acquisition. Image enhancement has been applied after acquiring image in the preprocessing phase. Hough transform technique is employed for License plate localization. Segmentation of characters is done in the segmentation phase and character recognition is done using OCR techniqueyielding a recognized plate as shown in Fig.1 in system flow chart.

Image of the vehicle is captured using acamera and different type of smoothing and sharpening spatial filters are applied on the image to achieve better quality enhanced image. Average filter has been used to achieve smoothing and noise removal in the image andGaussian high pass filter has been applied to achieve sharpness in the image. Further processing is done using the Hough Transform which localizes the plate, from the result obtained by image enhancement, which is a very famous technique used for separating features of a particular shape from an image. In segmentation phase, after license plate localization, the characters on the number plate are segmented followed by the key stage of character recognition which is done using template matching technique. Many other methods, such as neural networks are also used for template matching but the most famous and widely used is the template matching that has been employed in our system for finding those parts of the image which match with the template.

B.    Image Acquisition

A static camera has been used for image capturing which uses a resolution of 1280 x 1024 pixels for image capturing. Throughput of the system is influenced by many conditions e.g. position of camera, weather and muddy number plate etc. These factors are taken into account while capturing the image. Camera is pointed or located at a certain height to obtain a clear and accurate image.

Image capturing is a very tricky and sensitive task because several factors like noise, weather, sunlight, material of the number plate, reflection of the light affect image capturing process. The design of the proposed system helps capture the image successfully despite these factors.

C.    Image Enhancement

Gaussian high pass filter is applied in the preprocessing phase for image enhancement. To achieve smoothing and noise removal the proposed system uses averaging filter.

(a)

(b)

Figure. 2 (a) Originally obtained image, (b) Gaussian High Pass filter applied image

D.    License Plate Detection

First and the most important step in the main processing phase is the License plate detection. Connected component labeling has been used for License plate detection in the proposed system. Connected component labeling technique can operate on a variety of information; it is particularly helpful in detecting the connected regions in the binary image.

(a)

(b)

Figure 3 (a) Binary Image, (b)Connected Component labeled image

(b)

Figure 4 (a) Image for recognition, (b) Characters recognized

E.    Character Segmentation and Recognition

To break the image piece into possible character, peak to valley technique is applied to the candidate image in the proposed system. Number plate is selected on the basis of the image region which has maximum peak in candidate character. The threshold set for the Percentage width of the ridge is 15% of the whole number plate in order to be a possible character. Minimum values of the column histogram have been set as a threshold value in the proposed system.Object recognition techniques have been used for character matching in proposed system. Template matching with the use of correlation and measurement of distance technique, called distance measurement are the most common and widely used techniques for object recognition.

Representation techniques for object moments are used in distance measurement technique of character recognition. Moments are considered as those object measurements which can the associated features of the object, such as eccentricity and the center of gravity. Draw backs of distance measurement techniques include the reduction of efficiency of the algorithm’s execution due to extensive computations and also this technique is very difficult to implement.

Template matching using the correlation is the second technique used widely for character recognition and it has been employed by us in the proposed system. Correlation is performed between the character template image used for recognition and the current acquired image in which the characters need to be recognized. Template matching is considered efficient in comparison with the distance measurement. Only limitation on the template matching technique is that image template must be properly acquired.

(a)

(a)

(b)

Figure 5 (a) Image for recognition, (b) Characters Recognized

• IV.    EXPERIMENTAL SETUP AND RESULTS

The following section depicts the whole setup for experimentation and the results of the Algorithm.

A.    Application Interface

Static camera is used to operate the proposed system. Most commonly used camera for these purposes is the CCTV (Close Circuit Television Camera). There is camera called FXCAM that has been particularly designed for automatically recognizing the number plate. Other applications where outstanding quality is needed FXCAM can also be applied there. FXCAM is a multipurpose camera performing variety of operations, it has a camera, 2000 watts IR flash, a synchronized design for the identification of vehicle automatically and an IR filter [12]. In the proposed system an LG 1.3MP static camera has been employed for acquiring the image in the proposed system, and resolution used for capturing image is 1280 x 1024

pixels. The objective of the research was to design an economical system; keeping that constraint in view a low cost camera has been used.

B.    Results

Proposed system has been tested and experimented on a variety of License Plate images used in Pakistan and has shown significantly satisfactory results. The results of image enhancement for sharpening have been presented in Fig. 6. The required sharpening of image has been obtained through Gaussian high pass filter.

(a)

(b)

Figure 6.(a) Originally obtained Image, (b)Gaussian High Pass filter Applied Image

The License Plate Detection has been achieved through connected component labeling algorithm. The results of License plate detection have been shown in Fig. 7.

(a)

(b)

Figure 7. (a) Halftone Image, (b) Connected Component Labeled Image

The results of Character Recognition phase has been presented in Fig. 8,Fig. 9 and Fig. 10.

(a)

(b)

Figure 8. (a) Image for recognition, (b) Final Image of Recognized Characters

(a)

(b)

Figure. 9 (a) Image for recognition, (b) Characters Recognized

(a)

(b)

Figure.10 (a) Image for recognition, (b) Characters Recognized

The proposed algorithm has been tested on the images captured at various timings, some in the morning sessions, some in the afternoon and others in the evening sessions. The individual results of each phase right from the start phase of plate localization, then character segmentation, character recognition and finally the recognized plate with car number are shown in the Fig. 11, Fig. 12 and Fig. 13 respectively. The results are quite good in the morning session, excellent in the afternoon session and quite satisfactory for evening sessions. Moreover the tested License plate images were of both categories i.e. Pakistani standard license plates and Non-standard license plates in Pakistan. The results for the proposed algorithm were better for Pakistani license plates as compared to Nonstandard license plates. The complete analysis of the proposed algorithm has been presented in the bar charts as shown in Fig. 11 to Fig.13 where you can see the recognition rates of each individual function of the proposed algorithm in multiple sessions depicting various amounts of lighting conditions as well as considering both the Pakistani standard license plates and Non-standard license plates.

Figure. 11 Algorithm Results in Morning Session

Figure. 12 Algorithm Results in Afternoon Session

Figure. 13 Algorithm Results in Evening Session

• V.    CONCLUSION AND FUTURE WORK

The proposed research work is an effort towards implementing license plate recognition system for Pakistani standard and non-standardlicense plates. The proposed algorithm is tested over a huge image dataset maintained inthe database andthe results are monitored to check the proposed algorithm’s recognition rate. Theproposed research has been done to provide an efficient and economical solution for license plate recognition.

The proposed system can be extended to further improve the algorithm to provide more effective results for images captured in the evening sessions. The proposed algorithm can be further improved to perform better in the case of Non-standard license plates too.