A Study on Classification for Static and Moving Object in Video Surveillance System

Published Online May 2016 in MECS DOI: 10.5815/ijigsp.2016.05.07

Surveillance system for any object is an attractive field in computer vision and traffic monitoring. There are a lot of surveillance camera systems already installed around us but they do not monitor all of them continuously. A computer vision system monitors the suspicious human behavior and behavior of unauthorized person. Video surveillance system tells the human operator for investigation of event as well as to analyze video sequences. In this paper we have discussed video surveillance system that performs classification of static and moving objects in to several classes which can be further divided in to three classes such as human, animal and vehicle. Analysis of moving object like moving person helps in solving various problems in indoor surveillance applications. Human motion identifies the positions of human body and activities involved in video sequences. There are various activities involved in video surveillance system such as manual, semi-autonomous and fully autonomous. An intelligent video surveillance system is categorized into that system which is applied to widespread or very prevalent sensor network technology by using analysis and location recognition of images [1].

Motion of an object is used for the representation of data and its activity from other object in video applications [2]. There are three steps involved to recognize moving object.

• a)    The feature extraction

• b)    Classification

• c)    Personal identification

Feature extraction is used to extract various features from the video scene. When we extract feature then classification is used. Classification analyzes the extracted features to make classification decision very appropriate. The most important learning classification technique is used for video surveillance task [3]. This technique is called Support vector machine (SVM). When object classification is done, personal identification of human is used through face and gait recognition [4]. Face recognition involves face detection and face tracking. Gait recognition identifies humans on their walking style. The shapes of any object in video scene depend upon the different view angles of camera that may change from time to time. Appearance based method is one of the most important feature that achieves robust object classification [5]. Objects classification into predefined categories is used for operators to monitor the surveillance system with the help of some specific rules like alarming when a person is moving across a close expressway. There are various challenging problems in object classification for video surveillance system due to low resolution, different view angles, and illumination [ 6].

The remainder of the survey paper is organized as follows: In section 2 we will discuss related work. Section 3 described Object classification for static and moving object. Object classification approaches are presented in Section 4.Section 5 described benefits of object classification. In Section 6 described challenges in object classification. Finally, we draw our conclusion in Section 7.

• II.    Related Work

This section provides a survey of the related work in the areas of classification for static and moving objects in video surveillance system. Many different approaches have been developed for classification of objects over the past two decades. Various applications are used in these researches.

The proposed method represents multiple cues as intensity, motion and shape that helps to characterize the true static region and classifies their candidates in to true/false static regions using a SVM classifier, which avoids any dependency on predefined threshold values [7].We observe that true static region include low variance pixel intensity values, specific locations without motion changes and simple region boundary contours.

In Ref. [8] Vehicle detection techniques based on motion estimation include the interface-difference method, background subtraction method and Gaussian scale mixture model method. The algorithm is used to classify the moving objects including the humans or vehicles in a video is called feature based algorithm.

In Ref. [9] Reflections are the main major factors that create problems in image analysis for surveillance video, because it appears in the foreground mask extracted by using an adaptive background model. The main contributions of this paper are:

• i)    Correspondence points are extracted from a single view image sequence.

• ii)    Motion statistics for foreground classification and geometric model are integrated.

Ref. [10] proposed a 4-stage moving target detection and classification system. The crowed detection module was also performed and measured quantitatively on the PETS2006 test and PETS 2007 datasets. The PETS2006 test was comprised of 660 crowed and 1050 non-crowed samples. Further quantitative analysis was performed using the PETS 2007 database. Our PETS2007 dataset was comprised of 44 crowed and 53 non-crowed data samples. The classifier improves weak classifier by allowing them to make use of previous information when evaluating a frame.

Ref. [11] focused on GMM based representations and HMM based classifiers for motion trajectory representation and analysis. GMM based modeling face with one major problem that is the reliable estimation of the number of modes to be used. We propose to adapt the use of HMMS for trajectory analysis, classification and recognition applications.

The advantage of this technique is that it is robust time-in-variant modeling of the PDF, but the lack of temporal modeling in the formulation is the main drawback of this technique. We can use Gaussian Mixture-based HMMs for trajectory modeling to solve this problem.

In Ref. [12] we proposed human action classification that contains high accuracy and recognition method using hidden Markov model classifier. Hidden Markov model is used to classify the investigated behaviors. This model also detects abnormal events with high accuracy in comparison by other abnormal detection reported in previous works.

Ref. [13] proposed an automatic-based surveillance system that is used to retrieve multiple objects and to track multiple persons in a crowded area in different video scenes. The occurrences of a certain objects can be identified automatically during video footage by using the proposed system.

Rob Wijnhoven et al. [14, 15] presented a patch-based algorithm. This algorithm is mainly used for object classification in video surveillance. A feature vector is used in this method that is derived from template matching of a huge set of image patches. This technique has been implemented object recognition task from recent experiments.

• III.    Object Classification

A classification technique is used for management or division of various types of objects individually into the classes or groups. The characteristics of object define classes or groups. Object classification identifies the type of objects, so it can add crucial information that is needed to video analytics. Object classification therefore allows more detailed description of events or behaviors of objects. Furthermore it also increases detection accuracy and creates high level metadata that describes another data in the video sequence.

Fig.1. Classification of Static Object

• A.    Static Object classification

In static object classification the pixel of foreground are detected using three values. These values depend on intensity distance, intensity similarities and intensity probabilities [16]. When foreground pixels are detected then we remove the shadow pixels from foreground using the useful information such as chromaticity and brightness. Basically any shadow pixel in video scenes has similar chromaticity and different brightness with the background. After removal of shadow pixels, we detect static pixels using a confidence map[17] that is used to count the number of frames sequentially means that one by one and these frames belongs to the foreground for each pixel.

• B.    Moving object classification

In any video surveillance system object classification is an important factor that affects the quality of measurement of its applications e.g. to provide security in public area and searching of video. There are various challenges in classification of moving object, due to various reasons: (1) Environment conditions that are not controlled such as fog, rain, and lighting (2) Appearance details of moving objects that is not complete due to ambiguity problem, (3) distance between the moving object and camera is large, (4) video frames contains low resolution quality [18].

Now day’s video is becoming an important source for information record. Video object is becoming more attractive day by day due to its increased automation facility for security purpose in surveillance. The most important application area includes constructing of parking facility for various vehicles like car, bus, truck and makes different systems that are used for the extraction of object from videos scenes. To recognize any object is very difficult task because there are various reasons behind this task.

• 1.    The objects do not have same visual appearances and they vary from time to time due to different viewing angles.

• 2.    The distance of a camera represents the size of the moving objects.

• 3.    Object recognition for video should be real-time because when any events enter into system, system gives response to those events in time.

The main purpose of object classification is to extract distinct visual appearances or features and making appearance-based classifier [19].

All the methods in object classification are used to extract meaningful behavior of object that is useful for representation of data in video scenes. There are various features for classifying objects include size, descriptors of shape and compactness after detection of object that is based on motion. There are some limitations of the system based such features [20] is that they often work well in the restricted areas. There are few classes in which moving objects can be classified by these systems, mostly only distinguish pedestrians from vehicles. In object classification supervised learning method performs well where the target scene is the same or very similar to the source scene. If the target scene has few differences in view angle from the source scene, the classifier will decrease its performance.

• IV.    Approaches

In video object classification there are two different approaches used.

• A.    Shape-based classification

Information related to moving shape regions such as points, boxes, silhouettes and a region of an image in which some properties are constant are generally used for classification. The classification of moving object with the help of silhouette features is called shape-based classification. VASM [21] represents section of blobs in video scene that is used to classify all blobs of moving object into different classes like vehicles, group of people as classification metrics using a three-layer neural network classifier [22].Classification of objects will be more accurate with the help of temporal consistency constraints. There are various parameters based on human silhouette patterns shape used to differentiate person from other moving objects like car in a video scene [23].

• B.    Motion-based classification

Human motion that is non-rigid represents a periodic property, so human motion is seen as a perfect symbol in video scene for moving objects classification. If we want to detect and analyze periodic motion then similarity based method is used for this purpose [24]. Analysis of motion basically depends upon time and frequency, so the periodic motion is also detected and characterized with the help of analysis of motion. In video surveillance moving objects classification is applied or implemented in video surveillance using periodicity. As we know that residual flow is used to check the property of periodicity of moving objects. In motion based classification rigid objects contain little residual flow; on the other hand, an object that is not rigid has a higher residual flow for a human being. With the help of this useful information, motion of human like people or a group of people is differentiated with another moving object, such as vehicles. For moving object we use the ratio of vertical distance from the ground and the distance from side to side of each bounding box that is the component vector to separate people (pedestrians) and vehicles such as car, bus, bike etc. This value should be less than 1 for a vehicle and should be greater than 1.5 for a pedestrian. But in some situation if person are running or a long vehicle we provide flexibility for moving objects.

Fig.2. Moving Object Feature Extraction using Contours

The classification uses feature selection method that means it extracts features of objects that are needed for the classification of object [25].

Fig.3.Overview of Object Classification

There are various phases in object classification techniques:

• a.    Object detection

• b.    Feature extraction

• c.    Dimensionality reduction

• d.    Classification

In object classification technique the feature extraction phase in which we extract features of moving object is a crucial step [26]. This technique uses appearance features and geometric features. In some cases if we want to improve accuracy of classification then dimensionality reduction is used. It also reduces the time complexity of the overall process.

Fig.4. Object Classification Process [26]

Classification of vehicles is performed when a moving object such as vehicle is detected in a video. Basically classification of moving objects are done using feature based algorithm that includes persons or vehicles in a video [27]. The pixels of segmented regions that contain various features such as geometric and statistical features are considered for moving objects. Different types of classifiers are used to classify geometric and statistical features into predefined classes such as k nearest neighborhood (k-NN) that is a linear classifier [28], the support vector machine [29] that is a non linear classifier and neural network [30].

• 1)    K nearest neighborhood (k-NN):

K-nearest neighbor’s algorithm is used to recognize a pattern in pattern recognition. K-NN is used in both areas that are called classification and regression [31].

Fig.5. (a) 3-Class Classification using Nearest Neighbors (b) k-NN Regression

K-NN is also known as non-parametric method. K-NN classifier is a good classifier that calculates approximate value of function locally. All function calculation (reckoning) is delayed or postponed until classification of object is done. In K-NN we assign weights to the neighbors, so that the neighbors who is close to network contribute more than the more distant neighbors for classification and regression,

• •    K-NN classification: classification of an object depends upon the majority vote of its neighbors.

• •    K-NN regression: The main algorithm is applied for calculating roughly (imperfect data) continuous variables in K-NN regression. Examples: an example of regression using nearest neighbor.

• 2)    Support vector machines :

Supervised learning model is called SVMs. SVMs are used to recognize patterns as well as for the classification of object. SVM is a global classification model that generates partitions of object that is not over-lapped and usually employs all attributes. There is no concept of dependencies in SVM among attributes. We can represent different points in space with the help of SVM [32]. Regression and classification tasks are performed with the help of SVM. It can also handle multiple continuous and binary variables that means, a variable that can take on exactly two values. The main role of SVM is to classify latest data (new) with smallest possible errors.

Advantages: There are various advantages of support vector machines:

• •    SVM is used in high dimensional spaces, so this learning model is very effective.

• •    SVM will be very effective in those video where it uses the less number of sample in comparison of number of dimensions.

• •    SVM reduces the problems of multi class classification.

• •    SVM makes user to think about over-fitting issue in video.

Disadvantages: There are various disadvantages of support vector machines:

• •    The main drawback with SVM is that if the number of features is more than the number of sample that means there are more differences between them, then performances will be degraded.

• •    Speed and size are one of the most important limitations both in training and testing data.

• •    For a large task SVM contains high complexity to run an algorithm and it requires extensive memory.

The SVM algorithm

Fig.6. Support Vector Machines [32]

• 3)    Neural Network:

Neural network is a collection of neurons. It is used to process records one at a time. Neural network are used to perform different types of tasks like prediction, function approximation, pattern classification.

Iterative learning process that is most important part of neural network finds records (rows) that is available in the network and can be processed one at a time. In neural network the input values contains weights that are rectified each time. During this learning phase, we can predict the correct class label of input samples after adjusting the weights. In neural network, algorithm can be used to find one model that gives good classifications result of the new data.

Neural network is also used for complex data and signal classification task. Competitive neural network (CNN) provides a classification method using various features that used in segmentation. Unsupervised learning algorithm is used in this technique [33]. This learning algorithm involves network learning that is used to give correct response without any interference of an external behavior because external behavior (agent) is used in supervised learning.

Fig.7. Competitive Neural Network Architecture [34]

The network consists of nodes and there are two layers in each node – First layer is called an input layer and second layer is called a mapping layer (output layer). There are separate columns of the pattern matrix P, which describe features of an image. W is called weight matrix. The number of input and output initializes weights in weight matrix after adjusting the values. The weights can be chosen randomly from the inputs.

(a)

(b)

(c)

Fig.8. (a) All Detected Obj in Image (b) Classification of Object as tube (c) Classification of Object as Non-Tubes [29]

• V. Benefits of Object Classification

There are various benefits of object classification:

• •    A user can find an individual object very easily on the basis of its kind or group by using classification scheme.

• •    Object classification scheme provides the facility to detect duplicate objects.

• VI.    Challenges

The first challenge is to handle different camera angles and views is one of the most important challenges in object classification task for video surveillance system like zooming and lighting conditions. Due to this problem there is a variation in object appearance, shape and speed, so it gives shadows when object moves in video for a long time [35]. There are various existing approaches that only represent part of a variation not as a whole in a specific view, so these approaches is not suitable to face the challenges of multi-view invariance. If we want to differentiate one object with another moving object, the shape and size of an object is most important factor in the same camera view e.g. to differentiate multiple persons in a crowded area from moving object like car, bus and bike, which is similar in shapes and sizes. One of the most important challenges in this area is complexity of algorithm in object classification that is limited.

• VII.    Conclusion

Video surveillance system depends on human operators and it is used to detect behavior of human being that is suspected means that not safe. We can detect this type of behavior by considering information such as: color, shape, size, etc. In this paper various phases of object classification for video surveillance system has been studied.

Object classification affects the quality of measurement of its applications e.g. to provide security in public area and video semantic search. Surveillance system is also used to classify static and moving objects in a systematic manner. In object classification methods there is an important step to extract meaningful features or information for data representation in video scenes. In this survey paper two different approaches are used to classify moving objects. First approach is called shape based classification and second approach is known as motion based classification. The object classification process has various phases: Object detection, Feature extraction, Dimensionality reduction, and Classification.

In this paper various features of the pixels of segmented regions means that a small part of region for moving objects are considered like geometric and statistical features. We have also discussed two types of classifier and neural network for classification of object. First classifier is called a linear classifier (K-NN) and second classifier is called a nonlinear classifier (SVM). Both classifiers are used to classify moving object. In object classification scheme neural network are also used to classify geometric and statistical features into that classes that is predefined. There are various challenges and benefits in object classification. In this paper we also discussed transfer learning approach that is used to bridge the gap between the two scenes like source scene and target scene.