Systematic Review and Classification on Video Surveillance Systems

Published Online June 2013 in MECS

• I.    Introduction

  Video surveillance has received a great attention as extremely active application-oriented research areas in computer vision, artificial intelligence, and image processing. The early use of monitoring system was the tube camera that deployed to broadcast and monitor the industrial processing in the 1930s and 1940s [1, 2] . The traditional video surveillance systems – normally called Close-Circuit Television (CCTV) – was defective and costly since they were deployed by security teams to observe events in the scenes via visual display. To this end, automated video surveillance systems utilize integration of real-time and more effective computer vision and intelligence techniques. Therefore, automated video surveillance systems succeed to assist security staffs by generating real-time alerts and forensics investigation due to support advanced video analysis techniques.

In recent decades, expansion in video surveillance systems lead to inspire evolution in various prominent domains of technology and science such as homeland security [3, 4] , crime prevention through indoor and outdoor monitoring [2] , elder care [5] ,accident detection [6, 7] , traffic monitoring, controlling and traffic flow analysis [8-10] , airborne traffic management [11] , maritime traffic control [12, 13] , counting moving object (pedestrians, vehicles) [14] , human behavior understanding [15, 16] , Motion detection, activity analysis, identification, tracking, and classification of (vehicles, peoples, and any object of interest).There is also a growing demand for applications to support monitoring indoor and outdoor environments like parking lots, shopping mall, airport, train station and so on due to the development, availability, and low price of processors and sensors [17-20] . Thus, research in video surveillance systems are multidisciplinary field associated to image analyzing and processing, pattern recognition, signal processing, embedded computing, and communication.

Nowadays, many researchers have attempted to provide effective and appropriate video surveillance services to users by proposing and implementing network infrastructure, user interface, and middleware components. However, various published journal and conference articles have shown researchers interest in video surveillance systems; hence, it can be a time for an overview analysis. Due to diversity of articles in various journals it seems quite difficult to have direct comparision; therefore, the primary objectives of this review are:

• •    To summarize and classify research related to video surveillance systems.

• •    To prepare and offer a conceptual framework to integrate and classifiy articles correspondingly.

• •    To provide some suggestions for video surveillance systems researches according to this review.

Section I presents a methodology to perform the systematic literature. The overall features of the articles are shown in Section III. The proposed classification framework is shown in Section IV. Each characteristic of the proposed classification is described in Section IV, whereas discussions and suggestions are reviewed in Section 0. Finally, the paper is concluded with the concise conclusion in Section VI.

• II.    Method2.1    The Selection Criteria

  The area of this review was limited to academic research on video surveillance systems and its application. Furthermore, the scope of the study covered only the literature published in the time interval 20002011. The authors classify the criteria for selecting and searching articles, and devise a procedure to organize chosen articles. The research was performed using the keyword “video surveillance” through six online databases. Although, video surveillance is a broad area of research from computer science to computer graphic,

computer vision, etc., nevertheless, the authors limit the area for exploring through the online databases to the subjects which are summarized in Table 1, The following inclusion and exclusion criteria have been used for extracting of the articles.

The authors included journal articles due to these reasons: first, the highest levels of research publish in journal, and second, both academics and professionals alike mostly use journal articles to publicize their novel finding and obtain new information. Table 3 depicts the selected journal and distribution of the selected articles accordingly.

Table 1: Subject area of online databases

Online database		Subject area	Online database                  Subject area
IEEE Xplore		-    Computing and processing (Hardware/Software) -    Components, circuits, devices and systems -    Computer engineering -    Computer graphics -    Computational intelligent -    Communication networking and broadcasting -    Distributed computing/real-time systems -    General topics for engineering (Math, science and engineering) -    Robotic and control system -    Signal processing and analysis -    Software design development	-    Computer science ACM Digital Library              - Computer vision -    Digital imaging -    Engineering -    Intelligent technology -    Computer communication Springer Link Online Library      networks -    Computer science -    Image processing and computer vision -    Engineering -    Multimedia information systems -    Pattern recognition EBSCO (Discovery Services)      -Computational Complexity -    Computational modeling
Science Direct		-    Computer science -    Engineering -    Environmental science -    Social science	-    Machine learning -    Machine vision -    Communication systems Wiley Online Library              - Computer science -    General Computing

Fig. 1: Procedures of articles selection

The authors included the journal articles published in 2010 and 2011 for further detailed review and classification to examine the most recent literature;

although, in the first phase of research the exploration scope was the time frame of 2000 to 2011.

• •    The articles should discuss a novel video surveillance application or related  architecture,  techniques,

methods, algorithms, or framework.

• •    The authors excluded conference papers, text books, masters’ and doctoral dissertations, non-English papers, and unpublished working papers.

• 2.2    Articles Extraction Procedures

  The articles were selected based on the procedures illustrated in Fig. 1 and during two phases. In the Phase I, articles were searched through the six online databases. A total of 4037 articles from 2000 to 2011 were initially extracted. Afterward, 1210 articles were excluded since the words ‘surveillance video’ were not in their title or abstract. In the next step, 2132 conferences, master, doctoral or unpublished articles were excluded.

In the Phase II, 552 out of 695 articles were excluded since they do not meet the second exclusion criteria. Finally, 143 journal articles published in 2010 and 2011 remained for full review and included in the classification framework; thus, all of these articles fulfilled the predetermined inclusion and exclusion criteria.

• III.    Overall Features of the Articles3.1    Distribution by Publication Year

  In Phase I, the articles published from 2000 to 2011 that met the inclusion and exclusion criteria were retrieved. These articles are used to examine academic growth of video surveillance techniques and applications in the past decade. Fig. 2 illustrated distribution of articles by publication year. It is apparent that number of articles related to video surveillance steadily increased since 2000. There was a rise of more than 16% in number of publications from 2000 to 2011. It is clear that research on video surveillance showed continuous enhancement.

• 3.2    Distribution by online database

Fig. 2: Distribution of articles by publication years

The extracted articles by online databases are classified in Table 2. Among the six online databases, IEEE Xplore (47 out of 143 articles, or 32.9%) has the highest number of articles, since it supports publication of various journals such as: IEEE Transactions On (Image Processing, etc.), IEEE Signal Processing (Letters/ Magazine), IEEE Intelligent Systems, and IET (Computer Vision etc.).Science Direct (36 out of 143 article, or 25.2%) published the second largest percentage of video surveillance related research articles among the journals, since it also cover articles from many journals. The third and fourth ranked publication were retrieved from EBSCO (Discovery Services) (26 out of 143 articles, or 18.2%), and ACM Digital Library (17 out of 143 articles, or 11.9%) respectively. For any repetitive articles that were retrieved through different online databases only one of them considered for the classification.

Table 2: Distribution of articles based on the online databases

Online database             Number of articles

ACM Digital Library	17
EBSCO (Discovery Services)	26
IEEE Xplore	47
Science Direct	36
Springer Link Online Library	12
Wiley Online Library	5
Total	143

Table 3: Distribution of articles based on the journal

Journal articles	No of articles
Computers and Electrical Engineering	3
Computer Law & Security Review	2
Computer Networks	2
Computer Vision and Image Understanding	3
Engineering Applications of Artificial Intelligence	2
EURASIP Journal on Advances in Signal Processing	4
EURASIP Journal on Image and Video Processing	8
Expert Systems with Applications	7
IEEE Intelligent Systems	2
IEEE Signal Processing (Letters/ Magazine )	3
IEEE Transactions On (Image Processing, etc.)	43
IET (Computer Vision ,etc.)	4
Image and Vision Computing	2
Information Polity	3
Machine Vision and Applications	5
Microwave And Optical Technology Letters	2
Multimedia Tools Application	5
Pattern Recognition Letters	4
Others	39
Total	143

• 3.3    Distribution by journal

Articles associated with the application of video surveillance are distributed across 66 journals hosting at least two articles. Table 3 shows various journals that covered the publications of articles related to video surveillance. From Table 3, it can be seen that the journals of IEEE Transaction on (Circuits and Systems for Video Technology, Consumer Electronics, Image Processing, Multimedia, Parallel and Distributed Systems, Intelligent Transportation Systems, Systems

Man and Cybernetics) contain the most relevant articles (43 out of 143 articles, or 30.1%), follow by EURASIP Journal on Image and Video Processing (8 out of 143 articles, or 5.6%) and Expert Systems with Applications (7 out of 143 articles, 4.9%).

• IV.    Classification Framework

  • 4.1    Abstract Architecture of Video Surveillance

System

Fig.3 depicts the abstract architecture of video surveillance systems according to the explored articles, and it provides appropriate classification criteria for organizing the articles. The abstract layer architecture is prepared by fully reviewing, Visual Inspection and Evaluation of Widearea Scenes (VIEWS), Video Surveillance and Monitoring (VSAM), IBM Smart Surveillance System (IBM S3), Hierarchical and Modular Surveillance System (HMSS), Defense Advanced Research Projects Agency (DARPA), Proactive Integrated Systems for Security Management by Technological and Communication Assistance (PRISMATICA), Cooperative Surveillance MultiAgent System (CS-MAS), Gaussian Mixture ModeShape Adaptive Mean Shift (GMM-SAMT), Reading People Tracker (RPT), simultaneous localization and mapping (SLAM), Sensor Placement Algorithms, Facial Expression Recognition, Maximal Lifetime Sensor Target Surveillance (MLSTS), Surveillance Security (SurvSec). Therefore, the abstract layer architecture of video surveillance systems is extracted in accordance with reviewing the aforementioned articles.

As shown in Fig.3 the proposed architecture is composed of five layers: 1) Network Infrastructure Layer includes every hardware, network devices, sensor and other facilities which support video surveillance systems; 2) Processing Layer manages all components and algorithms need to store, compress, and fuse video surveillance information; 3) Communication layer is mainly responsible for the communication among its modules and external resources such as sensors, camera, data storages, and other alarming features, 4) Application Layer provides various applications and services corresponding to end user’s need, 5) User Interaction Layer offers interface to user for viewing, analyzing, adjusting and defining necessary parameters.

Layer

User

Intraction

Infrastructu re L

C ommunica tion Layer

Applicatio n Layer

Video Surveillance System

Fig. 3: Abstract level architecture of video surveillance system

• 4.2    Classification Framework of Video Surveillance System

The abstract architecture of video surveillance systems are used to develop the classification framework with the purpose of classifying extracted articles in this review. The proposed classification consists of these six layers: user interaction layer, application layer, communication layer, processing layer, network infrastructure layer, and concept and foundation layer. The abstract architecture and classification framework vary on concept and foundation layer; each classification’s layer is composed of its own categories in order to classify the articles appropriately as it is shown in Fig. 4.

User Interaction Layer

Monitoring Tools

Alarm Activation

Application Layer

Discovery

Object Analysis

Disaster relief

Communication Layer

Reasoning

Detecting

Video Analysing

Video Acquisition

Video encoding

Concept and Foundation Layer

Overview

Algorithm

Framework

Resource Management      Network Configuration

Fig. 4: Classification framework of video surveillance system

• V.    Results

The articles are categorized based on the proposed classification framework in

Table 4; while, Table 5 to Table 10 show all references accordingly. Table 4 shows that:

• i)    Research on video surveillance with subject “concept and foundation” has the highest articles distribution among other layers (36 articles, 25.2%).

• ii)    Research on video surveillance “application” has the second highest percentage with its 5 categories (33 articles, 23.1%).

• iii)    Research on video surveillance “communication” rank after application subjects (30 articles, 21%).

• iv)    The articles in the area of video surveillance “processing” are placed in the fourth position (21 articles, 14.6%).

• iv)    The distribution of articles related to “network infrastructure” are 12.6% (18 articles).

• v)    The fewest video surveillance articles are based on “user interaction” (5 articles, 3.5%).

Distribution of articles according to the classification framework is depicted in Fig.5, it shows, the concept and foundation has the highest percentage of video surveillance articles, while the most examined subject is the algorithm. Fig. 6 illustrates that research in video surveillance systems are continuously growing and attracting many researchers concern in this area. On the other hand, the fewest video surveillance articles are focused on user interaction subjects.

Fig. 5: Distribution of articles by the classification framework

Table 4: Distribution of articles based on the classification framework

Classification criteria	Number of	Subject percentage	Percentage of all
	articles	(%)	subjects (%)
1. Concept and foundation	36	100	25.2
1.1. Algorithm	14	38.9	9.8
1.2. Framework	6	16.7	4.2
1.3. Overview	7	19.4	4.9
1.4. Resource management	3	8.3	2.1
1.5. Network configuration	6	16.7	4.2
2. Network infrastructure	18	100	12.6
2.1. Camera	6	33.3	4.2
2.2. Sensor	5	27.8	3.5
2.3. DVR & NVR	1	5.5	0.7
2.4. Protocol	3	16.7	2.1
2.5. Wireless	3	16.7	2.1
3. Processing	21	100	14.6
3.1. Data fusion	4	19	2.7
3.2. Encode	3	14.3	2.1
3.3. Processing technique	11	52.4	7.7
3.4. compression technique	3	14.3	2.1
4. Communication	30	100	21
4.1. Detecting	10	33.3	7
4.2. Reasoning	5	16.7	3.5
4.3. Video analyzing	8	26.7	5.6
4.4. Video acquisition	4	13.3	2.8
4.5 Video encoding	3	10	2.1
5. Application	33	100	23.1
5.1. Protection and privacy	8	24.2	5.6
5.2. Discovery	9	27.3	6.3
5.3 Object analysis	6	18.2	4.2
5.4 Traffic controlling and monitoring	8	24.2	5.6
5.5 Disaster relief and monitoring	2	6.1	1.4
6. User interaction	5	100	3.5
6.1. Monitoring tools	3	60	2.1
6.2. Alarm activation	2	40	1.4
Total	143		100

As shown in Fig. 6, the majority of areas are still receiving researchers’ attention except for the wireless and monitoring tools in 2011. Video surveillance systems offer better service to users based on analyzing video contents, processing and detecting techniques. Furthermore, there is an increasing number of articles which mainly focus on surveillance reasoning in recent years due to deployment of machine learning algorithm such as support vector machine, AdaBoost, Bayesian network, decision tree and so on. Compression and resource management have been applied recently in conformity with developing subject of the other layers.

Articles in the network infrastructure and user interaction layer have the lowest publication rate in recent years, since most researches perform on a particular category of the proposed framework, ie. from concept and foundation to application. The inspiration of the latest research focuses not merely on concept and foundation but also case study or implication. Hence, articles of application layer related to our proposed classification framework have been raised constantly.

■ 2011

■ 2010

Subject Classification

Fig. 6: Distribution of articles by classification framework and year

• 5.1    Concept and Foundation Layer

• 5.2    Network Infrastructure Layer

Table 5 indicates the references related to the concept and foundation layer and its categories which cover fundamental methods and theories to construct video surveillance systems. Algorithm is crucial part of concept and foundation, so all researches in this category include new or developed algorithm based on computer vision, image or video processing, signal processing, and machine learning that offers the basis for an intelligent system technology.

Table 5: References of concept and foundation layer

Classification criteria	References
Concept and foundation Algorithm	[21] , [22] , [23] , [24] , [25] , [26] , [27] , [28] , [29] , [30] , [14] , [31] , [32] , [33]
Framework	[34] , [35] , [36] , [37] , [38] , [39]
Overview	[40] , [41] , [42] , [43] , [44] , [45] , [1]
Resource management	[46] , [47] , [48]
Network configuration	[49] , [50] , [51] , [52] , [53]

The framework of video surveillance systems looks like a structural design of a house. On one hand, the framework depicts a generic idea about how the system will work and how it can be implemented; on the other hand, it illustrates the common ability of the proposed system, data flow of system, system management, and user interaction part of the system.

The overview section provides the general history of video surveillance systems in addition to common classification. The resource management is focused on the framework to manage all available resources in the system from data (include video/audio/image) to hardware and network devices. Finally, all methods for configuring network and network devices are represented in network configuration.

Table 6: References of network infrastructure layer

Classification criteria	References
Network infrastructure Camera	[54] , [55] , [56] , [57] , [58] , [59]
Sensor	[60] , [61] , [62] , [63] , [3]
DVR & NVR	[64]
Protocol	[65] , [66] , [67]
Wireless	[68] , [69] , [70]

• 5.3    Processing Layer

• 5.4    Communication Layer

Table 7 demonstrates the references based on the processing layer categories. The processing layer depicts an intellectual part between network infrastructure and communication layer. The main functionality of the processing layer is to process and analyze video images and to reduce the necessity bandwidth of the network infrastructure.

Data fusion is used as an effective technique to enhance the operation of detecting systems by integrating data from different sources. Video encoding process prepares appropriate format and pattern for recording and transmitting the video for target monitoring. Video processing techniques (like model or feature driven approach, spatial differentiation, background removal, background frame differencing, optical flow field, feature tracking, edge detection, etc) not only offers segmentation of video into foreground and background elements to obtain the desired object of interest but also, it is a vital preprocess for applications such as human motion analysis and object based video encoding. Video compression techniques (such as H.261, H.263, and H.264/AVC) are used as a novel coding technique to lessen video stream redundancy and to enhance the compression efficiency. Most video compression techniques integrate temporal motion compensation and spatial image compression.

Table 7: References of processing layer
Classification criteria	References
Processing Data fusion	[71] , [72] , [73] , [74]
Encode	[75] , [76] , [77]
Processing technique	[78] , [79] , [80] , [81] , [82] , [83] , [84] , [85] , [86] , [87] , [88]
compression technique	[89] , [90] , [4]

As shown in Table 8, all the references of the communication layer categorize into detecting, reasoning, video analyzing, video acquisition and video encoding. Video detecting category addresses issues regarding to automatic detection of anomalous, forbidden, dangerous events or abandoned object (counting moving people, ship detection, after-the-fact event, intruder detection, trajectory-based unusual behavior detection, motion detection, multiple moving object detection, face detection, pedestrian detection, vehicle detection, unattended object detection, etc) .

Table 8: References of communication layer

Classification criteria	References
Communication Detecting	[91] , [92] , [93] , [94] , [95] , [96] , [97] , [98] , [99] , [100] ,
Reasoning	[101] , [102] , [103] , [16] , [104]
Video analyzing	[105] , [106] , [107] , [108] ,[109], [110] , [7] , [111]
Video acquisition	[112] , [113] , [114] , [115]
Video encoding	[116] , [8] , [117]

Object detection is performed by common statistical learning techniques with dynamically learning background model of the scene and applies the reference model to find out which section of the scene match with moving object. Reasoning refers to generating new explanations, facts and knowledge of dynamic scenes by applying inference engine and method (rule and case based reasoning, Bayesian network, decision tree). The applications of reasoning techniques in video surveillance systems overcome the problem of behavior interpretation, goal based trajectory, belief propagation, occlusion reasoning and reasoning under uncertainty situation. Video acquisition, video analysis and video encoding are three main components of each video surveillance systems.

The video acquisition is responsible for requesting image from local (stationary or active) camera on local network or from IP-camera localized on internet. Afterward, the video encoding part encodes the received image stream as sequence of frames and transmits them into video analysis component to perform basic content based video techniques and algorithms such as object recognition, tracking and motion detection and so on.

• 5.5    Application Layer

Table 9 indicates all references based on detailed categories of applications layer. Applications of surveillance systems emerged in areas such as crime prevention through homeland security, monitoring shopping malls and parking garages, monitoring and controlling critical infrastructures and highways. The most important characteristics of surveillance computing applications are summarized as: first, to provide autonomous service in order to operate without human control, second, to anticipate future event, behavior, and moving point, third, to monitor, control and alert any unplanned activities.

Table 9: References of application layer

Classification criteria	References
Application Protection and privacy	[2] , [118] , [18] , [6] , [5] , [119] , [19] , [17]
Discovery	[120] , [121] , [122] , [123] , [124] , [125] , [126] , [127] , [128]
Object analysis	[129] , [13] , [130] , [131] , [132] , [15]
Traffic controlling and monitoring	[133], [10] , [134] , [135] , [136] , [137] , [9] , [12]
Disaster relief and monitoring	[138] , [139]

• 1) . Protection and privacy category addresses the applications of public and private protection issues, smart home environment, home nursing, elder care, ATM setting, site security monitoring (museum, airport, train station, bank, etc).

• 2) . Discovery category is comprised of all researches focusing on identifying, tracking and monitoring activities of interest such as pedestrian tracking, gait-based analysis and identification (people identification), autonomous navigation, observing moving object, monitoring complex environment.

• 3) . Object analysis cover problems of segmentation, classification and recognition in area such as face recognition, object recognition (ball and player recognition in soccer analysis), license plate recognition, activity recognition, facial recognition, classification of moving vessels, behavior analysis and segmentation, multi-class object classification.

• 4) . Traffic controlling and monitoring addressing issues like, vehicle flow rate estimation, road network protection , automate speed enforcement, traffic flow density on freeway, maritime traffic.

• 5) . Disaster relief and monitoring includes applications such as fire detection and suppression, and safety of construction workers.

• 5.6    User Interaction Layer

Table 10 indicates the references based on user interaction layer’s categories. The research of this layer is mainly focused on:  1) developing dynamic environment which cover problem of small interface on mobility device, 2) devising interactive spatial navigation tools, 3) presenting visual interface for query processing, 4) enhancing technique for reducing false-alarm rate.

Table 10: References of user interaction layer

Classification criteria	References
User interaction Monitoring tools	[140] , [141] , [142] ,
Alarm activation	[143] , [11]

• VI. Discussion

  Research on video surveillance systems are continuously being enhanced as shown by evidence of publications from 2000 to 2011 (Fig. 2. ). Although, many articles, white papers or reports of various projects have been published, the fully automated and practical uses of video surveillance systems are not implemented comprehensively. Table 4 indicated that most of the researches have addressed the problem of concept and foundation layer. The coverage scope of application in the majority of articles is restricted to parking garage, shopping mall, hospital, airports and so on. Since, the knowledge and equipment of video surveillance system are almost standard, hence, the architecture, technique and method of video surveillance modeling and inference, user equipment, network implementation in each project are almost the same. Furthermore, communication and processing services and applications use various level of video content analysis and adjust the way they perform their functions according to the recent surveillance context. Despite, the cooperation and interaction among component of processing and communication layer are investigated in literatures; still there is a lack of standard for cooperation, interaction and process in different video surveillance systems. Finally, the authors figured out that video content analysis and processing, video and metadata storage management and video surveillance application and services are involved in ubiquitous computing context and should be considered as a key factor for new applications in this area. Answering the following question may define future research outline in video surveillance systems.

• 6.1    How to integrate heterogeneous data and extract contextual knowledge and information for various surveillance functions?

• 6.2    What are the video analysis challenges and needs on video surveillance systems?

There is a great need of developed multisensory data fusion methods and techniques to integrate dynamically heterogeneous flow of information supplied by various surveillance sensor networks into a coherent multimodal model of the observed subjects. Furthermore, the real time signal processing techniques should be developed for evaluating, identifying human behavior and environmental multimodal factors like, background sound, gestures, ambient lights, human gait, human voice, facial emotions, and so on that prepare contextual data and information for the particular surveillance purposes. A multidisciplinary situation assessment and context-system -which consist of multicultural sociology, cognitive psychology, distributed multimedia, artificial intelligence, learning systems and designing software components- should be eventually developed in order for real time evaluation of the emotional behavior and activity of the human recognized and identified as a potentially securityinterest observed in dynamic environment.

Although, many algorithms and techniques are developed to deal with video surveillance problems, only few of them are implementing under real case conditions and scenarios. Nevertheless, movement detection (license plate and specific object detection) and specific behavior detection (running, walking, and carrying and holding an object) are examples of well developed video analytic techniques; however, the following aspects need to be enhanced and developed:

• •    People counting in crowd from different view cameras angles.

• •    Real-time face recognition in crowded environment.

• •    Color areas segmentation, for instance improving the color information of a car may enhance the accuracy of the plate recognition technique.

• •    Data fusion for adapting current recognition and detection methods and algorithms (e.g., combining gait recognition with face recognition, or merging video data from temperature sensor, pressure sensor, infrared, and so on).

• 6 .3 How to compare and evaluate the performance of video analytic techniques on the video surveillance systems?

Many of video analysis algorithms works with stationary camera under good resolution and adequate constant lighting, if a sudden problems or changes occur in the condition of camera (moved camera, dust, spider webs) it can affect the accuracy and robustness of the most current tracking, detecting and recognizing algorithms [144] . Therefore, the video analysis algorithms need to be robust enough to challenge with different lighting changes (artificial and natural) and various weather conditions.

Some systematic evaluation methods are available for assessing the developed video analysis methods by applying ground truth (annotated data) and selecting a measurement distance to assess differentiation between the ground truth and the developed system’s responses. Several of these systems are listed follows:

• •    TRECVID (Text Retrieval Conference Video Retrieval Evaluation): are in progress workshops launched by the National Institute of Standards and Technology (NIST). These workshops concentrate on various area of information retrieval research in video content retrieval.

• •    CREDS (Challenge for Real time Event Detection Solution):  make it possible the performance

evaluation by predefined event detection based on Paris subway surveillance annotated.

• •    PETS (Performance Evaluation of Tracking and Surveillance):  This gives opportunity to the

researchers to submit their algorithms for online evaluation.

• •    FRVT  (Face Recognition  Vendor Tests): by

conducting various face recognition tests

• •    i-LIDS (Image Library for Intelligent Detection Systems): launched by the UK government to assist the evaluation and development of video analytics systems. It facilitates to make an evaluation for these five event detection scenario: parked vehicle, sterile zone, doorway surveillance, new technologies, and abandoned baggage.

• 6 .4 How to assess validity performance of video surveillance systems?

Most of these performance evaluation techniques and methods have been developed based on their specific measurement metrics and standard benchmark video sequence; thus, there is an essential need to develop a generic performance evaluation method with comprehensive metrics and evaluation tools for various video processing techniques and problem.

It is vital to enhance performance evaluation of designing and managing video surveillance systems. Hence, various models such as Information System (IS), Information Economics Balanced Scorecard (BSC), Control Objective for Information and Related Technology (COBIT), and so on, might be applied to recommend measurement means that can assess the accuracy of systems by examining different issues. As a consequence of the specific characteristic of video surveillance systems the current information system evaluation technology does not offer methods to formally verify describe, analyze and implement them. Hence, research on the performance measurement model which reveals the specification of video surveillance systems should be accomplished.

• VII. Conclusion

  This review was conducted based on the proposed framework with the intention of presenting a comprehensive review on video surveillance systems. The review initiated the video surveillance concept and foundation, network infrastructure, processing, communication, application, user interaction and listed all research related to each layer of the proposed framework on video surveillance consequently. The review examined all the research of the concept and applications, and reviewed them according to current and ongoing problem of video surveillance systems. The review conducted by exploring the video surveillance systems literature from 2000 to 2011 applying a keyword and article title research. The results revealed that interest toward video computing increased greatly in design and implementation of network infrastructure, user interaction over various devices such as handheld device and ordinary monitoring device, communication and processing which performed essential action to provide effective applications. The concept and foundation provided the basis techniques, methods, architectures, and frameworks of video surveillance services and applications. The proposed framework provided a comprehensive improvement guideline of fundamental elements and association among these elements in video surveillance systems research. The authors wish to attract researchers’ interest and increase motivation to examine video surveillance systems issues and applications. The video surveillance computing can be implemented in various research areas from cognitive science to human behavior.

Although, the authors attempted to develop a classification framework and to categorize extracted literatures respectively, there were the following limitations: firstly, some articles might have been ignored since the literature has only been extracted based on keyword, abstract and title, whilst, in most cases the title completely cover the content but this is not always correct. Secondly, this review was conducted by limiting extracted literature to 2010 and 2011 with the purpose of investigating how video surveillance systems have advanced during these periods. Thirdly, practical report, white paper, conference paper and many other papers related with video surveillance systems were not reviewed. Finally, the association between the real projects and the proposed framework of video surveillance systems are not fully clarified.

It is concluded that among assessed methods, techniques and algorithms for research in real time video surveillance processing very few researches have been conducted on user interaction aspect to present an appropriate means especially for offline inspection in after-the-fact activity video analysis, video surveillance mining and browsing.