Real Time Object Tracking using FPGA Development Kit

Published Online October 2014 in MECS

FPGAs are great fits for video and image processing applications [5-7], such as broadcast infrastructure, medical imaging, high definition video-conferencing, video surveillance, and military imaging. Video and image processing solutions from Altera [7] FPGAs include optimized development tools and kits, reference designs, video compression IP, and interface system IP. It also includes Altera's [8] video and image processing development kit. These solutions minimize the cost; improve the performance, and productivity for many video and imaging applications.

Methods of tracking objects by means of computer vision techniques exist for quite a few decades [9]. The Hawk-eye tennis system is a common example of real time object tracking. This system is commonly used to track the movement of ball in most of the major sport events like Wimbledon or any international cricket match.

Other existing applications include direction finding of rovers on Mars, tracking missiles for military purpose and controlling fighter jets, motion based recognition, traffic monitoring, human computer interaction, automated surveillance [10]. Our system is also based on real time object tracking by using image processing techniques which are implemented using FPGA development kit.

Another proposed the FPGA based object tracking system which is based on background subtraction and also try to improve the object tracking algorithm and object region identification, and also proposed the object tracking algorithm by implementing the highly parallel hardware. But this system needs to improve the sensitivity of tracking algorithm [11].

According to another proposed method object detection not only based on background subtraction but also classified into point detection schemes [12].

Another FPGA based real-time single object tracking system which is based on the chromatic information. This system can tract a solid color object [13].

Another proposed FPGA based tracking system which uses multiple cameras to track multiple objects [14].

Another proposed real-time object tracking system by using smart cameras in which uses local energy features for representing the target region and the phase features used to estimate the motion pattern [15].

Object tracking with a Pan-Tilt Camera is one such wonder of Image Processing which itself has applications in various civilian and military fields. These applications include air traffic control, direction-finding, error tolerant systems, judgment problems, inspection, target recognition, command and control, sensor management and weapon assistance.

Our system is based on tracking the selected objects using image processing algorithm. We first select the object then start the tracking process by controlling the movement of camera to capture the movement of tracked object. The hardware is integrated with LCD to provide a Graphical User Interface (GUI).

The paper is organized as follows: section 2 explains the design of the system. Section 3 discusses the working of the system. Section 4 is for the results and section 5 concludes the paper followed by references used in the work.

• II.    System Model

The work is divided into 4 parts. The first part is to process the video signal using FPGA development kit. The second part is to select the target using joystick connected with the FPGA. The third part is the tracking of the target. Finally, the last part is to control the movement of camera using DC gear motor.

The overall block diagram of the proposed system is illustrated below.

Fig. 1. Block diagram of the proposed model

• A.    Processing of Video Signal

The first step is to connect the camera with the FPGA development kit. The camera is fed in the composite video input port of FPGA kit. The FPGA development kit has TV decoder which converts the analog format into raw digital format after that it is fed to the FPGA development kit. This raw digital data is further processed by the module to give the compulsory YcbCr (luminosity and chrominance) components of the image. An YcbCr to RGB (Red-Green-Blue) module is also introduced to change the pixels consequently to show it on VGA (Video Graphics Array). The image acquisition and VGA display core is utilized for helping in image processing algorithm. Here the FPGA development kit take video from camera then processes the video and then displays its contents on the LCD or monitor. The particular frame of the video captured by FPGA kit is illustrated in fig below.

Fig. 2. Sample frame captured by FPGA kit

The next section explains the procedure to select target.

• B.    Target Selection

When the system is turned ON, initially the target selection switch, which is connected with the FPGA, is switched OFF. When it is turned ON, then the target selection box appears on the screen on its default position which is set through the programming and can be changed by reprogramming the device. The appearance of target selection box is illustrated in fig below.

Fig. 3. Appearance of target selection box

To select or change the position of the target selection box, we also connected the joystick with the FPGA kit. The joystick consists of four switches, which can move the selection box in order to capture the target. First we move the selection box on the target position then its position is captured by using the push button connected with the FPGA kit. In this mode we only select the object which we want to track by using camera. The movement and selection of target is illustrated in fig 4 below.

The next part explains the procedure to track the object.

• C.    Tracking of the Target

The FPGA development kit processes the video from the camera and displays it on the LCD which is connected with the development kit. When the object tracking mode is started, then the selected object is treated as white color on the screen. The FPGA is programmed in such way that it considers noise to all other areas which has other color intensities with the target selection box. It can be observed from the image below.

Fig. 5. Tracking of object

The tracking algorithm is designed in such a way that it checks the selected object first that weather it is in centre or not. If the object is at centre on the display screen than FPGA development kit gives 00 output through its general purpose input and outputs (GP-I/Os) and If the object is at right border on the display screen than FPGA development kit gives 01 as output and if the object is at left border on the display screen than FPGA development kit gives 10 output through its GP-I/Os. In this way the tracking is performed by moving the camera depending upon the object movement. The next part explains the movement of camera in order to track the target.

• D.    Camera Control using DC Gear Motor

The camera is attached through the DC gear motor which has windings that take an electric charge and gets discharge through a magnet, this in turn creates a field which drives the stator and makes the motor spin. We interfaced the FPGA development kit with the current driver circuit using an opto-coupler. The opto-coupler is simply used to protect the GP-I/Os of FPGA kit from the high current that may come from current driver.

After interfacing with FPGA, the opto-coupler circuit takes input from FPGA kit and gives output to the current driver circuit in which we used H-bridge Relay to rotate motor clockwise and anticlockwise for the input of 01

and 10. The motor is connected to H-bridge Relay. The main function of the motor is to rotate the camera according to the object. If object moves left side than motor moves anti-clock wise until the object position comes in centre and if object moves right side than motor moves clock wise until the object position comes in centre.

• III.    Working of the System

The object tracking system is designed by using the FPGA development kit. The first step is to get the video signal from the interfaced camera then process it with a color segmentation algorithm by using FPGA kit. Then the video is fed in the complex video input format and it is then converted into the raw digital video format by using the decoder. This raw digital data is additionally processed by the module to give the compulsory YcbCr components of the image to change the pixels of the image consequently to show it on VGA. Then to select target, we used four switches of FPGA development kit with the help of joystick. When the target selection box is on the selected object then the target is locked by using additional switches connected with the FPGA kit. When the object tracking is started the selected objects turns white on the screen. The FPGA kit is also programmed so that it checks the location of the target by generating different output values. If the selected object is at the center of the display screen, the FPGA kit generates ‘00’ output. If the object is at right and left border it displays ‘01’ and ‘10’ respectively through its GP-I/Os. The camera is attached to the dc gear motor for movement which has twist of wires that acquire an electric charge and then discharge through a magnet consequently field produces which drives the stator and makes the motor spin. The FPGA development kit and current driver circuit is interfaced through an opto-coupler circuit which isolates the current driver circuit and FPGA development kit. As opto-coupler circuit takes input from FPGA development kit and provides output to the current driver circuit with the help of H-bridge relay which is connected to the motor whereas H-bridge relay rotates motor clockwise and anticlockwise respectively for the input of ‘01’ and ‘10’.The motor starts moving according to the movement of the camera. If the object precede left edge than motor moves anticlockwise until the object place arrives to the centre and if object moves right edge than motor moves clockwise until the object position comes in centre. Therefore the working on the object tracking is obtained by throwing any object several times whose pixel is in sequence measure the time response.

The working of the system is explained using figures below.

The first image illustrates the tracking mode after the selection of object. It is showing the display of camera on LCD as it is also cleared that the object is in the center region and the FPGA kit is in tracking mode. On other hand, the second image is showing the attachment of camera with the hardware. At the moment the camera is at the centre because the object to be tracked is also in the center position. When object moves to left side than camera tracks the object and also rotates anticlockwise until object comes in centre region of the display screen.

• IV.    Results

The system is developed and calibrated by performing many trial experiments under different circumstances. The proportion error is then evaluated with numeral of times of successful responses. It is found that the error in tracking is directly proportional to the speed of the object to be tracked. The error in tracking also increases with the decrease in the frame rate at constant speed of the object. The efficiency of tracking of camera is much higher when the object contains uniform texture. The efficiency of tracking less shiny objects is much higher than of shiny surfaces. The lighting also plays an important role as intense lighting or extreme darkness can affect the working of the camera in order to take video signal.

Fig. 6(a). Snaps of working of the system

Fig. 6(b). Snaps of working of the system

V. Conclusion and Future Work

The system is developed to track objects in real time using FPGA kit. The algorithm is based in order to increase the speed and reduces the consumption of memory resources to an immense amount. The implementation of image processing algorithm on FPGA makes it feasible to be fitted on any system.

The system can be enhanced in the field of auto-aiming and missile tracking in order to select and track the target. Secondly, the system can be enhanced by using an auto focus system which can select the target automatically. It can also be used as a security system by using motion detection techniques. The obstacle detection and line tracking can also be achieved using the designed system. Furthermore, it can also be used for traffic controlling by velocity and distance estimation.