Change Energy Image for Gait Recognition: An Approach Based on Symbolic Representation

Published Online March 2014 in MECS DOI: 10.5815/ijigsp.2014.04.01

Gait recognition is a method of identifying an individual based on his/her style of walking, which is considered to be unique among individuals. Gait image sequences are captured at a distance without individual cooperation to make alert or to give early warnings when the person is far away [1, 2]. Gait recognition has gained much attention in the field of biometric research due to its ability in identifying an individual at a distance despite insufficient resolution in acquired images. Mainly gait recognition approaches are of two types, namely model-based and model-free [3]. In model-based approach modeling human body is achieved considering information such as ankle elevation, angle between joints, position of hip etc [4]. The method is found to be simple and straight forward but it is computationally expensive [5]. Model-free approach directly uses the silhouette sequences for extraction of features without any gait model. In model- free, binary gait silhouettes could be of low quality without any texture or colour information and both static (torso) and dynamic (hand and leg) features are used for gait recognition. Most of the investigations on gait recognition are based on model-free approach. Gait recognition finds its applications in the field of surveillance and other applications like robotics, clinical analysis and computer animation [1].

Several attempts have been made by many researchers to provide an efficient and effective gait recognition system. Following are the few interesting works found in the literature in this direction. Mridul Ghosh et al [6] proposed gait recognition technique, which uses corner points from edge of the image. Control points are selected from the determined corner points as features and Euclidean distance between the control points in a cyclic order are computed and stored in a database. sobel operator is used to find the edges in a gait image. Experimental results show the improvement is found in terms of recognition. Ju Han et al [7] proposed spatio-temporal representation of gait called Gait energy image for recognition of individual by gait. Change with time (temporal) information is acquired in a single image through Gait energy image. Recognition is achieved by combining statistical gait features obtained from real and synthetic poses (templates). The method is less sensitive to silhouette distortions like spurious pixels and scale changes. Experimental study shows the approach has given acceptable results. Xiaoxiang Li et al [8] have proposed a structural gait energy image (SGEI) by combining foot energy image (FEI) and Head energy image (HEI). The results show that the SGEI is better than GEI. Yumi Iwashita et al [9] extracted shadow area of a subject. Gait stripe and gait contour are the two gait features extracted by spherical harmonics and the same has been used in the reference database. A simple NN classifier using Euclidean distance measure is used for identification. Author claims that the proposed method outperforms the methods based on Fourier transform, gait energy image and active energy image. Toby et al [10] proposed a method of capturing motion part and static part such as hand, leg and torso respectively thru templates called motion silhouette contour templates (MSCT) and static silhouette template (SST). A simple

NN classifier is used in the classification. MSCT is proven to be best compared to SST in the case of SOTON dataset. The approach has shown 85% recognition rate on SOTON dataset and 80% on USF dataset. Nikoloas V Boulgouris et al [11] proposed human gait recognition approach based on comparing body components. A weight is assigned to different body components based on their contribution level for the recognition. The author claims improved performance in human gait recognition in the said approach. Sudeeep sarkar et al [12] proposed a baseline algorithm, which consists of a set of 12 experiments. Recognition is achieved through frame correlation between silhouettes using Tanimoto similarity measure. Identification rates for the 12 experiments range from 78% on the easiest experiment to 3% on the hardest experiment. Amitkale et al [13] considered width of the silhouette and the entire binary silhouette as two features of the image for gait recognition. Two approaches indirect and direct are applied. In indirect approach, feature is transformed from high-dimensional feature space to low dimensional feature space creating frame to exemplar (FED) distance. The generated vector has both structural and dynamic characteristics of individual. Direct approach considers feature vector directly i.e. without computing FED. Hidden Markov model is trained for representation of gait. The approach has proved that the side-view has higher recognizing accuracy compared to the frontal-view.

From the survey of literature on gait recognition, we understand that several methods have been proposed for gait recognition. Also, it is evident that there will be variations on gait features due to change in viewing angle, changes in cloth type, change in walking surface, carrying bag and elapsed time between sequences being compared. This variation can effectively be handled by the use of symbolic data, which are the extensions of classical data types. Symbolic data are more unified by means of relationships and they appear in the form of continuous ratio, discrete absolute interval, multi-valued and also multi-valued with weights [14]. The concept of symbolic data has been extensively studied in the area of cluster analysis [15, 16, 14, 17, 18, 19, 20, 21] and it has been experimentally shown that the approaches based on symbolic data outperforms conventional data analysis approaches [22, 15, 16]. A symbolic approach to shape representation and recognition has been explored in [23] and the concept of symbolic data analysis has also been explored in online signature verification and recognition [24]. It is also observed from the survey that the work reported in [25, 26] explored the suitability of symbolic data analysis approach for gait recognition. However, there is a scope for much more analysis in this direction particularly for gait recognition techniques.

With this backdrop, in this paper, we have introduced change energy image (CIE) and applied radon transform to these images. The radon transformation values obtained for various instances of a subject is consolidated to form an interval type symbolic data. Thus the symbolic representation of a subject in the knowledge is used for recognition. Experimental study is conducted on CASIA GAIT DATABASE B to know the efficacy of the proposed approach. Rest of the paper is organized as follows. Section II presents proposed methodology for gait recognition. Experimentation is presented in section III, followed by conclusion in section IV.

• II.    PROPOSED METHODOLOGY

The motivation for the proposed approach is to capture energy variations in static and dynamic part in a gait sequence of a gait cycle by generating change energy images (CIE) and to extract discriminative feature values for gait recognition. Input to our proposed methodology is a sequence of silhouettes of a gait cycle. A gait cycle is a basic unit of gait and it refers to the time interval between successive instances of initial foot-to-floor contact for the same foot [13]. We have assumed that the speed is constant within any specific subject gait sequence cycle (length of different instances of a subject like change in viewing angle, change in cloth type, carrying a bag is constant). However, speed could vary among reference (training) and probe (test) sequences. The proposed approach consists of the following steps. In the first step, sequence of change energy images (CIE) is generated from a sequence of silhouettes of a gait cycle. In the second step, Radon transform is applied to a sequence of change energy images. Standard deviation and maximum value from the radon transformed values is computed for four different angles, which covers the vertical, horizontal and two opposite cross sections (directions). In the third step, similarity between probe gait sequence and all reference gait sequences is established and the recognition task is accomplished.

• A. Change Energy Images (CEI)

The CASIA dataset B, used in our experiments, contain some silhouettes with spurious pixels and small holes. Morphological operations such as erosion and dilation are applied to remove spurious pixels and holes in the silhouettes. The silhouettes are extracted by applying bounding box and then silhouette images are normalized. Change energy images from pre-processed silhouettes for each gait cycle are generated as follows.

Let

I = { I1,12,......I. }                                   О be the n silhouette images in a given gait cycle. For a gait cycles consisting of n silhouettes, a set of n-1 change energy images are obtained.

Let

CEI = {CEI,,CEI2, ■ • ■,CEI ,,-•• CEI.-1}         (2)

be the set of change energy images.

First change energy image ( CEI 1 ) is computed as

CEI = 1 Z ( CEI - i ( x , У ) + I i + i ( x , У ) )               (4)

^CEI i = ¹ E ( I ( x , y ) + I 2 ( x , y ) )

I                      CEI

Where i+1 is the i+1th image in I and      1 1 is the i-1th image in CEI .

Where 1 ^{1 (} x , ^У ) and I ^{2 (} x , ^У ) are the intensity values of first and second silhouette images in a given gait cycle at ⁽ x , ^У ) coordinates. The Subsequent CEI is computed as

CEI n - 1 = 1 Z ( CEI n - 2 ( x , У ) + I n ( x , У ) )             ⁽⁵⁾

The

set of change energy images

(CEI)      t я generated

for an instance of normal walking is shown in Fig. 1.

B. Feature Extraction

The proposed feature extraction technique applies Radon transform on the CEIs obtained as discussed in the previous section. The Radon transform maps the 2-D image in Cartesian coordinate system ( x , y ) into polar coordinate system ( p , 9 ) . The Radon transform applied on CEI captures the intensity variations across various angles. The more theoretical description about Radon transform is found in [27]. The radon transform is applied on the change energy images as follows

ст = std ( R )

Thus the features extracted for every CEI in a gait

( r , c ) , ( r ? , C ) , ( г з , C ) , ( r , C )

sequence are 1    1    2    2    3    3     4    4  , which represents the maximum value r and the standard

deviation ^с

respectively for

of Radon transformed vector R

9 = 0,15,90,150.

R = radon ( CEI , 9 )

The features chosen are maximum value r and standard deviation ^с of Radon transformed vector R for a given angle ⁹ . The ⁹ chosen here are 0, 90, 15, and 150 degree, since these angles cover the vertical, horizontal and two opposite cross sections summation line respectively in radon transform of change energy images.

C. Representation

Since the gait of a person varies slightly due to change in carrying conditions, change in clothes and different normal conditions. The Radon features obtained for these instances also contain variations. These variations are handled by consolidating the features in the form of an interval type data as explained below.

Let

5 = [ ^ 1, S 2,.. SI, ,...... S . ]

r = max ( R )

be the N number of subjects.

Let

be the n instances of the subject S_I .

Let sB = [CEIX,CEI2,...,CEI,,...,CEIT]        (11)

be the change energy images for s_Ii .

Where T is the length of s_Ii (total number of change energy images). T is same for a particular subject s_Ii ( i = 1,2, n ) but T may vary across subjects, i.e., for different subjects S_I ( I = 1,2, n ).

The vector representing the features of s_Ii is given by

Si. = [ T , RF_m, RF_e 2 ," -№_в, ,•• -№_кт ]                      ⁽¹³⁾

Where RF_Iit = S_Ii ( t ) and T is the number of change energy images of s_Ii .

The minimum and maximum value of a feature r for

S kth angle of n instances of a subject I is given by r- (t) = Min ( ri 1 к (t) , ri 2k (t) , • • ■ rlk (t),•••, rink (t) )

ri+ (,) = Max(r, 1 k (t),ri2k (t),...гк( (,),...,r^ (t))

The minimum and maximum value of feature C for S kth angle of n instances of subject I is given by

Ck (t) = Min (Ci 1 k (t ),Ci 2 k (t ),• C (t )>• --^ink (t))

C+ (t) = Max (O> 1 fc (t) , Ci2k (,) ,... Сцк (t),.. . CTtok(t))

Thus, the reference sequence of a subject ^I (I = 1,2, N) in the knowledge base is represented in the form of interval-valued type symbolic feature vector as

Si =[Ti, RFi 1, RFi 2,.. .RFit,..., RFit ]

T

Where ^I is the number of change energy images of

ST a subject I and the value of I could be different for

S different I .

RF_It is the feature values of t ^th change energy images of all n instances consolidated in the form of interval for m different angles and is given by

RF it = { [ ^r i "1 ^{( t} ), ^rh^(t ) ] , ' ^c , ( ^{t ),} C i ⁺1 ^{( ,} ) ] , [ ^r i "^{( ,} ), r i 2 ^{( ,} ) ] , ' C i 2 ( ^, ), C i ⁺2 ^{( ,} ) ],...,

[ r ik ^{( t} ), r lk ^{( t} ) ] , [_°" 7k ^{( t} ), ^C Ik ^{( t} ) ], • • • , [ r Im ^{( t} ), r Im ^{( t} ) ] , [ ^C Im ^{( t} ), ^C Im ^{( t} ) ] }

Where for each k ^th angle r_Ik ( t ) , r ^k. ( t ) , C Ik (t ) and C^ (t ) is obtained as shown in equation (14), (15), (16) and (17) respectively.

The probe (test) sequence of an instance of subject

S

^I to be tested is represented as crisp feature vector and is given by

S p = [ T_P , PF 1 , pf₂ ,..., pf , ,..., PF t ]         (20)

Where T is the length of the probe sequence and PF_t is the crisp feature values of maximum radon value and standard deviation of radon values for m different angles and is given by

PF t = { Г 1 ^{( t} ), ^C 1 ^{( t} ), r ^{( t} ), ^C 2 ^{( t} ),   , r k ^(t )C_k^(t),..., Г т ^(t), ^C m ^(t ) }

Table I shows the feature values representing the CEI_t of different instances of a subject S_I and the interval-valued representation after consolidating the corresponding feature values.

TABLE I. EXAMPLE OF CRISP FEATURE VALUES REPRESENTING CEI T OF ALL THE INSTANCES OF A SUBJECT AND THE CORRESPONDING CONSOLIDATED INTERVAL VALUED FEATURES IN SYMBOLIC REPRESENTATION

Instances	r	C	r	C	r	C	r	C
	00	00	150	150	900	900	1500	1500
Bag1	30293	11341	23216	8190	8421	2064	19009	5415
Cloth1	32370	10972	24291	8325	9347	1981	19115	6105
Normal1	31002	10562	23453	8468	9003	1562	18437	5489
Normal2	31460	11159	23981	8392	9217	1681	18463	5568
Normal3	32117	10998	23775	8047	8992	1821	18081	5871
Interval	[31460,32370]	[10562,11341]	[23453,24291]	[8047,8468]	[8421,9347]	[1562,2064]	[18081,19115]	[5415,6105]

A. Similarity Computation

In order to recognize a probe sequence s _P , features are extracted from the probe gait sequence as discussed in sub section B of section II and represented as shown in equation (20). An example of feature vector values (of type crisp) of PF_t of probe sequence s_p is shown in Table II for 4 different angles. The obtained crisp feature vector is compared with the symbolic feature vector of the reference gait sequences (equation 18) in the gait knowledge-base.

for I = 1 to N

T = Min ( T , T )

^sim ( ^r k ⁽ t ). [ ^r i; ⁽ t ). ^r i + ⁽ t )] ) =

if ( ^r k ⁽ t )   ^{r - (} t ) ^andr k ⁽ t )^{s r} , + )        (25)

( r k ( t ) - r ) ( ^{r - (} t )^- r )

¹f ( ^r k ⁽ t )   r ^andr k ⁽ t )^s r - )

0             if ( r ( t ) < randr ( t )> r )

( r 4 ^- r k ⁽ t ) ) ^{( r} 4 ^- r n)

if ( r k ⁽ t ) ^a r»' t ) and r k ⁽ t ) ^{S r} 4 )

TABLE II. EXAMPLE OF CRISP VALUES OF PROBE SEQUENCE PF T FOR 4 ANGLES

Features r and p	values
r(00)	32090
p (0 0 )	11186
r(150)	24002
p (15 0 )	8412
r(900)	9340
p (90 0 )	1998
r(1500)	18863
p (150 0 )	5897

r 1 = r ik ^{- d}

^r 4 = rik + d

^d = r + ^{- r}Ik

And

To compute similarity between probe sequence and each reference sequence, the length of T in the probe sequence and reference sequence T are checked. Since the value T_I may vary from subject to subject, there is a possibility that the value T (number of change energy images of reference gait sequence of a subject) and the value T (number of change energy images of a probe gait sequence of a subject) may be different. Suppose if Tj <  T_p then, the first Tj number of CEIs of a probe sequence of a subject is compared with the reference sequence of a subject and the corresponding similarity value will be computed. If T = Tp then, the T number of CEIs of a reference sequence of a subject is compared with the T_P number of CEIs of a probe sequence of a subject and the corresponding similarity value will be computed. If T >  T p , then the first T_P number of CEIs of a reference sequence of subject is compared with the probe sequence of a subject and the corresponding similarity value will be computed. In all the above three cases similarity between probe and reference sequence of all the subjects Sj ( I = 1,2... N) in the knowledge-base is computed using equation (22).

TotalSimilarity ( s_p , s, ) = ( sim_r + sim_c )                   (22)

Where simp(sp,si) =Z Z (sim(n-(t),,n-(t),ст+(t>]))-TP-Til (29)

for I = 1 to N

T = Min ( T_p , T )

if ( ^p k ⁽ t ) ^{a p} I ⁽ t ) ^and p ⁽ t ) ^{s CT k} )

sim ( ст , ( t ), [ ст - ( t ), ст * ( t ) ] ) =

T ^ -P t )-^   if ( ^CT * ( t ) ' ^p 1 ^{and p} k ( t ) ^{s CT} I )

( ^p i>. ⁽ t ) ^{- p} 1 )

⁰             if ( ^ar ( t ) <  p ^{and p} k ( t ) >  ^p 4 )

^{( p 4    p} k ⁽ t I ) if ( ^p k ⁽ t ) ^{a p} -k ⁽ t ) _an_d ^p k ⁽ t ) ^{s p} 4 )

^{( p} 4 ^{- p} ik )

P1 = pik

p4 = p+ + d( d = p+k - p/P

• III.    EXPERIMENTS

In order to study the performance of the proposed method of gait recognition, we have conducted an experiment on the standard CASIA Dataset B. The dataset consists of 124 individuals (subjects). Each subject consists of 10 sequences, out of which 2 sequences is with carrying a bag, 2 sequences is walking wearing different clothes and 6 sequences is in normal conditions. The gait silhouettes used are in 90 degree (side view) viewing angle as this view provides more gait information than the silhouettes taken from other view angles. In the Dataset, some subjects do not have (complete) all the sequences and some of the sequence do not have a complete gait cycle. Therefore, only 120 subjects out of 124 subjects are considered in our experiment. We have measured the performance of the proposed approach for identification using cumulative match scores (CMS) suggested in [28].

The task of identification is an attempt to find given probe sequence to be one of the reference sequences, since closed-set identification is considered, which guarantees the availability of the subject in the reference database. For identification, first sequence of carrying a bag named as B1 (bag1), first sequence of coat named as C1 (cloth1) and first three different normal walking sequences named as N1 (normal1), N2 (normal2) and N3 (normal3) are used for training and second sequence of carrying a bag named as B2 (bag2), second sequence of coat named as C2 (cloth2) and rest of the sequences of normal walking named as N4 (normal4), N5 (normal5) and N6 (normal6) are used for testing. To measure the performance of identification, reference sequences S_I ( I = 1, 2… N ) are sorted, based on the similarity values computed with the given probe sequence s . If the correct reference sequence has highest similarity value with its corresponding probe sequence, then it is ranked as 1, otherwise its rank will be decided based on its position in the sorted order. We report results in terms of cumulative match scores (CMS). Table III shows the identification rate of the proposed methodology at rank 1, 5 and 10. The Cumulative Match curve for the proposed system shown in Fig. 2 represents that the performance at rank 1 is the correct classification rate (CCR). Table 1V shows the average correct classification rate (CCR) at rank 1 for other approaches and the proposed approach.

Also, the performance of the proposed system is compared with NLPR database (similar covariates i.e., normal sequences without coat and bag), which consist of only 20 subjects for lateral view with 124 subjects of normal sequences of CASIA dataset B. It is found that our method for 124 subjects achieved 95% CCR for normal sequences compared with the other methods shown in Table V for only 20 subjects.

TABLE III. IDENTIFICATION RATES AT DIFFERENT RANKS

Probe	Identification rate/Rank (%)
	1	5	10
N4	95.83	100.00	100.00
N5	94.16	100.00	100.00
N6	95.00	100.00	100.00
C2	84.16	93.33	95.83
B2	88.33	95.83	98.33

TABLE IV. CORRECT CLASSIFICATION RATE (CCR) FOR VARIOUS OTHER METHODS AND PROPOSED METHOD

Approaches	CCR (%)	Database
Yan –qiu Liu [29]	83.00	CASIA B
Su-li xu [30]	89.70
Xiaoxiang Li [8]	89.29
Khalid Bashir [31]	55.00
Proposed method	91.50

TABLE V. CORRECT CLASSIFICATION RATE (CCR) FOR VARIOUS OTHER METHODS AND PROPOSED METHOD

Approaches	CCR (%)	Database
Heesung Lee [32]	88.75	NLPR
Byungyun Lee [33]	92.50	NLPR
Proposed approach	95.00	CASIA B

Figure 2. Cumulative Match Score

• IV.    CONCLUSION

In this paper, features are extracted from change energy images (CIE) a variant of gait energy image. Change energy images provide static as well as dynamic information in detail. Radon transform is applied on change energy images with only few different angles (different directions) to extract discriminative features. The feature values are represented in the form of interval values, since it contains variations of gait feature due to change in cloth, carrying a bag and different normal conditions. Gait recognition is achieved by comparing crisp values of test (probe) sequence with interval values of training sequence (reference) in the gait knowledge-base. The results obtained from our experimentation have shown considerable improvement in terms of correct recognition rate (CCR), when compared to some of the existing methods.