Analysis of multi-modal biometrics system for gender classification using face, iris and fingerprint images

Published Online May 2019 in MECS DOI: 10.5815/ijigsp.2019.05.04

Biometrics is a technological system which measures the features or traits of individual based upon their physiometrics and behaviometrics. Humans also exhibits soft biometric characteristics like person’s gender, age, ethnicity, weight, height, gestures ,gait, accent, ear shape, length of legs and arms, skin color, hair color etc. To distinguish humans some natural ways are created using soft biometric instances. The classification of gender from biometric traits is one of the important steps in forensic anthropology, which is used to identify the gender of a criminal in order to minimize the list of suspects in a search. Very few researchers have worked on gender classification using fingerprints [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15], Iris[16,17,18,19,54], palmprint[20,21,22,24,57,58],face[23,26,27,28,29,30,31, 32,33,53,55,59,60], speech [34,35,36,37,38] and gait [42] etc and have gained the competitive results. These biometric traits which rely on any single biometric identifier often do not meet the requirements prudently as any uni-modal biometric system suffers from a variety of problems including distorted data, intra-class variations, inter-user similarity, constrained level of freedom, noncomprehensiveness, spoofs, Obfuscation (masking one’s own identity by altering the trait) and intolerable error rates and so on, which are shown in figure 1.a-g.

b

c

d

e

f

Fig.1.a- Distorted data b- Intra class Variation c- Noncomprehensiveness d- Spoofing Attack e- Effect of Ageing f-Inter user similarity g- Obfuscation

The above mentioned confines are overwhelmed by fusing multimodality which in turn provides more robustness to the results than any uni-modal system. Fusion [43, 51, 52] can be accomplished by four different ways namely they are: sensor-based, feature-based, score and a decision based fusion.

Multimodal-approach-based gender identification system has attracted attention in recent times because of ethical and reliability concern. Multiple biometrics-based gender identification represents an emerging trend and practically has many real-world applications like humancomputer interaction, gender-based advertisement, forensics, and surveillance etc. In this work, we present a multimodal biometrics-based gender identification system by integrating features from three different modalities viz Face, Iris, and Fingerprints. The paper is organized as follows: In section II, contains the literature review.The proposed methodology is described in Section Ш. After that, experimental results are discussed in section IV. In section V comparative analysis is done and finally, conclusion is drawn in the last section.

• II. Relatesd Work

Preceding research has shown that there is a possibility of authenticating an individual from their respective multi-modalities, as well as from the related literature. It is observed that limited work has been is carried out for identifying a person’s gender using their multi-modal biometrics. This section reviews the studies which have been reported on gender identification using multi-modal biometric, Xiong Li et al. [39] have performed multimodal based gender identification by combining local binary patterns and Bag of words features based on decision level fusion on the face and fingerprint traits of the internal database of 397 volunteers from Han nationality and obtained an accuracy of 94% using Bayesian Hierarchical model. Mohamed A et al. [40] have performed multimodal based gender identification by combining features binary features, Eigenvalue, Syntactic Complexity, Response length, shallow and deep syntax and mean heart rate max-min difference features are combined on five different traits i.e. visual linguistic, physiological, thermal and acoustic traits from a selfcreated database of 51 males and 53 females and obtained an overall accuracy of 80.6% using decision tree classifier. Abdenour H et al.[41] have utilized three different unimodal databases namely CRIM, vidTimit and Cohn-Kanade dataset of face and video which are further combined to form a multimodal dataset (face and video) by using local binary patterns and obtained an overall accuracy of 96.3% by support vector machine. Caifeng Shan et al. [42] performed multimodal gender identification by combining the face and gait modalities from CASIA Gait-b dataset from which frontal face image is extracted from 119 subjects which are further combined with gait videos. AdaBoost based face detector and Background subtraction feature extraction techniques are implemented respectively and using the support vector machine classifier an overall accuracy of 97.2% is achieved.

From the literature, it is observed that there are different types of fusion levels they are; match score level, sensor level, feature level and decision level fusions. Among four levels of fusion our aim in this work is to use the feature level fusion to address the issues such as efficiency, robustness, applicability and universality.

• III.    Proposed Methodology

Gender classification using multimodal biometrics generally involves three steps namely pre-processing, feature extraction and classification. The initial task is to perform pre-processing and the pre-processing techniques are application dependent. Feature computation step, which deals with extraction of textural information using fusion of multiblock local binary patterns and BSIF filters.

Further, method is evaluated with different binary classifier. The block diagram of proposed methodology is represented below in figure 2.

Fig.2. Block Diagram of Proposed Methodology

• A.    Dataset

Authors have used publicly available SDUMLA-HMT standard dataset which is collected by machine learning and application lab of Shandong University [25]. The dataset includes real multimodal data from 106 individuals out of which 59 are male volunteers and 47 are female volunteers. The dataset contains face images which were collected from different poses, expressions and accessories. For Iris, dataset images are captured giving proper direction to volunteers, images from the both eyes are collected. Likewise, for fingerprint dataset images are acquired with FT-2BU sensors, from each such subject images of both-hand thumb, index and middle finger were collected by giving prerequisite directions. Some samples of the dataset of male and female images are shown below in in figure 3.

Fig.3. Sample of SDUMLA-HMT Database

• B.    Pre-Processing

Pre-processing is very significant step and is application dependent. The fingerprint image is normalized, and then the background is eliminated. For the Iris images are normalized by contrast limited adaptive histogram equalization (CLAHE). Then, for the faces, the images are resized and normalized to 250x250. Lastly, contrast limited adaptive histogram equalization is applied.

• C.    Feature Extraction

In this step, features of an image are expatiated which results in recognition of accuracy with a simple classification module. In this work, the feature level fusion of well-known texture descriptor multi-block-local binary pattern and binary statistical image feature features are implemented. However, from available literature and empirically testing it is witnessed that fusing scheme significantly improves the gender classification accuracy and leads to promising performance.

• 1)    Multi-Block Local Binary Patterns

Local binary pattern [45] provides micro and local information from an image. Likewise the processing is quite similar to the LBP’s operation, except that summed blocks are used instead of individual pixel values of each neighboring sub regions, 16 equal sized non overlapping rectangles blocks are implemented for computing the features. From each such rectangle, the sum of pixel intensity is computed. The operator compares the central block average intensity g c with those of its neighboring rectangles {g 0,…… g c } and gives a binary sequence as an output value of MB-LBP’s operation.

MB - LBP = ^ s(g 0 - g c )² k

Where g c is the average intensity and g k = {k=1…15} for neighboring the blocks.

Г 1 x >  0 5 ( x ) = j

[ 0 x <  0                  (2)

Resulting binary pattern from each non overlapping block furthers these pattern values integrated and formed as feature vector. Comparing with original LBP, MB-LBP’s can capture fine macro structure which represents an image more precisely and is compact in size than the original LBP .

• 2)    Binarized Statistical Image Features

Binary Statistical image feature technique encodes textural information from image sub-regions. These are mere generalization over Local Binary Patterns and Local Phase Quantization [10] which is based on the statistics of natural images. The basic vector obtained by projecting patches on subspaces from images locally by Independent Component Analysis [44]. Pixel coordinate values are threshold and respective binary codes are calculated. The image intensity patterns are represented by value in neighborhood of considered pixel for I (m, n) and filter WiKxK, the response is attained as ri=Zm, n I(m, n)x Wxk (m, n)          (3)

Convolutional operation ‘x’ performed on I(m, n) which specifies patch dimensionality. A length of filter varies i.e. WiKxK = {1, 2, L}, K x K indicates the size of BSIF filter.

b ⁽ i )         r i >  ⁰

1 ⁰                              (4)

Likewise, for a given pixel (m,n) and a corresponding binary representation bi(m,n) BSIF is calculated as

BSIF^{k x k} ( m , n) = Z l = 1 ( b ( m , n) x (2 i - L ))        ^

Binary Statistical Image Feature are worked out as a histogram of pixels binary codes for each sub-region of images to characterize texture properties from it. In this experiment, filter size 11 *11 filters and 8bit length is fixed empirically which is capable of capturing sufficiently information from an image. Further this filter as a result produces a feature vector of 256 elements from each image.

• 3)    Feature Fusion

In this work, feature level fusion technique is accomplished by augmenting the feature vectors obtained from a face, iris and fingerprint modalities of a single subject. The feature is computed by concatenating vectors. Let us say FAi={fa1,fa2,fa3….fan} represents features of face modality and IRi={ir1,ir2,ir3…irn} represents features of Iris modality and Fgi ={fg1,fg2,fg3….fgn} represents features of fingerprint modality of an volunteer .Then after the concatenation a new feature vector FAiIRiFgi {fa1,fa2,fa3….fan+ ir1,ir2,ir3…irn+ fg1,fg2,fg3….fgn} will be obtained which represents individuals. The purpose is to combine FAi, IRi and Fgi features to perform gender identification using multimodal biometrics. We have implicated two local feature extraction techniques i.e. MB-LBP and BSIF; the combined feature vector is computed from MB-LBP operations i.e.160 features which are fused with 256 features of BSIF and final feature vectors are generated by concatenation rule. The computed feature vector contains416 features obtained from each male and female image. Further, these features are stored and are used to train and test the system to classify males and females subjects. For this, we have trained our system with different parametric and non-parametric classifiers to check the performance and evaluation of earlier venerated classifying techniques.

• D.    Classifiers

Linear Discriminant Analysis: is one of basic classifier which has less computational complexity commonly used for dimensionality reduction [9]. The LDA classifier works by enhancing the separation by varying variance between classes by maximizing the rationality between and within the classes [10]. It’s a linear classifier, which classifies two-class problems optimally. In this particular instance, the LDA predicts either class label 1 for male or class label 0 for the female based on their class variances [9].

K-Nearest Neighbor classifier: will classify the class label based on measuring the distance between testing and training data. KNN [46, 48, 49] will classify by suitable K value which in turn finds the nearest neighbor and provides a class label to un-labeled images. Depending on the types of problem, a variety of different distance measures can be implemented. In this work, City-block distance, Cosine, Correlation and Euclidean distance is considered with K=3 which is empirically fixed throughout the experiment. Basically, K-NN is nonparametric which finds the minimum distance d between training sample M and testing pattern N using dctybiock(M, N) = Z nj - 1|Mj- Nj|

M Nt decs(MM, N)St = (1 - , cosine        st         (MsNt)(MsNt)

dEuclidean(M, N) = V(M — N ) T (M — N )

( M Nt )( M Nt ) ^'

dcorrelatin(M,N)st = (1 - I               s        s,

( M s Nt ^' )( M s Nt ^' ) ( M s Nt ^' )( M s Nt )

Support Vector Machine : offers an opportunity to discover a non-linear transformation of the original input space into a high dimensional element space. Where an ideal isolating hyperplane can be found [47] seeking out an optimal hyperplane. In our work support vector machine endeavors and separates the two-class label form a vectors. Yi is a discriminant function F(x) = WT .Yi – Z splits data elements into two classes

by using these parameters Precision, Recall and Accuracy are computed by the following equations [50].

T

Pr ecision =---- — *100

T +F

- + -

F (x ) = WT Y, - Z >  1

Re call =    ^{(T p} )   *100

(T - + F n )

Here Yi will predict either class label 0 for female or class label 1 for male.

Accuracy =

(T P ^{+ T}N )     *1₀₀

( T p + T_N + F p + F n )

• IV.    Results and Discussion

In this proposed work, gender discrimination ability of multimodal biometrics is investigated by fusing MB-LBP and BSIF correspondingly. The overall result observed is optimal by fusion of MB-LBP and BSIF rather than using them alone. As MB-LBP gathers minor and local appearance information from each block while BSIF encodes textural information over a wider range of scale from the images. The algorithmic steps are as follows:

Input: Multimodal images of SDUMLA-HMT database.

Output: Gender Identification using Multimodal biometrics.

Step 1: Images of SDUMLA-HMT dataset is considered for the experimentation.

Step 2: Image normalization is carried out accordingly based on the requirement of modalities, and further background is eliminated and contrast limited adaptive histogram equalization is applied.

Step 3: Feature level fusion of well-known texture descriptor multi-block-local binary pattern and binary statistical image feature are implemented.

Step 4: Final feature vectors are generated by concatenation rule.

Step 5: Further, these features are trained and tested using various parametric and non-parametric binary classifiers.

The experimentation were carried out by 10 cross validation over different binary classifiers i.e. LDA, K-NN and SVM classifier on publically available SDUMLA-HMT database witch we have used where each volunteers contributes 10 images from each modality respectively. Further the Chi square test for testing the goodness of fit [56] is performed to check whether there exists any correlation between the respective modalities and it is observed that the modalities are co- related. After computation of individual feature sets of BSIF, MB-LBP and combined feature set BSIF+MB-LBP, the results have been classified into four sections False Positive (Fp), False negative (Fn), True Positve (Tp) and True Negative(Tn)

The detailed experimental results along with confusion matrix are shown from table 1 to table 3.

Table 1. Confusion matrix and results of MB-LBP

Classifier	Male	Female	Precision	Recall	Accuracy
LDA	529	61	89.66	90.89	89.2
	53	417
KNN Cityblock	555	35	94.06	91.88	92.1
	49	421
KNN Cosine	536	54	90.84	90.38	89.5
	57	413
KNN Euclidean	554	36	93.89	89.35	90.4
	66	404
KNN Correlation	539	51	91.35	90.43	89.8
	57	413
SVM Quad	566	24	95.93	96.25	95.7
	22	448

From the Table 1, it is observed that by using SVM classifier, the highest accuracy of 95.7% is achieved and lowest accuracy of 89.2 % is noted by Linear Discriminant Analysis. K-NN with a value of K=3 which is empirically fixed City-Block distance has performed with an accuracy of 92.1% and cosine distance has yielded lower accuracy of 89.5% than the city block. Similarly, with Euclidean distance, 90.4% accuracy is observed and Correlation distance tends to yield an accuracy of 89.8% respectively.

Table 2. Confusion matrix and results of BSIF

Classifier	Male	Female	Precision	Recall	Accuracy
LDA	525	65	88.98	88.38	87.4
	69	401
KNN Cityblock	567	23	96.10	95.77	95.5
	25	445
KNN Cosine	568	22	96.27	94.19	94.6
	35	435
KNN Euclidean	568	22	96.27	94.98	95.1
	30	440
KNN Correlation	568	22	96.27	93.72	94.3
	38	432
SVM Quad	586	4	99.32	99.15	99.1
	5	466

From the Table 2, it is observed that by using SVM classifier, the highest accuracy of 99% is achieved and the lowest accuracy of 89.4 % is noted by Linear Discriminant Analysis. K-NN with a value of K=3 which is empirically fixed City-Block distance has performed with an accuracy of 95.5% and cosine distance has yielded lower accuracy of 94.6% than the city block, likewise, with Euclidean distance, 95.1% accuracy is observed and Correlation distance tends to yield an accuracy of 94.3% respectively. From the above experiment noticeable results are obtained. Further, feature level fusion is carried out and the results are predicted in table 3.

Table 3. Confusion matrix and result analysis of fusion (MB-LBP+BSIF)

Classifier	Male	Female	Precision	Recall	Accuracy
LDA	543	47	92.03	92.50	91.4
	44	426
KNN Cityblock	567	23	96.10	97.08	96.2
	17	453
KNN Cosine	561	29	95.08	95.24	94.6
	28	442
KNN Euclidean	560	30	94.91	95.07	94.4
	29	441
KNN Correlation	561	29	95.08	94.92	94.4
	30	440
SVM Quad	589	1	99.83	99.49	99.8
	3	467

From table 3, we observed that by using SVM classifier, the top result of 99.8% accuracy is achieved and lowest accuracy of 91.4 % is noted by Linear Discriminant Analysis. K-NN with a value of K=3 which is empirically fixed City-Block distance has performed with an accuracy of 96.2% and cosine distance has yielded lower accuracy of 94.6% than the city block, Correspondingly Euclidean distance and correlation distance performed similarly and yielded an accuracy of 94.4% respectively.

• V.    Comparative Analysis

To realize the effectiveness of the proposed method, authors have compared the proposed work with similar works found in literature, the major drawback of the reported literature is the self-created database and required conventional setup is inapt and has limited modalities. On the contrary the proposed method outperformed other methods available the literature using a fusion of MB-LBP and BSIF filters with SVM Classifier on publicly available SDUMLA-HMT multimodal database, which yielded the result of an accuracy of 99.8% and are reported in the table 4 and fig-4.

• VI.    Conclusion

Biometrics is one of the vital indexes for gender identification through the face or other traits of the humans. Each parameter of the biometrics can be characterized as better or worse depending on the data of the individual which is acquired for identification or authentication. Likewise, in this paper, we have analyzed and identified different biometric traits for identifying gender using multimodal biometrics. Multimodal biometrics has emerged as a choice of securing the system as it has better accuracy by combining different modalities as the unimodal system imposes some limitations that can be overcome using multimodalities, through which one can enhance matching performance, increase the population coverage by reducing failure to enroll rate and also anti-spoofing. In this experiment, MB-LBP and BSIF feature descriptors have been fused together to extract the features from multi-biometrics traits i.e. Fingerprints, Face and Iris. The proposed approach is tested on SDUMLA-HMT multimodal dataset and compared to the state-of-art in terms of accuracy. The experimental results illustrates that this approach can achieve a better performance to conduct the gender classification based on multimodal biometric system. Further, this work can be extended for the fusion of different modalities and different feature sets to address the number of issues faced in implementation of biometric system with respect to the gender classification.

Conflict of Interest

The authors declare that there is no conflict of interest regarding the publication of this work/paper.

Acknoweldgement