A new way to age estimation for RGB-D images, based on a new face detection and extraction method for depth images

Published Online November 2018 in MECS

Aging effect, Influences our life aspects during time. With having right knowledge about this phenomenon, we can manage it properly for the rest of our lives. But how much is the effect value of this phenomenon? Using image processing techniques, it is possible to answer this question. Automatic age estimation is very useful in medicine, psychology, 3D modeling, soft biometrics [2], human-computer interaction (HCI) [1], security control [2], surveillance monitoring [3], and electronic customer relationship management [2]. This section pays to some of the basic definitions of the subject. Sections II, III, IV and V respectively represented, some prior works on the subject by other researchers until 2017, our proposed method on age estimation using color-depth (RGB-D) images and proposed face detection and extraction method for depth images, evaluation results on three benchmark face databases and finally conclusion and suggestion on subject for improving the proposed method.

• A.    Image processing in pattern recognition

Using of computer algorithms for manipulating digital images is called image processing. Digital image processing focuses on two major tasks [4]:

• •    Improvement of pictorial information for human interpretation

• •    Processing of image data for storage, transmission and representation for autonomous machine perception

Pattern recognition is the process of classifying input data into objects or classes based on key features. There are two classification methods in pattern recognition: supervised and unsupervised classification. Pattern recognition has applications in computer vision, radar processing, speech recognition, and text classification [5] [6]. Changing, learning, improving and manipulating digital image based on some patterns by computer algorithms called image processing in pattern recognition.

• B.    Age estimation

Aging effect will determine by gender and race, and it is distinct in different weather, and also it is depended on other factors like job, family environment, diet, sicknesses and hereditary effects. So we can’t say, “age detection” and should use “age estimation”, because it is not a precise subject. The only way that we can use age detection, is when using the face patterns during a person's life time, and that is just belonged to that specific person. However, using image processing techniques, there is high possibility to estimate different race people ages. With extracting proper patterns from digital image and using appropriate pre and post processing methods, age estimation is possible with good accuracy. Figure 1

represents the aging effect on two people from the FG-Net aging database [8] in different ages [7].

• C.    Depth image

Depth images mostly are using for range and distance detection. There are several depth sensors, which most of them use infrared spectrum to sense the distance between the object and the sensor. Kinect is one of the most popular depth sensors in the world, especially among researchers. For more information about Kinect sensors it can be referred to [9]. Depth images are so popular in 3D modeling in making animation and--games [10], HumanComputer Interaction (HCI) [11], industry [12], robotics [13] and more. Here we are using it for increasing the accuracy of age estimation along with color image.

Fig.1. Aging effect on two persons from the FG-Net aging database [8] in different ages [7]

• D.    Face detection and extraction

The process of distinguishing a face object from other objects in the image is called face detection. After face detection process and for working on just face object, it is needed to extract the face, which it is possible by cropping technique. There are diverse face detection methods like ellipse fitting or viola and Jones face detection algorithm, which in this paper we used from viola and Jones algorithm for face detection of color images. This algorithm is demonstrated in detailed in section 3. Figure 2 represents some examples of face detection from RGB and Depth images [14, 15]. Face detection is first part of face recognition, gender recognition, facial expression recognition and other like activities.

• II.    Related Works

Age estimation and face detection are active field in image processing. Because of the lack of space and better readability of the paper, History and prior works on age estimation and face detection from last decade, until now, is listed in Table 1. Despite of Table 1, there are mentionable researches on automatic age estimation and face detection purposes such as Geng, Xin, et al, work in 2006 on AGES (AGing pattErn Subspace) method for automatic age estimation. The basic idea was to model the aging pattern, which is defined as a sequence of personal aging face images, by learning a representative [35]. Also, Hayashi, Jun-ichiro, et al, succeeded to make a system for age and gender estimation based on wrinkle texture and color of facial images in 2002 [36]. Also [37, 38, 39] researches on age estimation using color images are so valuable. Yang, Guangzheng, and Thomas S. Huang made a new method to locate human faces in a complex background in 1994 [40]. The system could locate unknown human faces spanning a wide range of sizes in a complex black-and-white picture. In 2000, Wang, Jianguo, and Tieniu Tan made a system for face detection based on shape informations [41]. Using fuzzy theory in face detection was Wu, Haiyuan, Qian Chen’s idea in 1999 [42]. A great survey on face detection algorithms could be found in [43].

Fig.2. Examples of face detection from RGB and Depth images [14, 15]

Table 1. Some of the prior researches on age estimation and face detection algorithms in last decade

Function	Features and descriptions	Classifier	Database	Accuracy	Ref	Cite
Age Estimation	PCA	SVM	FG-NET	Mean Absolute Error (MAE): 6.77	[16]	728
Age Estimation	Patch-based Gaussian Mixture Models (GMM) and Hidden Markov Models (HMM) Supervector	Nearest Centroid Classifier (NCC)	Novel	MAE: Male= 5.397 Female= 6.333	[17]	49
Age Estimation	Preprocessing = Active Appearance Model (AAM) Feature = Least Angle Regression (LAR)	Support Vector Regression (SVR)	FG-NET	MAE: 5.70	[18]	43
Age Estimation	Ordinal discriminative aging features and Preserving Locality and Ordinal Information (PLO) method		FG-NET	MAE: 4.82	[19]	90
Age Estimation	Facial Component Localization using Active Shape Model (ASM) and Biologically Inspired Feature (BIF)	SVM	FG-NET PCSO	MAE: 4.7 MAE: 7.2	[20]	115
Age Estimation	convolutional neural network (CNN)		MORPH	MAE: 3.63	[21]	56
Age Estimation	convolutional neural network (CNN)		Novel	84.7% +,- 2.2	[22]	211
Age Estimation	convolutional neural network (CNN)		MORPH AFAD	MAE: 3.27 ± 0.02 MAE: 3.34 ± 0.08	[23]	30
Age Estimation	group-aware deep feature learning (GA-DFL)		FG-NET MORPH Chalearn-Challenge	MAE: 3.93 MAE: 3.25 MAE: 4.21	[24]	6
Face Detection	RGB images (Weak in Depth images)				[25]	16250
Face Detection	RGB images (Using Ellipse fitting in the paper, page 6)				[26]	23
Face Detection	Depth images				[27]	22

• III.    Proposed Method

This section pays to the important algorithms made by other researchers and proposed method's procedure.

• A.    Viola and jones face detection algorithm

Viola and Jones algorithm is one of the best face detection algorithms, which is using for a long period of time in face image processing. This algorithm is so fast and robust, and it could be used for depth images too, but with lower accuracy. This algorithm is an object detection algorithm and could be used for any learned object but mainly uses for face. The algorithm has four stages:

• 1.   Haar Feature Selection

• 2.   Creating an Integral Image

• 3.   Adaboost Training

• 4.   Cascading Classifiers

• 5.    After detecting face by this algorithm, simply it can be extracted by cropping technique. For more information and details about this algorithm could refer to [25].

• B.    Proposed face detection and extraction from depth image

There are a lot of face detection algorithms for color or RGB images, but there are few face detection algorithms for depth images like [27]. So it is decided to make a fast, precise and robust face detection and extraction algorithm for depth images. The main body of the processes is to find the closest pixel to the sensor (nose tip) and using the standard deviation filter [28] and ellipse fitting [29] techniques on depth image. The complete procedure of proposed face detection method is demonstrated in Figure 3.

Fig.3. Proposed face detection and extraction method’s workflow out of depth data

• C.    Image entropy

Entropy is a statistical measure of randomness that can be used to characterize the texture of the input image. Image entropy is a quantity which is used to describe the business of an image, i.e. the amount of information which must be coded for by a compression algorithm. An image that is perfectly flat will have an entropy of zero. Consequently, they can be compressed to a relatively small size. On the other hand, high entropy images such as an image of heavily cratered areas on the moon have a great deal of contrast from one pixel to the next and consequently, cannot be compressed as much as low entropy images [4]. Image entropy formula is as (1):

Entropy = - Z ptog^P (1)

In the above expression, Pi is the probability that the difference between two adjacent pixels is equal to i, and Log2 is the base 2 logarithm [30].

• D.    Proposed age estimation method

After face detection and extraction for color images by viola and Jones algorithms and the same procedure for depth images by proposed face detection and extraction method from depth images, it is time to implement proposed age estimation method on the preprocessed images of databases. The main idea is to use a specific filter, for color and entropy filter for depth images and sum of the gray-scale pixels for final normalization. Proposed edge detection filter is clear in Table 2. The complete procedure of proposed method is shown in Figure 4. It is clear that for the color image from FG-NET database, just first row of the process will be done (due to having just color data). Figure 5 presents the proposed method’s flowchart to estimating age.

Table 2. Proposed edge detection filter

-1.2	-0.8	-0.6		1.2	0.8	0.6
0	0	0		0	0	0
1.2	0.8	0.6		-1.2	-0.8	-0.6
1.2	0	-1.2		-1.2	0	1.2
0.8	0	-0.8		-0.8	0	0.8
0.6	0	-0.6		-0.6	0	0.6

A New Way to Age Estimation for RGB-D Images, based on a New Face Detection and

Extraction Method for Depth Images

[Applying proposed edge detection filter

Fig. 4. Proposed age estimation method’s workflow

Fig.5. The proposed method’s flowchart to estimating age

• IV.    Test and Validation

In this section, proposed method is tested and validated by Mean Absolute Error (MAE) [31] factor on three RGB and RGB-D databases, which are FG-NET [8], Curtin face [32] and FEEDB [33]. The evaluations are done for all age ranges and in four ranks. In addition, these evaluations are done for male, female and both genders respectively.

Figure 6 shows some of the samples from FG-NET, Curtin face and FEEDB databases. Table 4 shows the evaluations results.

• A.    Databases

We used three face databases for evaluation, which one of them was a color database and other two were RGB-D type. The complete information about databases is listed in Table 3.

Fig.6. Some of the samples from FG-NET, Curtin face and FEEDB databases

Table 3. Used databases details

Database	FG-NET	Curtin face	FEEDB
Authors	FG-NET Group	Li, Billy YL, et al.	Szwoch, Mariusz
Samples	82	52	50
Sensor	Different types	Kinect V.1	Kinect V.1
Function	Facial expression recognition, Face recognition, Gender recognition, Age estimation	Facial expression recognition, Face recognition, Gender recognition, Age estimation	Facial expression recognition, Face recognition, Gender recognition, Age estimation
Data type	1002 color images	5000 RGB and Depth images	30 Frames videos
Resolution	Different resolutions and quality	RGB=640*480 Depth=640*480	RGB=640*480 Depth=640*480
Year	2004	2013	2013
Size	43 Mega byte	12 Giga byte	Around 100 Giga byte
Reference	[5]	[26]	[27]
Cite	365 till 2014 based on [28]	150 till 2017	21 till 2017

• B.    Mean Absolute Error (MAE)

Mean Absolute Error (MAE) [31] is a measure of difference between two continuous variables. Assume X and Y are variables of paired observations that express the same phenomenon. Examples of Y versus X include comparisons of predicted versus observed, subsequent time versus initial time, and one technique of measurement versus an alternative technique of measurement. Consider a scatter plot of n points, where point i has coordinates (xi, yi)... Mean Absolute Error (MAE) is the average vertical distance between each point and the Y=X line, which is also known as the One- to-One line. MAE is also the average horizontal distance between each point and the Y=X line. The mean absolute error is a common measure of forecast error in the timeseries analysis, [34] where the terms "mean absolute deviation" is sometimes used in confusion with the more standard definition of mean absolute deviation. Table 4 represents our method results. Additionally, Figure 7 shows Table 4 result’s in graphical chart form.

MAE =

Z "J y- - * 1 ₌Z^

nn

Table 4. Evaluation’s results (MAE for each rank and database)

Database	FG-NET	Curtin Face	FEEDB
Rank 1 (25 %)	Female: 11.2 Male: 9.67 Both:10.43	Female: 9.14 Male: 9.36 Both:9.25	Female: 14.83 Male: 19.11 Both: 16.97
Rank 2 (50 %)	Female: 12.8 Male: 12.19 Both: 12.49	Female: 10.10 Male: 9.82 Both: 9.96	Female: 16.98 Male: 21.24 Both: 19.11
Rank 3 (75 %)	Female: 15.9 Male: 13.22 Both: 14.56	Female: 11.99 Male: 11.67 Both: 11.83	Female: 18.37 Male: 25.44 Both: 21.90
Rank 4 (100 %)	Female: 19.40 Male: 21.7 Both: 20.55	Female: 13.20 Male: 12.18 Both: 12.69	Female: 26.95 Male: 31.03 Both: 28.99

Fig.7. Table 4 result’s in graphical chart form

• C.    Evaluations results and discussion

As it mentioned before, because of our singled image age estimation method, there is no need to have a different image in different ages from a subject, and using a single image could be satisfying enough. So due to high speed calculation and no need to learn, this method could be used in real time systems. But in such a condition, low accuracy could be experienced, which is normal. The results are gained from 25%, 50%, 75% and 100% of databases in male, female and all subjects’ forms. We select best samples from different ages of mentioned databases and divided them into four mentioned ranks, and used MAE factor to calculate error between real and acquired ages result. The final MAE value is the average of acquired MAE values for each Female, Male and both states. It is clear, the more rank; the less accuracy and best results with low errors are belonged to first rank and worst results belonged to last rank. As it is clear, the best result with lowest MAE is belonged to rank 1 of Curtin face database with MAE of 9.25 and worst one is belonged to rank 4 of FEEDB database with MAE of 28.99. Also because of high run time speed of our method, we could not to increase FG-Net database accuracy according to other researcher’s results, but in relation of our method’s speed, acquired results are completely satisfactory. Also it is mentionable that age estimation process on Curtin face and FEEDB database and even on RGB-D images is done for the first time in this paper.

• V.    Conclusion and Suggestion

In this paper, a new age estimation method based for RGB-D images is presented. Also a new method for face detection and extraction for depth images is presented, which is very robust and fast. The golden point in this paper is the high run time speed and having real time capability. Due to need for learning and even single image based ability of this system, run time speed against learning based system is 10 times better, but low accuracy is vivid. Because of the high speed of the algorithm, it is suggested to use this method on mobile phone apps. Moreover, it is suggested to use proposed method on other different color and depth database such as: MORPH, KDEF, EURECOM, VAP RGB-D and VAP RGBD-D-T.