Online trial room based on human body shape detection

Published Online February 2019 in MECS

This research has proposed a model by which the size of the t-shirt can be measured from a 2D image. Online trial room based on body shape recognition will be a unique system as there is no other similar system is available currently in Bangladesh. There exists a few works or research paper on this idea. A controlled surrounding is needed for taking photos which is possible for each user to provide by using our proposed web-based application.

With the introduction of smartphones and tablets, we can enjoy online shopping anytime while sitting in any part of the world. Online shopping has certainly replaced the traditional way of shopping for daily goods and clothing. When we choose online shopping, we get the benefit of credibility. Today, almost every online store offers cash on delivery, free shipping, and discounted prices. These online shopping stores eliminate the hassles of parking, getting stuck in traffic jams and standing in long queues for billing. They have also benefited those people who always complain about the shortage of time. This is the reason, a majority of the people have turned to online shopping. Made-to-Measure System (m-t-m) allows customers to create better fitting garments which are suitable for their bodies [1].

• II.    Research Background

There have been some related works in this field specially T-shirt size detection. Four types of people (tall fat, short fat, tall thin and short thin) have been detected from 2D images by using a proposed method [2]. This recognition has been done by assuming some conditions: the subject must be in a standing position and subject must consist of five segments: two hands, two legs, and torso. The system has extracted the size and the advantage of them from the image. It also has extracted some features of the human body such as length and width. They have used Canny Edge Detection Algorithm in their preprocessing stage and proposed a Semiautomatic Feature Extraction Algorithm.

Feature points on the outline of the human body have been detected by using a proposed systematic method from 2D frontal and side images [3]. An efficient shape coding algorithm has been used to represent the human body contour from binary images. They have identified the feature points by evaluating the difference between the coding sequences. They have provided methods for the automatic feature extraction from full body outlines and for the estimation of 2D human models from measurements extracted from the features.

The pictorial structures (PS) method has been developed and applied for human pose estimation [4]. The partial body masks have been evaluated directly at each level of the growing process via shape matching with exemplars, without considering the formation of the hypotheses. Multiple image segmentations have been included at each of the levels of the growing/parsing, to augment existing hypotheses or to introduce ones. A shape parsing method which uses a bottom-up fashion has been presented for the construction and verification of shapes. This research has been done in a different way than the traditional bottom-up parsing by scoring shapes against a set of exemplars using IDSCs.

A model has been proposed to analyze women body scan data (body shape, posture, and weight) and also an expert system has been developed for realigning sizing and pattern development for markets [5]. A body scanner has been used for providing body shapes. BMI (Body Mass Index) has been used for determining body categories such as underweight, normal etc. Reliability checking has been done and for automatic body shape analysis, the software has been developed based on the body shapes identified by experts.

An approach has been proposed to detect viewpoint invariant human by applying example-based body-part detectors [6]. For the head, arms and legs detection 10 sub-classifiers have been integrated. For human action recognition (HAR), stochastic behaviors of body-parts have been utilized. Human body parts have been detected and then the changes of those body parts have been learned for action recognition. Approach to recognize actions have been extended in this research, based on component-wise hidden Markov models.

A new model type for 2D human body has been proposed in [7] which retain the standard part-based structure of classical pictorial structures models. With the expressive power of a detailed 3D model and the computational benefits of a simple 2D part-based model, a new “contour person (CP)” model has been defined. The CP model has also captured the non-rigid deformations of the body that result from articulation. Their 2D model was view-based and there they had only shown examples for frontal bodies.

For the garment industry, a technique of virtual human modeling has been developed with the requirement of two orthogonal views of a human body in two photographs [8]. To obtain the human body contours an efficient segmentation method has been applied to these two photographs. A template-based feature extraction algorithm has been introduced to determine the feature points on the human contours. The Mumford–Shah segmentation technology and the level set formulation have been used for the derivation of the segmentation algorithm, and it has been accelerated by applying multipyramid levels.

The methodology of automatic computer-based measurement has been described for taking anthropometric data [9]. For designing a complete and fully automatic 2D anthropometric system a summarization of needed equipment has been presented here. A new technique for automatic detection of several spatial human body parts has also been proposed. To outline how landmark localization without any other manual steps could be done and to introduce algorithms to solve this problem in 2D non-contact anthropometry system, a solution has been tried out.

A new approach for defining body shapes of the female has been proposed based on the 2D images analysis [10]. For human interpretation, the improvement of pictorial information has been the main focus of this research. For calculating widths of different body parts, such as waist, hip, bust etc. from 2D images, pixel numbers have been used. The prerequisite of a calibration process or a special camera for capturing images has been eliminated. Plain and dark color dresses along with a light and the plain color background are the only requirement here. A webbased application has been developed for defining body shapes from 2D images and also for collecting additional information such as region, city, and age etc. for building a log file.

A problem of detecting 3D human shape under clothing has been defined and a solution has been presented that holds a learned model of body shape [11]. The relative pose-independence of body shape model has been exploited to integrate constraints from multiple poses and this has been done by solving for the body pose at each time instant and a single 3D shape across all time instants. For parts of the unclothed body, a skin detector has also been integrated to provide tight constraints on 3D shape. The idea that: based on the shape of the body gender could be reliably classified has been also showed in this research.

A system has been described for person recognition in indoor images, which can learn from examples [12]. The shape and color-based features have been used to represent images of full-body persons. Different combinations of SVM have been used to carry out the recognition process. The exploration and comparison between k-Nearest Neighbors classifiers and different types of multi-class strategies based on SVMs have been executed. The system works in real time and the experimental results have shown a high recognition rate. It has also indicated the strength of SVM-based classifiers to improve both generalization and run-time performance.

Online fashion shopping has become one of the great sources to purchase clothes. In the very near future, it might be the main source. But buying clothes without a trial room is the main challenge of this field. Our main goal is to overcome this problem that is presenting a virtual system for the online trial room. Our system allows anyone at home to measure their body size from

2D images and use the results to comfortably shop online. This minimizes the risk of buying unfit clothe. Our system can make the online shopping experience more satisfying. We have measured the shoulder, upper chest, lower chest, waist and hip size from images. Finally, we have used a system to calculate the rest of the measurements and found the suitable cloth size for the customers.

• III.    Proposed Method

To implement the system we have pre-processed the image to improve the image and suppress unwanted distortion. A controlled domain is needed for data collection which includes a light and plain background color. In this system, we have used Canny Edge Detection Operator to pre-process the image. Different types of algorithms have been used for noise removal. After pre-processing the image, we have extracted some features by using Kollman’s distribution algorithm, so that the measurement of the t-shirt as well as the body can be measured. Then we have compared the result to find a better system.

• A.    Framework

At first, we have gathered data by using a conventional camera. Then after some pre-processing, we have detected some features. At last, we have found the size of the t-shirt by using our web system. The framework of this method is shown in Fig. 1.

• B.    Data Acquisition

In this system, we have used the conventional camera to take an image. For this system, the image must be taken in an environment with some restrictions. We have taken two images: first, background image and then original body image in front of the same background. Some conditions have been provided for capturing images in our proposed model: both background and original body images should be captured in a fixed length and height from the background; background must be light colored and both images should be captured in front of the same background.

We have taken two sample images. First one is only background image (Fig. 2) and the second one is original body image (Fig. 3) taken in front of the same background. Both images have been captured in a fixed distance maintaining a fixed height from the camera lens and background. A light colored background is used so that the system can easily detect the edge of the body and measure the required features of our proposed system.

Fig.1. A Block Diagram of “Online Trial Room”

Fig.3. Body image

Fig.2. Background Image

• C.    Pre-processing

We have pre-processed our data to overcome unwanted distortions and to enhance some image features important for our further processing.

• 1)    Color Space Conversion

There are a variety of color spaces, such as RGB, CMY, HSV, HIS, GREYSCALE, BINARY. In this system firstly we have converted RGB color space to HSV color system for shadow removal. Then we have converted it back to RGB color space. Then we have converted it to the Grey Scale color space. The color space conversion Process is shown in Fig. 4.

Fig.4. Color Space Conversion. (a) HSV image; (b) RGB image; (c) Grey Scale image

• 2)    Enhancement and Noise Reduction

We have used Normalization to enhance the images. Normalization (aka. contrast stretching or histogram stretching) has been used to change the range of pixel intensity values. We have used Median blur and Gaussian blur in this system for the noise reduction. Steps of image enhancement and noise removal are shown in Fig. 5.

Normalize the image for enhancement

I

Apply Median Blur to reduce big salt and pepper size noise in the image ;

Apply Gaussian Blur to reduce small noise

Fig.5. Steps of image enhancement and noise removal

• 3)    Edge Detection

There are various edge detection algorithms: Sobel, Canny, Prewitt, Roberts and Fuzzy logic method. In this system, we have used the Canny Edge Detection Algorithm to detect the edge of the image. Edge detection has been used for finding the boundaries of objects within images. Though canny edge detection algorithm is a bit time consuming compared to others, it has been used here because it gives a good localization, response and immune to a noisy environment. The edge detection of the human body is shown in Fig. 6.

Fig.6. Edge Detection

• 4)    Dilation

We have used a 7×7 structuring element for our dilation process. Dilation has been used for background shrinking and foreground thickening.

• 5)    Background Removal

We have used background removal technique to make our object (human body) ready for body contour selection in the next step. The removed background stage of our input is shown in Fig. 7.

• 6)    Selecting Body Contours

We have selected body contours for better shape analysis and object detection and recognition. Body contour detection is shown in Fig. 8.

Fig.7. Background Removal

Fig.8. Body Part Detection

• D.    Feature Detection

In this system, neck, shoulder, upper chest, lower chest, waist, and hip sizes are the main features of calculating t-shirt size and body shape measurement. We have completed our feature detection process in two steps: scaling of body parts and measurement of body parts.

• 1)    Scaling of Body Parts

The scanning starts from a starting point which has been chosen at the bottom of the image. We have started at the starting point and continued the scanning up to the top of the image. We have scaled the image into 19 sections to get the required features. This scaling of body parts has been shown in Fig. 9.

• 2)    Measurement of Body Parts

We only needed to measure neck, shoulder, upper chest, lower chest, waist and hip size in inches. For this reason, we have converted the pixel values into inches by dividing inch with the pixel. Then we have subtracted the minimum pixel value from the maximum pixel value on the selected line to get the size of the body parts. The measurement of the body parts has been shown in Fig. 10.

Fig.10. Measuring the parts of the body

• E.    System Output

We have used flask framework to create our web page. Flask is a small and powerful web framework for Python. A window appears in the browser (Fig. 11) when the system starts. After clicking the "choose file" button an image can be chosen from any folder of the computer or laptop. After selecting an image one has to click the "upload" button to find the measurement of clothes. After clicking the “upload” button the final output of the system has been shown in the system window (Fig. 12).

Fig.11. System User Interface

Fig.9. Scaling of Body Parts

Fig.12. Final Output of our system

• IV.    Result Analysis and Discussion

We did not find any existing dataset for our proposed system. So we have created our own dataset by capturing images of random people. But for our proposed system we need a fixed and constant environment for capturing pictures. We have visited some renowned shop's sites in our country for the measurements of clothes such as Ecstacy, Le Reve, Arong, Artisan etc. In this proposed system we have chosen Artisan brand for the standard measurement of t-shirt for men [13]. This standard measurement is shown in Table1.

Table 1. The standard measurement of t-shirt for men

Length (in inch)	Shoulder (in inch)	Size
27	16	Small
28	17	Medium
29	18	Large
30	19	Extra Large

Fig.14. Sample2 Output

• A.    Comparing Our System Result with Original Measurement

There are few sample data on which we have applied our system. We have managed to collect 20 images of different volunteers and among them, 12 images have shown correct results. We have taken the measurement of clothes manually by a tape with the help of wikihow [14].

The difference between the result of our system and original measurements have been shown in Table 2 (for sample1 and sample2). All values in the table are in inch. The system value outputs of sample1 and sample2 have been shown in Fig. 13 and Fig. 14 respectively.

Table 2. Comparison between the result of our system and original measurement for sample1 and sample2

Body Parts	Sample1		Sample2
	Original Value	System value	Original Value	System value
Neck	6	4.915	6	4.596
Shoulder	17	16.714	17	14.488
Upper chest	22	20.842	22	19.459
Upper waist	18	13.289	16	13.289
Lower waist	20	14.043	18	14.66
Length	29	28.4	29	28.1

Fig.13. Sample1 Output

• B.    Error Analysis

We have used Root Mean Square Error (RMSE) to find the deviation between the original value and the system value. Root Mean Square Error (RMSE) also known as Root Mean Square Deviation measures how much error there is between two data sets. The formula of RMSE has been shown in equation (1).

RMSE =

n

• 2 0 ,    - P i ) 2 i = 1

n

Here,

O = Original value

P = Predicted value of our system

• n = Total number of sample data

We have calculated the RMSE value for the neck, shoulder, upper waist, lower waist, and length. Table 3 shows the deviation values and RMSE values for different body parts. All values in the table are in inch. Here sample1, sample2, ….sample12 is represented as S1, S2, …. S12 respectively.

Table 3. Deviation values and RMSE values for different body parts

Deviation Values	Body Parts
	Neck	Shoulder	Upper waist	Lower waist	Length
S1	1.1	.3	4.8	3.9	.6
S2	1.5	2.6	2.8	1.9	.9
S3	.43	.6	4.8	4	1.3
S4	1	1.7	4.4	3.7	.9
S5	.8	1.9	4.3	4.3	.7
S6	.4	.7	4.9	4.4	.6
S7	.7	1.6	4.7	3.7	1
S8	.8	1	4.5	3.6	1.1
S9	.5	.7	3.1	3	.8
S10	.4	1	5	4.2	.9
S11	.2	1.3	5.3	4.2	.7
S12	.9	2.3	4.9	4.3	1.1
RMSE	.808	1.478	4.454	3.83	.907

Here we can see that the RMSE values of upper waist and lower waist are much more than other features (Fig. 15). That means the deviation between the original values and the values of our proposed system for upper and lower waist is much greater, as our proposed system could not find the exact values. RMSE values of upper and lower waist are much higher because of the round shape of clothes.

RMSE value Vs Features of our system

Fig.15. Chart of RMSE value Vs Features of our system

• C.    Comparative analysis of our work with existing ones

Although to the best of our knowledge no work on online trial room has been found in Bangladesh, we have compared our work with the existing similar works where body shape detection has been used. The comparison of our work with existing works has been shown in Table 4.

From Table 4, we can see that most of the existing works used their own dataset whereas we used our developed one. So, it is really difficult to compare our work with the existing works. Though Result comparison is a very difficult issue as there is no fixed parameter of the existing works but we tried our best to compare our result in terms of RMSE Value and achieved good results (i.e. less error rate) compared to the existing works. With an enriched dataset, we hope to achieve a better result in the future.

Table 4. Comparison of Our Work with Existing Works

Work	Dataset	Result
Five Body Parts (torso, two hands and two legs) Detection [2]	Own Developed	No numerical comparison has shown
Recognition and Parsing of the Human Body [4]	Baseball Dataset	Segmentation Scoring: a vast majority of average errors <= 20 pixels Joint Position Scoring: Full body overlap scores: 0.44 to 0.84 Average joint position errors: 10.20 to 37.96
View-Invariant Human-Body Detection [6]	KTH’s Database and HERMES Indoor Sequence Dataset	KTH Database (according to ROC curve): Leg detection > 90% Head detection < Leg Detection Arm detection: not very high HERMES Dataset (according to ROC curve): Leg, Head and Arm detection < KTH database detection
Virtual Human Modeling from Photographs for the Garment Industry [8]	Own Dataset	The difference between Tailor and Detected Measurement (unit: CM): Height: 0.72, Arm Length: 1.52, Chest: 0.05, Lower Chest: 0.07, Lower Waist: 2.23, Below-ankle Girth: 1.33 etc.
Human Body Modelling [9]	Own Dataset (9 males and 2 females)	Average error [Comparison between tape measurement and developed 2D system (unit: CM)]: Height: 2.21, Neck Circumference: 3.08, Chest Circumference: 3.75, Waist Circumference: 2.83 and Hips Circumference: 3.175
Full-body person recognition system [12]	Own Dataset	People recognition rates vary from 45.9% to 98.3%, depending on several parameters and method used
Human Body Shape Detection for Online Trial Room	Own Developed	Root Mean Square Error (RMSE): Neck: 0.808, Shoulder: 1.478, Upper Waist: 4.454, Lower Waist: 3.83 and Length: 0.907

• D.    Limitations

To get a proper output we need to follow some conditions. A user has to face some restrictions for capturing a picture: keep the same colour at the background as more than one colour can create a problem for edge detection and has to capture the picture of the user from a fixed distance maintaining a fixed height from the camera lens. Low light can also be problematic for our proposed system. Some problematic situations are shown in Fig. 16 and Fig. 17.

Fig.17. Same background and dress colour

• V. Conclusion

Now a day apparel industries are using computers, especially for manufacturing and process control functions. Online trial room based on body shape recognition is a unique system proposal as there is no other similar system available currently in Bangladesh. As we had inadequate data on which we could apply our system it was not actually possible to find the highest accuracy of our proposed system.

In future we have to create a sufficient dataset following the restriction strictly. Then we will apply our proposed system to the dataset to test and find the accuracy of our system. In this proposed system, we have extracted five features, but have used only two features for measuring the size of t-shirt. In the future, we will try to extract more features so that we can measure not only t-shirts but also every type of clothes for men and women. We will try to add machine learning process in our proposed model so that the system will be familiar with the environment and accurate measurement will be found out. If the result of our proposed system is quite accurate then we will try to integrate the system into the web, so that people can buy their required size clothes without going for shopping.