Image Identification Based on Shape and Color Descriptors and Its Application to Ornamental Leaf

Oxygen is an essential part for all living things in the world. Plant plays an important role to produce oxygen and supply it for their sustainable life. The cycle between human and the plant is the interesting one. Carbon dioxide as the output of human respiratory is needed by plant for photosynthesis activity. Then, this activity is resulting oxygen which vital for human. According to this cycle, human supposed to preserve the plant to maintain availability of oxygen.

Based on International Union for Conservation of Nature and Natural Resources, the number of identified plant species in the world which consist of Mosses (M), Ferns and Allies (FA), Gymnosperms, Flowering Plants (FP), Red and Green Algae (RGA) is about 307.674 species [1].

Figure.1 Column charts number of identified plant species

On the other side, the approximate number of unidentified species is 86.429 species. It consists of Flowering Plants with 83.400 species, Ferns and Allies with 3.000 species, Mosses with 29 species [2]. Considering the highest number possessed by Flowering Plants, identification of the plants, which include also ornamental plant, has become a challenge for us.

As a recognition step of the unidentified species, this research has an emphasis on ornamental leaf which has medicinal function. Although mainly in Indonesia this plant is cultivated in front of the house; only few people know about its medicinal function. If this easiness and its medicinal function are taken into consideration, this plant has to be an initial treatment or option towards full chemical-based medicines.

Identification of leaf image is possibly done through identification of some leaf features, i.e. shape, venation, and color. Nevertheless, most of the researchers were using shape feature to identify the leaf. Shape is a substantial part to describe image content. Naturally, we lost one dimension of information while projecting of a real world three-dimensional object onto the twodimensional projection plane. This reason makes the shape features are often only represent projected object not real world object. Proper methods as the solution of real world object problems are needed.

The representation of shape can be separated into contour-based and region-based. Both representations can be further separated into structural and global [3]. Dyadic wavelet transformation and Zernike complex moments are representing global method of contourbased approach, and global method of region-based approach, respectively. We are expecting nice representation of shape using a combination from contour-based and region-based.

Formerly, many researchers had conducted image processing using Dyadic wavelet transformation. As in [4] and [5], they were applying this method for image denoising purpose. The results for both researchers were satisfying. In [7], they were adopting Zernike complex moments to classify benign and malignant masses on Mammogram images. The moments were divided into two groups. First group was low-order of the moments, and second was high-order of the moments. It was concluded that low-order magnitude of Zernike complex moment was preferred in comparison to high-order.

In leaf identification area, Park, Hwang, and Nam [11] had developed image retrieval system for leaf images collection. Utilization of leaf venation became their primary point to identify the leaf. Then, followed by the work that proposed by Wang et.al. [13], they classified leaf image with complicated background. Segmentation process of the leaf utilizing automatic marker for Watershed method, and Classification process was done by combining 7 Hu moments and 16 Zernike moments. This work inspires us to be further developed in segmentation and moment area. The color feature is also important to identify the leaf. In real life, frequently, the identification process is affected by aspect, i.e. lighting changed from the sun. We hope utilization of HSV can guarantee us to identify the leaf properly.

In this paper, we describe the ornamental leaf in the section 2, followed by the proposed method for image identification based on shape and color descriptors, flowchart of this research in the section 4, information of classification in the section 5, and the experiment and results as the final section.

• II.    ORNAMENTAL LEAF AS MEDICINAL LEAF

Ornamental leaf in this research is not general ornamental leaf. Besides the function of ornamental leaf as an ornament in an open space, also can be used as herbal medicine, which cure basic disease.

Image data set of ornamental leaf in this research was obtained from direct acquisition using a digital camera. This data set is taken based on tropical ornamental plant that usually cultivated in front of the house in Indonesia. This data set contains 8 classes with 15 images for each class. The classes are Bay ( syzygium polyanthum ), Cananga ( canagium odoratum, lamk ), Mangkokan ( nothopanax scutellarium merr. ), Jasmine ( jasminum sambac [soland] ), Cocor bebek ( kalanchoe pinnuta ), Vinca ( catharanthus roseus ), Kestuba ( euphorbia pulcherrima, willd ), Gardenia ( gardenia augusta, merr ). In order to avoid expensiveness of computation, size of the image is 256 x 256 pixels. The sample images of each class are presented as follows:

Figure.2 Sample Image of each class inside data set

The table.1 shows specific medicinal function of corresponding ornamental leaf. Most of them, ways to apply are the leaf is boiled together with water, and then apply as drinks, the other example is for skin cure, the leaf smears directly to the skin [14].

TABLE.1 MEDICINAL FUNCTION OF ORNAMENTAL LEAF

Name	Medicinal function
Bay	Diarrhea, scabies and itching
Cananga	Asthma
Mangkokan	Mastitis, skin injury, hair loss
Jasmine	Fever, head ache, sore eyes
Cocor Bebek	Ulcer, diarrhea, gastritis
Vinca	Diabetes, fever, burn
Kestuba	Bruise, irregular menstrual
Gardenia	Sprue, fever, constipation

• III.    IDENTIFICATION BASED ON SHAPE AND COLOR DESCRIPTORS

A Dyadic wavelet transform

The downsampling wavelet, which samples the scale and translation parameters, is often failing when deal with some assignments such as edge detection, features extraction, and image denoising [5]. Different with the downsampling wavelet, the dyadic wavelet samples only the scale parameter of a continuous wavelet transform, and does not samples the translation factor. In one side, it creates highly redundant signal representation, but the other side, since it has the shift-invariance ability, this method is a convincing candidate as a shape descriptor method.

Let L2(R) be the space of square integrable functions on real line R, and define the Fourier transform of the function гр E L2(R) by to p (w) = I p (t )e—lMtdt

-to

a_j+1 ^[n^] = У R^[k^]a_;-^[n + 2^j k^]; j = 0,1, .„,       ⁽⁹⁾

к

If there is exist A > 0 and В such that

Z ^to         2

|^т)| <В,

-to

d j ₊₁^[n^] = ^ g^[k^]C j ^[n + 2^jk^]; j = 0,1, .„,     ⁽¹⁰⁾

к then p (t) is called dyadic wavelet function. Dyadic wavelet transform of f (t) is defined using this p(t) by

^to , . 1 t

Wf ⁽u, 2^j) = I f ⁽t) -=p(-

J —to     V 2^j

- и

2j )

from (2), ^(O) = 0 must be satisfied, i.e., J_—to ^(t) dt = 0. In order to design the dyadic wavelet function, we need a scaling function ф⁽0 satisfying a two-scale relation

here a₀^[n^] is given by a₀^[n^] = J_—tof⁽t⁾ ф⁽1 — n) dt.

The (9) and (10) are dyadic decomposition formula for one-dimensional signals.

Shape descriptor from this method is represented by variance, standard deviation and centroid. The details are start from approximation, horizontal, vertical, and diagonal details will have these representation numbers. The following image shows the details:

ф⁽t) = У Я^[k^]V2ф⁽2t - k).

The scaling function ф (t ) is usually normalized as J — _toф⁽t⁾ dt = 1.

By (4), the Fourier transform of the scaling function resulting

ф(™) = V,^m;-),

Figure.3 Representation of dyadic wavelet transformed image

where Я denotes a discrete Fourier transform

fi(m) = У Я [k]e^—^

к

since ф⁽0) = 1, we can apply (5) and (6) to obtain Я⁽0) = V,. Using the scaling function and the wavelet filter g [k], we define a dyadic function by p ⁽t) = E_k g^[k^]V2 ф ⁽2t — k).

The Fourier transform of tpto,

B Zernike complex moments

The following steps are steps to calculate Zernike moments:

• 1.    Computation of radial polynomials

• 2.    Computation of Zernike basis functions

• 3.    Computation of Zernike moments through projecting the image onto its basis functions

First step is computation of radial polynomial and represented by the following equation:

# M=v, ^ wm),

(n—m)/2

(-1) ^k (n - k)!

R n^m(p) = У -7n+m      n—m \ ^p n-

“ k!L - k)! L -k)! ^k = ⁰         2             2

will be needed later.

Let us denote the discrete Fourier transform of the filters Я [к], Я^[к^], Я [к], and g[k], by R(m), Я (m), R(m) and g(m) respectively. We suppose that these Fourier transforms satisfy the below condition

R(m)Я * (m) + gT(m) g(m) =2, me [-л,л],     ⁽⁸⁾

where * denotes complex conjugation. This condition called a reconstruction condition.

Under condition (8), we have

n and m are representing the order and the repetition of azimuthal angle, respectively. The length of vector from original point to point ( x,y ) denoted by p.

From (11), complex two-dimensional Zernike basis functions are constructed by:

V n, _m⁽p, 0)=R m (p)^jme,     Ip I < 1              (12)

The following orthogonality condition is satisfied by the complex Zernike polynomials:

Г

2п

Г

J 0

V n*m (P, 0)V p,q (p, 0)pdpd0

f— n+1 0

; n = p,m = q ; otherwise

Sign * denotes complex conjugation. This orthogonality defines no redundancy inter-moments with different order and repetitions.

Here is the definition of the complex Zernike mo ments

6	0,2,4,6	4
7	1,3,5,7	4
8	0,2,4,6,8	5
9	1,3,5,7,9	5
10	0,2,4,6,8,10	6
Total moments 32

n + 1 2n 1

Z n,m = ------ I I f(P, 0)V n,m (P, 0)pdpd0

,          ⁿ 0     0

Image function, f(a, b) , for digital images can be replaced by summation operation.

The discrete Zernike moments for image with size N x N is defined as

7„ n,m

n+1

A n

N-1N-1

£ £ f(a,b)V ,, _m(a,b)

a=0b=0

N-1N-1

n+1

=       > > f⁽a,b)R m ⁽P ab )e^-,m6ab

^{A n} a=0 b=0

C HSV-based color histogram

The general use of color space RGB only defines color using a combination of primary color. Different with RGB, HSV color space defines color similar to how the human eye tends to perceive color. HSV color space is hexacone with center vertical axis is intensity [19]. Hue represents color type, and it consists of red at angle 0, green at 2π/3, blue 4π/3, and red again at 2π. Saturation is depth of color and measured as radial distance from the central axis to the outer surface.

In this research, we are not proposing to use global approach histogram where one whole image is measured by one histogram. We propose to use local approach, which one whole image is separated into 4 parts and measure the histograms. As an illustration, refer to Fig.4.

Where A_n is normalization factor and 0 < p_ab< 1. Transformed distance p_aband the phase 0_ab at the pixel position (a,b) are showed with following equations:

7(2a - N + 1) 2 + (2b - N + 1)        (16)

Pab =              N

(a)

0 ab

tan 1

/N - 1 - 2a

\2a - N + 1

The magnitude of Zernike moments from original image with rotated image will be equal.

This rotation-invariant ability became a reason to propose this method as another shape descriptors. This research only used low-order of Zernike complex moments because of computation inexpensive and not sensitive to the noise [7,15]. The following table shows the proposed low-order Zernike complex moments:

(b)

(c)

Figure.4 Local approach color histogram. (a) original image, (b) sliced images, (c) histogram of each slice

TABLE.2 LOW-ORDER ZERNIKE COMPLEX MOMENTS

Order n	Iteration m	Number of moments
3	1,3	2
4	0,2,4	3
5	1,3,5	3

IV. FLOWCHART

The following image is the flowchart of this research.

Figure.5 Flowchart of this research

Leaf data set is separated into training and testing sets, respectively. Image is segmented to remove unnecessary background object and obtain the foremost leaf for overlapping image. Next step is feature extraction using three methods. First, dyadic wavelet transformation, second is Zernike complex moment, both are as shape descriptors, and last is HSV-based color histogram. The data classification that extracted from these methods is classified utilizing SVM classifier. The classification process is involving two corresponding methods, classification towards the training set, and classification towards the testing set, respectively.

n If . j = 1

min - || w ||² + C

«Л.ь 2

subject to : y , ⁽w^Tx_j + b) - 1 + f > 0, f     (18)

> 0.

where w, C, f, b are the weight vectors, the penalty of misclassification or margin errors, the margin error, the bias, respectively.

In (18) can lead us to an efficient kernel method approach. A kernel method is an algorithm that depends on the data only through kernel function, which computes a dot product in some possibly high dimensional data. Using the function ф training vector, the input space x is mapped into higher dimensional space. K(x , ,X j ) = ф(x , ^)Tф(x j )is called kernel function. The degree of the polynomial kernel can control the flexibility of resulting classifier [17]. It will be appropriate with this research when we classify 8 classes of leaf. Polynomial kernel is shown in equation (19).

K(x , , X j ) = (yxjx j + r)^d, у >  0. (19)

Where у, r, d are kernel parameters, and i, j denote ^ th , jth vector in data set.

In this research, we propose to use Sequential Minimal Optimization (SMO). SMO act as efficient solver of the optimization problem in the training of support vector machines. SMO also solves the problems analytically by way of breaks the problems into a series of smallest possible problems.

B Various test aspects

In relation to measure the performance comprehensively, we are involving various test aspects. These aspects covered rotation, scaling, translation, lighting and perspective aspects. The following image is showed for visualization purpose:

V. CLASSIFICATION

A SVM classifier

SVM is a powerful tool for data classification. The indicators are the easiness to apply and impose Structural Risk Minimization (SRM). SRM made the SVM has strong ability in generalization of data. Its function is to minimize an upper bound on the expected risk. In principle, SVM learns to obtain optimal boundary with maximum margin that able to separate set of objects with different class of membership.

In order to achieve the maximum margin classifier, we have two options. Hard margin and soft margin are the options that totally depend on linearity of the data. Hard margin SVM is applicable to a linearly separable data set. However, often the data is not linearly separable. Soft margin SVM emerged as its solution [16,17]. The optimization problem for the soft margin SVM presented as below:

(a)

(c)

(b)

(d)

(e)                            (f)

Figure.6 Various test aspects. (a) original image, (b) rotated, (c) scaled, (d) translated, (e) lighting changed, (f) perspective changed.

• VI.    EXPERIMENT AND RESULT

A Segmentation results

Here are steps of segmentation for this data set:

• 1.    Create structuring element with the anchor point is the middle of the image

• 2.    Do a erosion operation to obtain 2 separated marker for next step

• 3.    Choose the biggest eroded shape as a target

• 4.    Set the markers; the biggest shape as foreground and smaller shape as background.

• 5.    Do a watershed segmentation using above markers

• 6.    Single foremost leaf is obtained

Figure.7 Segmentation results using Watershed method. Left images are original overlapping leaves and right images is

There are limitations of segmentation using this method. In this research, the position of anchor point is set manually which made this method cannot solve full arbitrary overlapped regions. The other is the number of overlapping leaves must be equal with two. However, through this method a real life problem such as segmentation besides of above points can be properly solved.

B Classification performances

• 1)    Decomposition level selection of dyadic wavelet transform

Selection of best decomposition level for this leaf image data set is based on three test data set. Original data set consists of leaf images without any changes, rotation data set consists of leaf images with rotation changes, and translation data set consists of leaf images with translation changes.

TABLE.3 CLASSIFICATION FOR LEVEL SELECTION PURPOSE (IN PERCENT)

Data set	level 1	level 2	level 3
Original	94.79	95.83	94.79
Rotation	92.59	85.19	88.89
Translation	100.00	100.00	100.00
Average	95.79	93.67	94.56

Level denotes decomposition level, and from the experiments were obtained that decomposition level 1 was the suitable level. This level will be used for further classification measurement. Besides the level, the table.3 has shown that Dyadic wavelet transformation is translation-invariant.

• 2)    Overall performances

Color distribution inside the data set is only limited to green. In this research, we only use a small number of bins. Hue and saturation bins were 4 bins and 6 bins. Therefore, one histogram has 24 bins and one whole image has 96 bins as a representation of color descriptor.

segmented single leaf

TABLE.4 CONTRIBUTION FROM COMBINATION OF EVERY DESCRIPTOR (IN PERCENT)

Method	Dyadic + Zernike	Dyadic + Histo	Zernike + Histo	Dyadic + Zernike + Histo
Training set + test set	81.25	62.5	81.25	93.75

TABLE.5 CLASSIFICATION PERFORMANCES TOWARD TEST ASPECTS (IN PERCENT)

Method	Rotation, Translation, Scaling (1)	(1)+Lighting changed (2)	(1)+Perspective changed (3)	Combination (1+2+3)	Average
Training set (a)	100	100	100	100	100
(a)+Test set	93.75	75	87.5	70.83	81.77

The number of training and testing sets for the experiments were 13 images and 2 images, respectively. Table.4 is presented as comparison for contribution from every combination of descriptors. Dyadic, Zernike, and Histo denote dyadic wavelet transformation, Zernike complex moments, and HSV-based color histogram, respectively. The best performance was gained from the combination of shape and color descriptors.

As shown in table.5, this system was able to identify the data of the entire training sets and created a proper model showed by number 100% of all training set field. The cooperation from shape descriptors that represented by Dyadic wavelet transformation and Zernike complex moments was resulting excellent results in the rotation, translation and scaling data set indicating by 93.75% of supplied test-set result. The changes in perspective view of two-dimensional projection plan were affected only to differentiation of the shape itself. Moreover, SVM classifier also gave a positive contribution separated the data through their polynomial kernel function.

Different with the other aspects, in the lighting aspect, although we found that color descriptor plays an important role as a representation of a leaf image, dominant part of the shape is being distorted because of this change. This issue could not properly be solved by this system. However, the overall results of previously mentioned test aspects were fine.

• VII.    CONCLUSION

It is found that combination between shape descriptors from Dyadic wavelet transformation and Zernike complex moments that emphasis on scaling, translation and rotation-invariant were resulting fine results. Because coupled with HSV-based color descriptor, this system could solve lighting invariance problem, though the result was not perfect. Future works from this current research are development of lifting scheme of Dyadic wavelet transformation, for the sake of easiness of calculation and memory efficient reasons, implements fast Zernike moments computation in connection with the recursive calculation of radial polynomial, and find another proper method to overcome arbitrary lighting change problems.