Leaf Vein Extraction Based on Gray-scale Morphology

Published Online December 2010 in MECS

Leaf is one of the important organs of the plant. Leaf features can play important roles both in plant species identification and plant taxonomy. Leaf vein is an integral part of the leaf. The type of the leaf vein is an important morphological characteristic of the leaf in botany[1]. Leaf vein should be extracted from a leaf in the image before identifying its type in computer based leaf feature extraction[2,3].

Several methods [4-8] have been proposed for leaf vein extraction. Some of them can not achieve satisfied results. Others are based on artificial neural network method, which makes the software systems developed based on these methods are only applicable to leaves of some specific kinds of plants. When applied to one new kind of plant, the software systems need to be adjusted by learning before they can turn out good result.

Leaf vein extraction is usually thought as a problem of image segmentation. The methods of image segmentation can be divided into two types[9]: region based method and edge based method. If the color difference is holistic between the object and its background in the image, the region based method can be used to achieve a good segmentation result. The key point of the region based method is how to select the right threshold. If the color difference is local between the object and its background, that is to say, there is color overlap in the whole object and the whole background, the edge based method is often used. The edge based method firstly segments the image into some regions and then identifies each region is the part of object or the background.

Usually the color difference is local between the leaf vein and its background in the image. So the region based method is not applicable to leaf vein extraction. As leaf vein is so thin that many noise will be introduced into the segmentation result when the region based

Figure 1. The procedure of leaf vein extraction method is applied. The region based method is not applicable to leaf vein extraction, too.

In this paper a new method is proposed for leaf vein extraction based on gray-scale morphology. It includes five steps: gray transformation, gray-scale morphology processing, image enhancement, image segmentation and processing on details, as Fig.1 shows.

The parts of the leaf vein which can be detected by human eyes in the image can be found with the new method. If the method is applied to a new kind of plant, only one parameter needs to be adjusted, which can be done by users easily. The method is also applicable for uneven illumination images.

The paper is organized as follows. After a short description of the research background in Section I, the new method is described via examples in Section II. Some experiments are shown in Section III. The discussions are given in Section IV.

• II.    METHOD

As mentioned above, Extracting leaf vein from the leaf in the image is usually regarded as a problem of image segmentation. As leaf vein is so thin and the color difference is local between the leaf vein and its background in the image, the common methods of image segmentation are inapplicable to the problem.

The problem of leaf vein extraction can be seen from other point of view. Leaf vein is thin and distributes on the leaf surface widely. It can be seen as the noises on the leaf surface. The methods of image noise detection may be usable for leaf vein extraction.

Mathematical morphology is an active and growing area of image processing and analysis. It is based on set theory and topology. It can be applied on binary image, gray image and color image. It has been widely used in image processing [10-12], such as image segmentation, image filtering, image enhancement etc.

The new method of leaf vein extraction is designed based on gray-scale morphology, which consists of gray transformation, gray-scale morphology processing, image enhancement, image segmentation and processing on details. Fig. 2 shows a complete example of the new method. Fig. 2(a) is a raw image of plant leaf, which is captured by a set of HP Scanjet 4070 photosmart scanner. Other images in Fig. 2 are the results of each step.

• A.    Gray transformation

The processing of gray transformation is to turn the color image to the gray image. The purpose of gray transformation is to reduce the amount of color data in the image so as to speed up the following processing. The color difference between the leaf vein and its background in the color image should be kept as well as possible in the gray image.

Equation (1) [13,14] is usually used for gray transformation in image processing..

Y = 0.299 R + 0.587 G + 0.114 B (1)

(g)

(a) The raw image (b) Gray transformation

(c) Gray-scale morphology processing (d) Image enhancement (e)Image segmentation (f) Processing on details (d) The leaf vein

Figure 2. An example of leaf vein extraction

In (1) R , G and B are the three components of a pixel color in RGB color space; Y is the transformation result, a gray value.

Equation (1) highlights the intensity information and discards most color information in the color image. It is applicable to image display but not to image segmentation, especially when there is great hue change in the image, as shown in Fig. 3.

There are two raw leaf images in Fig. 3(a), which are captured by a same scanner. It can be seen that there are a little hue changes on the surface of the left leaf and more on the surface of the right one.

Fig. 3(b) shows the results of gray transformation applied on the two raw images in Fig. 3(a) using (1). The left one in the result image is good but the right one is

(a) The raw images

(b) Results of the common method

(c) Results of the new method

Figure 3. Gray transformation

bad. The red leaf vein is conspicuous in the right raw image but vague in the result image. So (1) can not be used for leaf vein extraction.

Aiming at leaf vein extraction, a new method is designed for gray transformation, in which both hue and intensity information are used, as (2) shows.

Y = ((( H + 90)%3 60)/3 60 + 1 - V )/2 (2)

In (2) H and V are respectively hue and value (intensity) components of a pixel color in HSV color space.

Colors of leaves are usually green, yellow or red, while the colors on both sides of 0 degree line of hue are red in HSV color space. Considering the characteristic of leaves, the 0 degree line of hue is set as the original 270 degree line of hue in (2).

Fig. 3(c) shows the effectiveness of the new method. The two images in Fig. 3(c) are the results of gray transformation using (2) applied on the two raw leaf images in Fig. 3(a). It can be seen that the color difference between the leaf vein and its background in the color image has been reserved in the result image, whether in the left leaf image or the right one. The new method has a higher adaptability than the common method and it is adaptive to the gray transformation processing in leaf vein extraction.

Fig. 2(b) is the result of gray transformation using (2) applied on Fig. 2(a).

• B.    Gray-scale morphology processing

The processing of image segmentation can not be conducted on the gray image at present, for the gray difference between the leaf vein and its background is still local. There is gray overlap in the whole leaf vein and the whole background, which means the pixels with a same gray value may belong to the leaf vein or its background. If conducting image segmentation now, some parts of the leaf vein will be regarded as the background and be discarded, at the same time, some part of the background will be regarded as the leaf vein and be left in the result. The gray overlap in the whole leaf vein and the whole background should be eliminated before the image segmentation processing.

The purpose of gray-scale morphology processing is just to get rid of the gray overlap in the whole leaf vein and the whole background and make ready for image segmentation. The local difference between the leaf vein and its background should be well kept or even enhanced in the result image.

Mathematical morphology[9,13,14] provides two basic morphological operations: erosion and dilation. The different combinations of these two operations produce other operations of mathematical morphology, such as opening operation, closing operation, bot-hat transformation and top-hat transformation.

Let f be a gray image and b be a structuring element. The six operations of gray-scale morphology are defined respectively as follows.

• (1)    Erosion operation

( f ® b )( 5 , t ) = min{ f ( 5 + x , t + y ) - b ( x , y ) |

( 5 + x , t + y ) e D_f ,( x , y ) e D_b }

Applying erosion operation on a gray image will reduce the brightness of the image and the darker parts in the original image will expand in the result image.

• (2)    Dilation operation

( f Ф b )( 5 , t ) = max{ f ( 5 + x , t + y ) + b ( x , y ) |

( 5 + x , t + y ) e D f ,( x , y ) e D b }

Applying dilation operation on a gray image will enhance the brightness of the image and the brighter parts in the original image will expand in the result image.

• (3)    Opening operation

f o b = ( f ® b ) Ф b          (5)

Applying opening operation on a gray image can eliminate the little brighter regions in the image or weaken the brightness of these regions.

• (4)    Closing operation

f • b = ( f ® b ) ® b          (6)

Applying closing operation on a gray image can eliminate the little darker regions in the image or increase the brightness of these regions.

• (5)    Top-hat transformation

f - f o b (7)

Applying top-hat transformation on a gray image can extract the little brighter regions in the image or get the gray differences between these regions and their backgrounds.

• (6)    Bot-hat transformation

f • b ^- f (8)

Applying bot-hat transformation on a gray image can extract the little darker regions in the image or get the gray differences between these regions and their backgrounds.

In the gray image of a leaf, as shown in Fig. 2(b) and Fig. 3(c), the leaf vein is usually darker than its background. The bot-hat transformation can be used to get the gray difference between the leaf vein and its background.

The uneven illumination often occurs in the image in practical application. The bot-hat transformation can not avoid the disturbance of the bright spots. The top-hat transformation can be a useful supplement.

The complete gray-scale morphology processing is designed as (9) shows.

g = ( f • b - f ) - ( f - f o b ) ( 9 )

In (9) f is a gray image; g is the result of gray morphology processing, which is still a gray image.

As the values of the gray differences are small between most parts of the leaf vein and their backgrounds, which are usually less than 60, the result image g looks very dark and the effect of the gray morphology processing can not be seen from the image g . So the image g is transformed to its inverse image in order to watch the effect, as Fig. 2(c) shows.

It can be seen in Fig. 2(c) that the whole background of the leaf vein is almost white and the visual difference between leaf vein and its background is even more obvious than that in the raw image. we deem that there in no gray overlap in the whole leaf vein and the whole background now.

• C.    Image enhancement

As it is shown in Fig. 2(c), the gray difference between the leaf vein and its background is very small, whether it is in vision or value. In order to enlarge the value difference between the leaf vein and its background so as to make for image segmentation, the processing of image enhancement should be conducted on the result of gray-scale morphology processing.

The methods of image enhancement include linear intensity adjustment, Gamma correction, histogram equalization, decorrelation stretching, etc. Linear intensity adjustment has been chosen, as (10) shows.

g ( x , y ) = 255 g ( x , y) ^- minv maxv - minv

In (10) g(x,y) is the gray value of a pixel in image g , minv is the minimal gray value of all pixels and maxv is the maximum one.

Fig. 2(d) shows the result of linear intensity adjustment applied on the result of gray morphology processing. The leaf vein in the image is more obvious and the background is almost black, which state that the gray value difference between the leaf vein and its background has been enlarged successfully.

• D.    Image segmentation

The problem of leaf vein extraction is simplified to a image segmentation problem at present, whereas there are some questions need to be considered.

Although the background of the leaf vein seems to be black in the result image of image enhancement, as shown in Fig. 2(d), the gray values of pixels which constitute the background are not all zero, which are only very small. On the other hand, there are some parts of the leaf vein can not be detected by human eyes in the result image of image enhancement, for the gray values of the pixels constituting them are very small, too.

The accuracy of leaf vein extraction is very important to the following process. As the purpose of leaf vein extraction in our research is to make ready for identifying the type of leaf vein, the parts of leaf vein which can be detected by human eyes can just meet the requirement, for people also use them to identify the type of leaf vein in plant taxonomy.

So the goal of image segmentation processing is to extract the parts of leaf vein which can be detected by human eyes easily.

As there is no gray overlap in the leaf vein and its background now, the region based method of image segmentation can be applied now. There are many methods for threshold selection, such as iteration method, OSTU method and clustering method.

Considering requirements of the processing on the speed and effect, the OSTU method has been chosen to calculate the threshold for image segmentation. The image segmentation processing is shown as (11).

d ( x , y )

J 1 1 0

g ( x , y) ^ t g ⁽ x , y) <  t

In (11) g(x,y) is the gray value of a pixel in the result image of image enhancement, t is the threshold calculated by the OSTU method and d(x,y) is the binary value of the pixel.

Fig. 2(e) shows the result of image segmentation processing on Fig. 2(d). The leaf vein is white and the background is black. The leaf vein which can be detected by human eyes in Fig. 2(e) is more than that in Fig. 2(d).

• E.    Processing on details

The defect of threshold segmentation is that some isolated points and discontinuous lines often emerge in

(a) Part of the result image of image segmentation

(b) Part of the result image of linking the discontinuous lines

(c) Part of the result image of eliminating the isolated points

(d) Part of the color leaf vein

Figure 4. Partial image of processing on details the result image when the content of the image be segmented is relatively complicated. The leaf image is just so.

Fig. 4(a) shows a part of Fig. 2(e), which is displayed on 100% scale. It can be seen that there are some isolated points and discontinuous lines in the image. Some processing need to be conducted to eliminate the isolated points and link the discontinuous lines so as to get a more accuracy result.

• (1)    Linking the discontinuous lines

Search the black pixels in the result image of image segmentation. When a black pixel has been found, count the number of white pixels in its neighborhood. If the number is more than a specified number, the current pixel is regard as a pixel of the leaf vein and it should be turned to white, in other words, its gray value should be set as 1.

• (2)    Eliminating the isolated points

After the discontinuous lines have been linked, the isolated points can be eliminated using the opening operation of binary morphology, as (12) shows.

e = d ^' o b (12)

In (12) d’ is the result image of linking the discontinuous lines, b is also a structuring element and e is the result, which is still a binary image.

Fig. 2(f) shows the result of processing on details of Fig. 2(e). The procedure and effect of processing on details can be seen in Fig. 4(b) and Fig. 4(c) clearly .

The color information of the leaf vein can be got according to Fig. 2(a) and Fig. 2(f), as shown in Fig. 2(g). It can be seen that the parts of the leaf vein which can be detected by human eyes are extracted successfully.

Fig. 4(d) shows a part of Fig. 2(g).

Ш. EXPERIMENTS

Some experiments have been conducted on several images with the method, as Fig. 5 shows. There are five raw images of plant leaf in the Fig. 5(a), which are captured with a same HP scanner. The results are shown in Fig. 5(b).

It can be seen that the results of the front four images are acceptable and the result of the last one is not good like others.

One reason for the poor result of the last leaf is the colors of many parts of the leaf vein are too close to their background in the raw image and even people can not detect these parts of leaf vein with their eyes easily, which can be resolved by adjusting the threshold in image segmentation processing if needed. The other reason is the right part of the leaf in the raw image is too bright and the leaf vein in this region can not be seen with eyes. The method can just find the parts of leaf vein which can be detected with eyes.

W. DISCUSSIONS

• A.    About gray transformation

The hue and intensity information are used in gray transformation. Such a method of gray transformation is more applicable than the common one. It is appropriate not only for the image without great hue changes, just like the common method, but also for the image with a wide rang of hue changes. However, it does not mean the method is applicable to all kinds of image, for example, an image with all hue in HSV color space. Fortunately, this seldom occurs on the surface of a life body in the natural world. So the method has a broad application fields.

• B.    About gray-scale morphology processing

The bot-hat transformation and the top-hat transformation are combined in gray morphology processing. The bot-hat transformation is used to enhance the dark details in the gray image and the top-hat transformation is used to eliminate the disturbances of the bright spots. The top-hat operation can be omitted if there is no uneven illumination in the raw image, while which seldom happens in practice.

The b structuring element is very important in gray morphology processing, including its shape and size.

The shape of the b structuring element can be circular, square, cross-shaped etc. It should be chosen or designed according to the shape and the extending orientation of the object. As the leaf vein is linear and the orientation of the leaf vein varying with the position on the leaf surface, square is chosen for the b structuring element in the processing.

The width of b structuring element should be selected in accordance with the width of the object. If it is too wide, the processing speed will decrease and the result may be affected. The width of the b structuring element should be no less than the maximal width of the leaf vein in the image. In our experiments the width of the b structuring element is set as 7.

The width of b structuring element is the only parameter in our method which need to be revised when the plant species or the resolution ratio of the image changes. Fortunately, it is just a simple work. It is easy for users to finish it according to the processing result by themselves.

• C.    About image enhancement

The processing of image enhancement is to prepare for image segmentation. If all of the leaf vein or only the primary leaf vein need to be extracted in other application, other methods of image enhancement should be selected, such as Gamma correction and logarithm method.

• D.    About image segmentation

OSTU method has been applied in the processing of image segmentation. It is propitious to extracting the parts of leaf vein which can be detected with eyes easily in the raw image. If a refined result desired in some application, such as leaf vein grading, the clustering method may be a good choice.

• E.    About processing on details

The specified number in linking the discontinuous lines is related to the width of b structuring element in gray morphology processing. If the width is n , the specified number is (n-1)/2 .

The width of b structuring element in eliminating the isolated points is also related to the width of b structuring element in gray morphology processing. If the latter is n , the former should be n-2 .

• F.    About the result

It can be seen in many result images that the leaf vein is discontinuous in some locations. The main reason is that the color of the leaf vein is so close to its background at these locations in the original image. Such results usually will not do harm to the type identification of leaf vein.

• G.    About the whole method

The method can be used to extract the parts of leaf vein which can be detected with people eyes easily in the image. If a more precise or coarser result needed, the processing of image enhancement and image segmented need to be adjusted.

Figure 5. Pome experiments

The main factor which affect the processing speed of the method is the width of the b structuring element. The smaller the width is, the faster the speed is. The width of the b structuring element also affect the result. So it should be no less than the maximal width of the leaf vein in the image.

V. CONCLUSIONS

In this paper a new method for leaf vein extraction has been put forward, which is based on gray-scale morphology. The main idea of the method is to look upon the leaf vein as the noise on the leaf surface and adopt the method of noise detection to extract the leaf vein. The experiments show the method is feasible. Compared with the former research, the new method is more practical.

The basic idea of the method is also applicable to other linear objects extraction, such as cardiovascular, cracks on road, etc. When applied on other objects, the method should be adjusted according to at least three factors:

• (1)    the color difference between the object and its background, which determines how to design the mathematical morphology operation;

• (2)    the shape and the extending orientation of the object, which determines the shape of the b structuring element;

• (3)    the width of the object, which determines the width of the b structuring element.