Real-time Deep Learning Based Mobile Application for Detecting Edible Fungi: Mushapp

Published Online on October 8, 2024 by MECS Press

Mushroom is one of the fungal species with the most powerful nutrients on the plant [1]. Mushroom is a good source of vitamin B and an important fungus containing high amounts of protein compared to all other vegetables. It helps prevent cancer, is useful in weight loss and increases human immune power. On the other hand, some mushrooms are poisonous and can be dangerous when we eat them. Therefore, it is an important task to distinguish between edible and poisonous mushrooms [2]. Mushroom poisoning occurs as a result of mushrooms accidentally eaten by humans or consumed unconsciously when collected. There may be many causes of mushroom poisoning and different mushroom species may contain different toxic substances [3]. Some mushrooms contain natural toxins. These toxins vary depending on the type of mushroom and the growing environment [4]. Often, it is difficult to accurately determine whether the collected mushrooms are dangerous or not [5], and people unconsciously eat poisonous mushrooms and face serious health problems or even death [6]. Mixing harmless mushrooms with toxic mushrooms also causes mushroom poisoning. Some mushroom species that look similar may be very similar to toxic ones, which causes confusion in the choice of consumable mushrooms. Worldwide, cases of mushroom poisoning vary according to different regions. The frequency of these cases usually varies depending on the mushroom picking season and consumption habits. In addition, the prevalence of poisonous mushroom species in some regions may increase this risk. Mushroom species with different characteristics can grow in different parts of the world. For this reason, it is very difficult to collect a data set for mushroom species growing all over the world. This study was narrowed down to the species growing and collected in Turkey.

The aim of this study is to increase the level of awareness of the society about mushrooms and to minimise the occurrence of negative cases such as poisoning. In order to achieve this, we have designed a deep learning based mobile application that detects the mushroom species from the camera image and shows the information to the users whether it is poisonous or not. With the development of this application, it is aimed to prevent the unconscious consumption of poisonous mushrooms in places where mushroom picking is common and to take preventive measures against poisoning cases.

Deep learning is a sub-branch of machine learning that enables learning and prediction on large datasets using artificial neural networks that mimic the way the human brain works. It is particularly effective for complex tasks such as image recognition, speech recognition and natural language processing.

In this study, we tried to maximise the performance and success rate of the system by using different image algorithms and a good data set. Firstly, a data set was created for model training. Thousands of images of different mushroom species were obtained, and the images were pre-processed in order to make them most suitable for the use of the artificial intelligence model. In the pre-processing stage, Adobe Photoshop software was used for visual operations such as sizing the images and cleaning the backgrounds. Python programming language and TensorFlow and Keras deep learning libraries were used in the design of the artificial intelligence model. The mobile application was developed in Android Studio IDE environment. Java programming language was preferred while developing the application.

TensorFlow is an open-source machine learning library developed by Google. It is used to build and train deep learning models. TensorFlow provides both high-performance computing and flexibility, which makes it popular among researchers and developers.

Keras is a high-level artificial neural network library that simplifies building and training deep learning models. It supports various backends such as TensorFlow, Microsoft Cognitive Toolkit (CNTK) and Theano. Keras offers an easy-to-use API and is ideal for rapid prototyping.

Android Studio is the official Integrated Development Environment (IDE) provided by Google for developing Android applications. It supports app development in languages such as Java, Kotlin and C++. Android Studio includes many tools and features such as code editing, debugging, performance analysis and application testing.

In the second part of the study, similar studies in the literature are analysed. In the third section, the preparation and augmentation processes of the data set are explained in detail. In the fourth section, deep learning models and configuration technical details are shown. In the fifth section of the study, numerical values obtained from model training and their reasons are analysed. In the last part of the study, the conclusions drawn from the whole study and the experiences obtained are presented.

2.    Related Works

Researchers have carried out studies on mushroom species focusing on different aspects. Some of these studies have focused on mushroom species identification, some on the separation of poisonous and non-poisonous species [7, 8], some on mushroom quality [9-11] and optimisation of mushroom collection methods [12, 13].

Akın et al. developed a deep learning based mobile application to increase the awareness level of people in mushroom consumption. In the study, VGG16 model, one of the deep learning methods, was used for image classification. Some statistical analyses were performed on the research findings and the accuracy rate of the application was calculated as 81.75% [14]. In their study, Maurya and Singh focus on developing a method for mushroom classification using texture feature based on machine learning approach. The performance of the proposed approach is 76.6% using SVM classifier which is found to be better than other classifiers such as KNN, Logistic Regression, Linear Discriminant, Decision Tree and Ensemble classifiers. Rahman et al. developed a system for a remotely controlled IoT-based mushroom farm. They also used machine learning methods to detect poisonous mushrooms among the mushrooms grown. The proposed machine learning model uses an ensemble model consisting of six classifiers: Decision Tree (DT), Logistic Regression (LR), K-nearest neighbour (KNN), Support Vector Machine (SVM), Naïve Bayes (NB) and Random Forest (RF). From the combinatorial classification of these algorithms, classification was achieved with 100% accuracy [15]. Tongcham et al. designed a machine learning algorithm to classify oyster mushroom eggs and used filtering to reduce noise. Five different machine learning methods were tested, and the most successful results belong to the DNN algorithm with a successful classification value of 98.8% [16]. Chumuang et al. proposed a method that takes into account physical characteristics and will be a solution to the mushroom classification problem. Using Machine Learning (ML), they performed different experiments on a mushroom dataset containing a total of 800 samples consisting of 22 attributes and divided into two classes as toxic and non-toxic. Several ML results such as Naive Bayes Updatable, Naive Bayes, Naive Bayes, SGD Text, LWL and K-Nearest Neighbour (k-NN) were compared to ensure the best success. The results showed that K-NN gave the highest classification accuracy rate of 100% [17]. Tutuncu et al. propose a method using machine learning algorithms to distinguish between edible and poisonous mushrooms. The researchers used Mushroom ready dataset in their study. In their experiment, Naive Bayes, Decision Tree, Support Vector Machine and AdaBoost algorithms achieved 90.99%, 98.82%, 99.98% and 100% classification success respectively [18]. Zahan et al. studied classification using different image-based machine learning algorithms on 8190 poisonous and non-poisonous mushroom images. As a result of their experiments, they obtained the highest classification success rate of 88.40% by applying the contrast limited adaptive histogram equalisation (CLAHE) method to Inception v3 [1]. Kiss and Czuni investigated deep learning methods on classifying mushroom images of 106 different species. There are 29,100 images in the data set they used for training. Transfer learning was performed in deep learning algorithms, and they obtained the highest 92.6% successful classification value [19]. Ketwongsa et al. used convolutional neural network (CNN) and regional convolutional neural network (R-CNN) to classify five poisonous and edible mushroom species to solve the difficulties in distinguishing edible and poisonous mushrooms. In this research, a method for classification of edible and poisonous mushrooms is proposed and the test time and accuracy of three pre-trained models, AlexNet, ResNet-50 and Google Net, are compared. In mushroom classification experiments using CNN and R-CNN, the proposed model showed accuracy levels of 98.50% and 95.50%, respectively [20].

In addition to the existing studies on mushroom species identification and poisonous and non-poisonous discrimination, deep learning and machine learning algorithms are also utilised to facilitate the work of mushroom growers and increase production efficiency. Fang et al. propose a highly efficient channel pruning mechanism using the YOLOX deep learning method for determining and grading mushroom quality [21]. In Table 1, the criteria in this study are presented comparatively in terms of the data set, study years, method technique used by the existing studies.

Related Works	Year	Dataset	Technique & Algorithm	Results
[1] Zahan et.al.	2021	8190 images	InceptionV3, VGG16 Resnet50	Acc.: %88,40
[2] Maurya & Singh	2020	250 images	KNN, LR DA, DT, EL, SVM	Acc.: %76.6
[14]Akın et.al.	2023	220 images	VGG16	Acc.: %81.75
[15] Rahman et.al.	2022	8125 images	Ensemble Model	Acc.: %100
[16] Tongcham et.al.	2020	-	SVM,DT,NCC,DNN,KNN	Acc.: %98.8
[17] Chumuang et.al.	2020	800 images	Naive Bayes Updateable, Naive Bayes, SGD Text, LWL, KNN	Acc.: %100
[18] Tutuncu et.al.	2022	8125 images	Naive Bayes, DT, SVM, AdaBoost	Acc.: %100
[19] Kiss and Czuni	2021	29100 images	Transfer Learning	Acc. : %92.6
[20] Ketwongsa et.al.	2022	623 images	AlexNet, ResNet-50, Google Net, CNN, R-CNN	Acc.: %98.5
This Work	2024	5150 images	DenseNet121, ResNet50, VGG16, CNN, InceptionV3	Acc.: %99.8

3. Data Set and Data Augmentation Method

A software has been prepared for the collection of the data set of mushroom images that are openly available. In order to use this software, the address where the data set is located is entered in the relevant field and then the target folder to be downloaded is shown. When the download button is pressed, all images are converted into the format to be used in model training and downloaded to the desired destination by numbering. The visual of the data collection software developed for this purpose is shown in Figure 1.

Fig.1. Images of the data collection tool software

The data set was prepared in such a way that 20 different classes were formed with photographs taken from the internet. All photographs were divided into 2 classes as "training" and "test". There are approximately 150 photographs of each species in the training class and 50 photographs in the test class. There are subfolders containing photographs of each mushroom species under the training and test folders.

In our project study, 5150 mushroom photographs of 20 different mushroom species were used. These mushrooms are Agulu Mushroom, Honey Mushroom, Beefsteak Mushroom, Borazan Mushroom, Pine Cabara Mushroom, Muffin Mushroom, Emperor Mushroom, Oyster Mushroom, Kanlıca Mushroom, Kantarel Mushroom, Cauliflower Mushroom, Winter Mushroom, Köygöçüren Mushroom, Morel Mushroom, Culture Mushroom, Maitake Mushroom, Panther Mushroom, Beef Tongue Mushroom, Fly Mushroom, Devil Mushroom, Shimeji Mushroom, Shitake Mushroom and Truffle Mushroom. Sample images of these 20 different species are shown in Figure 2.

Fig.2. Sample images of 20 different mushroom species

It was preferred to increase the samples in the data set used for model training. The aim of this process is to increase the success of the model and to develop a real-time mobile application capable of solving the situations that people encounter in the real field. Mushrooms can grow in many different shapes and angles. For this reason, increasing the data will increase the learning diversity of the developed model and enable the mobile application to produce successful results in the real field. For data augmentation, new images were obtained by first shifting the images of each class 20 pixels each to the right, left, down and up. Afterwards, the existing images were rotated by 30-degree angles and recorded. In this way, 48 images shifted and rotated in different directions are obtained from one image. This process enriches the data set by diversifying it. Pseudo code of the algorithms used in the realisation of these processes is shown in Algorithm 1.

Algorithm 1 Data Augmentation Pseudo Code

1:	function getImage ( image_iterator[i])
2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 12: 13: 14: 15: 16:	Shift Range_Width(-20) SaveImage(iterator[i].jpg) Shift Range_Width(+20) SaveImage(iterator[i].jpg) Shift Range_Height(-20) SaveImage(iterator[i].jpg) Shift Range_Height(+20) SaveImage(iterator[i].jpg) function Rotate (image_iterator[i]) ←obj Rotation Range (i + 30) If (i =<330) SaveImage(iterator[i].jpg) i+=30 else return

In order to realise the whole scenario, the project proposed in this study is divided into different work packages. These work packages represent a living and continuously growing process. These work packages consist of research and identification processes related to fungal species, creation and collection of data set, performing model detection experiments for classification, model training, testing and validation of the model, and integration of the most efficient model into the mobile application. The cycle of living work packages is shown in Figure 3.

Fig.3. Working processes of the work packages prepared for the realisation of the project scenario

4.    Machine Learning Models and Hyperparameters

Machine Learning models include algorithms used in fields such as data analysis and pattern recognition. These algorithms are used to identify patterns in data sets and predict future data. Commonly used algorithms such as VGG16, DenseNet121, CNN, InceptionV3 and ResNet50. VGG16 is a convolutional neural network containing 16 layers. In this model, weights are learnable parameters and are removed from the data set during training. Activation functions determine the outputs of neurons in each layer and influence the learning capacity of the model. Generally, ReLU (Rectified Linear Unit) and Softmax function are preferred. The loss function measures the difference between the model's actual labels and its predictions. Mean Squared Error or cross entropy are frequently used loss functions. The optimizer algorithm is used to update the model's weights during training. Methods such as Stochastic Gradient Descent and Adam optimizer are frequently preferred. Epochs indicate how many times the data set is processed by the model during the training process. This can increase the model's ability to recognize more patterns and generalize. Validation split allows a portion of the data set to be reserved for validation during training. This helps prevent overfitting of the model and increases its generalization ability. These variables are important factors that affect the performance of Machine Learning models. Architecture values of the machine learning models used in this study are shown in Table 2.

n p u t s

224x224x3 ■ИНН Вак tn и UBES			Image to Matrix 224x224x32
ВИЯЖ ВНИИ USED			0	1	1	0		1	0	1	1
			1	1	1		0	• 1	1	0	1.
			0	1	1		0		0	0	0
ВИЯЖ SK^B LiSSEJO ■ИНН ■вив QBHO ВИНН ■вив ыввв			0	1	0		0	0	1	0	.0
		t	0	0	1		0	0	0	0	0
			0	1	1	0		0	4_	0	1'
			0	1	1	1		1	0	1	O'
			1	1	1	0		0	1	0	0

Fig.4. CNN architecture prepared for model

Figure 4 shows the CNN architecture prepared for model training. In the training results, the highest accuracy value was obtained from the CNN architecture.

Classifications in output layer

Table 2. Architectural variable values of algorithms

Model Architecture Variables	VGG16	CNN	DenseNet121	ResNet50	InceptionV3
Weights	imagenet	-	imagenet	imagenet	imagenet
Activation Function	reLU, SoftMax	reLU, SoftMax	SoftMax	reLU, SoftMax	reLU, SoftMax
Loss	Categorical cross entropy	Categorical cross entropy	Categorical cross entropy	Categorical cross entropy	Categorical cross entropy
Optimizer	Adam	Adam	Adam	Adam	Adam
Epochs	20	20	20	50	50
Validation Split	0.2	0.2	0.2	0.2	0.2

The training success graph and training loss graph of the CNN algorithm, which produces the most successful results among the trained models, are shown in Figure 5. The trained CNN model was subjected to 20 iterations of learning. Figure 6 shows the confusion matrix.

Fig.5. Training success graph and training loss graph of CNN algorithm

Fig.6. Confusion matrix graph of CNN algorithm

Table 3 shows the metric values for the training of all algorithms in detail.

Table 3. Values of training and test experiments of machine learning models

Algorithm & Metrics	Loss	Acc.	Val_Loss	Val_Acc.	Precision	Recall
CNN	0.075	0.998	0.186	0.964	0.998	0.998
VGG16	0.269	0.984	0.664	0.962	0.983	0.984
DenseNet121	0.471	0.944	0.542	0.914	0.944	0.944
ResNet50	0.427	0.941	0.631	0.909	0.942	0.942
Inception-v3	0.726	0.936	0.802	0.892	0.935	0.936

5.    Developed Mobile Software: MushAPP

TensorFlow Lite is an important tool that optimises and lightens models trained with deep learning libraries and makes them runnable on mobile devices. By converting the trained model into TensorFlow Lite, it can be integrated into mobile applications. TensorFlow Lite has an API that can be easily used in Android Studio. A command line tool called tflite_convert or Python API can be used to convert to TensorFlow Lite. The converted TensorFlow Lite model is included in the Android Studio project. This file is then moved to the assets folder in the mobile application and used directly in this area. Figure 7 shows the classified state of a mushroom using the existing MushAPP, the classification accuracy value and information about the mushroom species.

Fig.7. Sample mushroom classification and information screen in MushAPP

6.    Limitations

This study is limited to 20 different mushroom species in the data set. The most suitable species for the location of the study were included in the study. The pre-processing process of the data set should be planned correctly while the study is being carried out. It is predicted that cleaning the backgrounds of the images used for model training can increase the accuracy of the results. However, this process is quite labour intensive. In addition, as many training repetitions as possible will increase the model performance. For this reason, the resources of the hardware where the training will be carried out should be quite powerful.

7.    Conclusions

This study addresses the critical issue of mushroom poisoning caused by the accidental or unconscious consumption of toxic mushrooms. Recognizing the importance of raising public awareness about mushroom species, we developed MushAPP, a mobile application designed to distinguish between edible and poisonous mushrooms using advanced deep learning models. Our approach incorporated various machine learning and deep learning techniques, with the CNN model achieving the highest accuracy rate of 99.8%. Key findings from our research include.

Effective Model Training: Through rigorous training and data augmentation, our CNN model demonstrated a high accuracy rate, reducing the likelihood of misclassification.

Real-World Applicability: MushAPP successfully integrates the deep learning model into a mobile application, offering users a practical tool for identifying mushroom species in real-time.

Scalability and Future Improvements: The study highlights the potential for improved accuracy with the inclusion of more diverse training data and additional mushroom species.

The implications of this work are significant. By providing an accessible and reliable tool for mushroom identification, we can mitigate the risks associated with mushroom poisoning, enhance public health, and increase awareness about the diverse mushroom species. Future research will focus on expanding the dataset, incorporating more species, and further refining the model to achieve even greater accuracy and robustness. This will ensure that the application remains a valuable resource for both casual foragers and professional mycologists.