Time – Delay Simulated Artificial Neural Network Models for Predicting Shelf Life of Processed Cheese

Published Online May 2012 in MECS

Our biological nervous system is simulated to solve complex functions in various applications. The Simulated Neural Networks (SNN) is based on several nodes that are connected to each other. From the operational point-of-view, such a network operates as a black box: one side is used for inputs while the other side provides the required outputs. For this a training session is needed [1]. Several transfer functions [2] can be used to calculate outputs using normalized input values appropriately weighted. Fundamentally, there are two types of Artificial Neural Network (ANN) learning models: supervised and unsupervised learning [3]. Processed cheese is a protein rich food, and is a comparable supplement to meat protein. It is generally manufactured from medium ripened (6 months old) Cheddar cheese, but sometimes less ripened Cheddar cheese is also added in lesser proportion. Its manufacturing technique includes addition of emulsifier, salt, water and selected spices, if desired. The mixture is heated with steam in a stainless steel double jacketed vessel with continuous stirring with a flattened ladle in order to get homogeneous paste. This variety of cheese is very tasty and has longer shelf life.

Shelf life is defined as the length of time that a product is acceptable and meets the consumer’s expectations regarding food quality. It is the result of the conjunction of all services in production, distribution, and consumption. Shelf life dating is one of the most difficult tasks in food engineering. Market pressure has led to the implementation of shelf life by sensory analyses, which may not reflect the full quality spectra. Moreover, traditional methods for shelf-life dating and small-scale distribution chain tests cannot reproduce in a laboratory the real conditions of storage, distribution, and consumption on food quality. The consumer demands foods under the legal standards, at low cost, high standards of nutritional, sensory, and health benefits [4].Food engineers are facing the challenges to monitor, and control the quality and safety of food products. The new technologies , viz., nanotechnology, multivariate sensors, information systems, and complex systems have revolutionized the food industry.

The architecture of ANN is displayed in Fig.1.

Figure.1. Architecture of simple artificial neural network

Input Layer — A vector of predictor variable values ( x1...xp ) is presented to the input layer. The input layer (or processing before the input layer) standardizes these values so that the range of each variable is -1 to 1. The input layer distributes the values to each of the neurons in the hidden layer. In addition to the predictor variables, there is a constant input of 1.0, called the bias that is fed to each of the hidden layers; the bias is multiplied by a weight and added to the sum going into the neuron [5].

Hidden Layer — Arriving at a neuron in the hidden layer, the value from each input neuron is multiplied by a weight ( wji ), and the resulting weighted values are added together producing a combined value uj . The weighted sum ( uj ) is fed into a transfer function, σ, which outputs a value hj . The outputs from the hidden layer are distributed to the output layer [5].

Output Layer — Arriving at a neuron in the output layer, the value from each hidden layer neuron is multiplied by a weight ( wkj ), and the resulting weighted values are added together producing a combined value vj . The weighted sum ( vj ) is fed into a transfer function, σ, which outputs a value yk . The y values are the outputs of the network [5].

• A. Single and multiple hidden layer architectures

The single hidden layer organization, in which all units are connected to one another, constitutes the most general case and is of more potential computational power than hierarchically structured multiple layer organizations [6] as represented in Fig. 2.

In multiple layer networks, units are often numbered by layer, instead of following a global numbering supervised learning which incorporates an external teacher; so that each output unit is told what its desired response to input signals ought to be [6]. Architecture of multiple hidden layers is displayed in Fig.3.

Figure.2. Single hidden layers model

Time -Delay Neural Networks are special artificial neural networks which receive input over several time steps. Time delay neural network is an alternative neural network architecture whose primary purpose is to work on continuous data. The advantage of this architecture is to adapt the network online and hence helpful in many real time applications, like time series prediction, online spell check, continuous speech recognition, etc. The architecture has a continuous input that is delayed and sent as an input to the neural network [7, 8].

II. LITERATURE REVIEW

Shankar and Bandyopadhyay (2007) developed a backpropagation ANN models to predict the properties of extrudates generated by extrusion cooking of fish muscle-rice flour blend in a single screw extruder [9]. ANN models are utilized in two ways to model the pores formation during air drying [10]. Cascade single and multiple hidden layers ANN models for predicting sensory quality of roasted coffee flavoured sterilized drink were developed. Colour and appearance, viscosity, flavour and sediment were taken as input variables,

Figure.3. Multiple hidden layer architecture while overall acceptability score was taken as output variable. The study revealed that more the number of neurons in single hidden layer, less is the error for cascade ANN models [11]. Brain based artificially intelligent scientific computing models, viz., Cascade Neural Network (CNN) and Probabilistic Neural Network (PNN) models for shelf life detection of cakes stored at 30o C have been developed. In this study, the data pertaining to moisture, titratable acidity, free fatty acids, peroxide value, and tyrosine were taken as input variables; and overall acceptability score was output variable. Mean square error, root mean square error, coefficient of determination and nash - sutcliffo coefficient were used in order to compare the prediction performance of the developed models. The CNN model with single hidden layer having twenty five neurons gave good results [12]. Elman and self organizing simulated neural network models were developed for predicting shelf life of soft cakes stored at 10oC. Moisture, titratable acidity, free fatty acids, tyrosine and peroxide value were input variable; and overall acceptability score was output variable. The network was trained with single as well as double hidden layers; transfer function for hidden layer was tangent sigmoid while for the output layer, it was pure linear function. The investigation suggested that simulated neural networks are excellent tool for predicting the shelf life of soft cakes [13]. The efficiency of Cascade single and multiple hidden layer models was tested for shelf life prediction of Kalakand, a sweetened desiccated dairy product. For developing the models, the network was trained with 100 epochs. Cascade models with two hidden layers having twenty neurons in the first layer and twenty neurons in the second layer gave best result (MSE 0.000988770; RMSE: 0.03144471; R2: 0.988125331) [14]. Radial basis artificial neural engineering and multiple linear regression models for forecasting shelf life of instant coffee drink were developed and compared with each other. The values of colour and appearance, flavour, viscosity and sediment were used as input variables, while overall acceptability score was taken as output variable. The investigation revealed that multiple linear regression model was superior over radial basis model for forecasting shelf life of instant coffee drink [15]. In another study, Cascade forward and feedforward backpropagation artificial intelligence models for predicting sensory quality of instant coffee flavoured sterilized drink were developed. Different combinations of several internal parameters, viz., data pre-processing, data partitioning, number of hidden layers, number of neurons in each hidden layer, transfer function, error goal, etc., along with backpropagation algorithm based on Levenberg– Marquardt mechanism as training function were explored. The network was trained with 100 epochs. The number of neurons in each hidden layer varied from 1 to 20. The results of cascade forward and feed forward artificial intelligent models were evaluated with three types of prediction performance measures, viz., Root Mean Square Error, coefficient of determination R2 and

Mean Square Error. Feedforward backpropagation artificial intelligence model exhibited best results (3.70% RMSE; 0.998 R2; 0.0013 MSE), followed by cascade forward artificial intelligence model (5.36% RMSE; 0.996 R2; 0.0028 MSE) for predicting sensory quality of instant coffee flavoured sterilized drink [16]. Elman and generalized regression artificial intelligence models were developed for detecting the shelf life of instant coffee flavoured sterilized drink. For developing the artificial intelligence models, the values of colour and appearance, flavour, viscosity, and sediment were taken as input variables, while the overall acceptability score was used as output variable. The dataset consisted of experimentally developed 50 observations. The observations were randomly divided into two sets, namely, training set consisting of 40 observations (80% of total observations) and validation set containing 10 observations (20% of total observations). Mean Square Error and Root Mean Square Error were used as prediction performance measures. The observed best result for Elman models were Neurons 4:4; MSE: 0.001101304; RMSE: 0.033185894; and for generalized regression models Spread Constant:2; MSE: 0.164212158;RMSE: 0.405230993[17]. Recently, linear layer (train) and generalized regression models were developed and compared with each other for predicting the shelf life of milky white dessert jeweled with pistachios. Neurons in each hidden layers varied from 1 to 30. Datasets were divided into two sets, i.e., 80% of data samples were used for training and 20% for validating the network. Mean square error, root mean square error, coefficient of determination and nash -sutcliffo coefficient were applied in order to compare the prediction performance of the developed models. The study revealed that artificial neural networks are quite effective for determining the shelf life of milky white dessert jeweled with pistachios [18]. For predicting shelf life of brown milk cakes decorated with almonds ANN models , viz., Radial basis (exact fit) and Radial basis (fewer neurons) were developed and compared with each other. Both the developed models performed equally exceedingly well in detecting the shelf life of the product, and there was no significant difference in the predicted shelf life and the experimental shelf life of the product, establishing that ANN computing models are easy and powerful alternative tool to laboratory’s experimental expensive and long time consuming shelf life testing methods for predicting shelf life of foods [19]. Time-delay and linear layer (design) intelligent computing expert system models for predicting shelf life of soft mouth melting milk cakes stored at 6oC were implemented. The best results for time-delay model with single hidden layer having 20 neurons were MSE: 0.001332342, RMSE: 0.036501259, R2: 0.984011897 for time-delay model with double hidden layers having 8 neurons in the first and second layers MSE: 0.001318004, RMSE: 0.036304329, R2: 0.984183948. Best results for linear layer (design) model were MSE: 0.000293366, RMSE: 0.017127919, R2: 0.996479613,suggesting that the intelligent computing expert system models are efficient in predicting the shelf life of soft mouth melting milk cakes [20].

• III.    METHOD AND MATERIAL

In the present research datasets consisted of 36 experimental observations, which were divided into two subsets, i.e., 30 for training (80% of total observations) and 6 (20% of total observations) for testing the developed Time -Delay ANN models (Fig. 4).

The training pattern for ANN models is represented in Fig. 6.

The best training procedure is to compile a wide range of examples (for more complex problems, more examples are required), which exhibit all the different characteristics of the problem. To create a robust and reliable network, in some cases, some noise or other randomness is added to training data to get the network familiarized with noise and natural variability in real data.

Weights are selected

Data Division

Figure.4. Data Division

Time –Delay ANN models are trained

Calculation of error

Adjustments to weights

Training ANN

The experimental values of body & texture, aroma & flavour, moisture, and free fatty acids of processed cheese stored at 7-8º C, were taken as input variables; and sensory score as output variable for developing ANN models (Fig.5).

Error evaluation

Moisture

Body &T exture

Free fatty acids

Sensory Score

Figure 5. Input and output parameters for ANN models

No

Minimum

Error is calculated

End

Yes

Figure.6. Training and testing of Time-Delay ANN models

Poor training data inevitably leads to an unreliable and unpredictable network. Usually, the network is trained for a prefixed number of epochs or when the output error decreases below a particular error threshold. Special care is taken not to over train the network. By overtraining, the network may become too adapted in learning the samples from the training set, and thus may be unable to accurately classify samples outside of the training set [21]. Levenberg Marquardt algorithm, Polak Ribiére Update conjugate gradient algorithm, Fletcher Reeves update conjugate gradient algorithm, Gradient Descent algorithm with adaptive learning rate, Bayesian regularization, Powell Beale restarts conjugate gradient algorithm and BFG quasi-Newton algorithm were tried. Bayesian regularization algorithm gave good results; therefore it was selected as training function. Number of neurons in each hidden layer varied from 1 to 20. The network was trained with upto 100 epochs.

The ANN was trained with single as well as multiple hidden layers. Several problems were faced while training ANN’s, too many neurons in the hidden layers resulted in overfitting. Overfitting occurs when the neural network has so much information processing capacity that the limited amount of information contained in the training set is not enough to train all of the neurons in the hidden layers. Using too few neurons in the hidden layers also resulted in underfitting. Underfitting occurs when there are too few neurons in the hidden layers to adequately detect the signals in a complicated dataset. A second problem can occur even when there is sufficient training data resulting in increase in training time of the network. Obviously some compromise must be reached between too many and too few neurons in the hidden layers. Ultimately the selection of the architecture of neural network came down to trial and error. There are two trial and error approaches that are used in determining the numbers of hidden neurons: the "forward" and "backward" selection methods. The first method, the "forward selection method", begins by selecting a small number of hidden neurons. The second method, the "backward selection method", begins by using a large number of hidden neurons. Then the neural network is trained and tested. This process continues until the performance improvement of the neural network is no longer significant [22].

• IV.    RESULTS AND DISCUSSION

Time-Delay single hidden layer (Table 1) and multiple hidden layer models (Table 2) were developed and compared with each other for predicting shelf life of processed cheese.

TABLE 1: RESULTS OF TIME-DELAY WITH SINGLE HIDDEN LAYER

Neurons	MSE	RMSE	R2	E2
3	8.38414E-06	0.002895539	0.997104461	0.999991616
4	5.73084E-05	0.007570234	0.992429766	0.999942692
5	1.11999E-05	0.003346628	0.996653372	0.9999888
6	0.000543848	0.023320554	0.976679446	0.999456152
7	1.6482E-05	0.004059802	0.995940198	0.999983518
8	0.00192582	0.043884171	0.956115829	0.99807418
9	3.9438E-05	0.006279968	0.993720032	0.999960562
10	0.03135071	0.177061316	0.822938684	0.96864929
11	1.88315E-05	0.004339528	0.995660472	0.999981168
12	1.92494E-05	0.004387409	0.995612591	0.999980751
13	4.07142E-05	0.00638077	0.99361923	0.999959286
14	4.55449E-05	0.006748697	0.993251303	0.999954455
15	0.001166341	0.034151734	0.965848266	0.998833659
16	4.32613E-05	0.006577334	0.993422666	0.999956739
17	1.58939E-05	0.003986721	0.996013279	0.999984106
18	0.000266091	0.016312291	0.983687709	0.999733909
19	0.031925352	0.178676668	0.821323332	0.968074648
20	2.48971E-05	0.004989701	0.995010299	0.999975103

TABLE 2: RESULTS OF TIME-DELAY WITH MULTIPLE HIDDEN LAYERS

Neurons	MSE	RMSE	R2	E2
3:3	0.001206792	0.034738905	0.965261095	0.998793208
4:4	1.12337E-05	0.003351668	0.996648332	0.999988766
5:5	4.41724E-06	0.002101723	0.997898277	0.999995583
6:6	4.70788E-06	0.002169764	0.997830236	0.999995292
7:7	8.67856E-06	0.00294594	0.99705406	0.999991321
8:8	9.46223E-07	0.00097274	0.99902726	0.999999054
9:9	4.29104E-06	0.002071482	0.997928518	0.999995709
10:10	0.000671258	0.025908646	0.974091354	0.999328742
11:11	6.7636E-06	0.002600693	0.997399307	0.999993236
12:12	9.97059E-06	0.003157624	0.996842376	0.999990029
13:13	0.000268315	0.016380333	0.983619667	0.999731685
14:14	3.02355E-06	0.001738835	0.998261165	0.999996976
15:15	5.13301E-05	0.007164505	0.992835495	0.99994867
16:16	1.21996E-05	0.003492791	0.996507209	0.9999878
17:17	2.29386E-06	0.001514551	0.998485449	0.999997706
18:18	6.86566E-05	0.008285928	0.991714072	0.999931343
19:19	2.57595E-05	0.005075383	0.994924617	0.99997424
20:20	4.45971E-06	0.002111803	0.997888197	0.99999554

MSE =

£ ( Q e

z

V

exp

—

n

Qcal

RMSE =

1 n

" Л ( Q exp - Q ca. ) _" Г V    Q exp    )_

(2)

R ² = 1 -

E ² = 1 -

N z 1

N z 1

^Q exp

- ^Qcal

V

^Q exp

^Q exp

V

Q exp

^{- Q}cal

Q

Z— exp у

Where,

Q = Observed value;

Q = Predicted value;

Q =Mean predicted value;

n = Number of observations in dataset.

Mean Square Error MSE (1), Root Mean Square Error RMSE (2), Coefficient of Determination R2 (3) and Nash - Sutcliffo Coefficient E2 (4) were used in order to compare the prediction ability of the developed Time-Delay ANN models. The comparison of Actual Sensory Score (ASS) and Predicted Sensory Score (PSS) for Time-Delay ANN models are illustrated in Fig.7 and Fig.8, respectively. Time-Delay single hidden layer model with 3 neurons got simulated best (MSE: 8.38414E-06, RMSE: 0.002895539, R2: 0.997104461, E2: 0.999991616), and for multiple hidden layer models combination of 7:7 neurons in the first and second hidden gave even better results (MSE: 8.67856E-06, RMSE: 0.00294594, R2: 0.99705406, E2: 0.999991321).

Figure.7. Comparison of ASS and PSS for single layer Time-Delay ANN model

After comparing the single and multiple hidden layer models, 4 ^ 7 ^ 7 ^ 1 combination gave the best results with less than 1% root mean square error. It was observed that multilayer Time-Delay ANN models simulate very nicely in predicting the shelf life of processed cheese stored at 7-8 ^oC.

• V.    conclusion

Time–Delay single and multilayer models were developed for predicting shelf life of processed cheese stored at 7-8 ^oC. The experimental values of body & texture, aroma & flavour, moisture, and free fatty acids were taken as input variables; and sensory score as output variable. The data consisted of 36 experimental observations, which were divided into two disjoint sets, i.e., 30 (80% of observations) for training, and 6 (20% of observations) for validation of the developed models. Combination of 4 ^ 7 ^ 7 ^ 1 gave the best fit with a root mean square error of less than 1%. Therefore, from the study it is concluded that Time-Delay ANN models are quite efficient in predicting the shelf life of processed cheese stored at 7-8 ^oC.

Figure.8. Comparison of ASS and PSS for multilayer