Clustering of handwritten digits by Kohonen neural network

Clustering of handwritten digits by Kohonen neural network

Автор: Latypova Dina Sergeevna, Tumakov Dmitrii Nikolaevich

Журнал: Программные системы: теория и приложения @programmnye-sistemy

Рубрика: Искусственный интеллект, интеллектуальные системы, нейронные сети

Статья в выпуске: 3 (54) т.13, 2022 года.

Бесплатный доступ

Clustering of handwritten digits is carried out for sixty thousand images contained in the training sample of the MNIST database. For clustering, the Kohonen neural network is used. For each handwritten digit, the optimal number of clusters (no more than 50) is determined. When determining the distance between objects (images of handwritten digits), the Euclidean norm is used. Checking the correctness of building clusters is carried out using data from the test sample of the MNIST database. The test sample contains ten thousand images. It is concluded that the images from the test sample belong to the "correct digit" cluster with a probability of more than 90{\%}. For each digit, an F-measure is calculated to evaluate the clusters. The best F-measures are obtained for digits 0 and 1 (F-mean is 0.974). The worst values are obtained for the number 9 (F-mean is 0.903). A cluster analysis is also carried out, which allows drawing conclusions about possible errors in recognition by the Kohonen neural network. Intersections of clusters for images of handwritten digits are constructed. Examples of intersections of clusters are given, as well as examples of images that are incorrectly recognized by the neural network.

Еще

Kohonen neural network, clustering, mnist

Короткий адрес: https://sciup.org/143179404

IDR: 143179404   |   DOI: 10.25209/2079-3316-2022-13-3-241-254

Список литературы Clustering of handwritten digits by Kohonen neural network

  • D. Latypova. Neural networks using for handwritten numbers recognition, Master’s Thesis, Czech Technical University, Prague, 2020, 77 pp. hUtRtpLs://dspace.cvut.cz/bitstream/handle/10467/88142/F3-DP-2020-Latypova-Dina-Master%20Thesis%20Dina%20Latypova%20v3.pdf?sequence=-
  • M. Rexy, K. Lavanya. “Handwritten digit recognition of MNIST data using consensus clustering”, International J. of Recent Technology and Engineering, 7:6 (2019), pp. 1969–1973. hUtRtpLs://www.ijrte.org/wp-content/uploads/papers/v7i6/F2408037619.pdf
  • S. Nhery, R. Ksantini, M. B. Kaaniche, A. Bouhoula. “A novel handwritten digits recognition method based on subclass low variances guided support vector machine”, Proc. of the 13th Int. Joint Conf. on Computer Vision, Imaging and Computer Graphics Theory and Application. V. 4, VISIGRAPP 2018 (27–29 January 2018, Funchal, Madeira, Portugal), 2018, ISBN 978-989-758-290-5, pp. 28–36. hUtRtpLs://www.scitepress.org/papers/2018/66111/66111.pdf
  • S. A. Shal, V. Koltun. “Robust continuous clustering”, Proc. the Natl. Acad. Sci. USA, 114:37 (2017), pp. 9814–9817. https://doi.org/10.1073/pnas.1700770114
  • E. Miri, S. M. Razavi, J. Sadri. “Performance optimization of neural networks in handwritten digit recognition using Intelligent Fuzzy C-Means clustering”, 2011 1st International eConference on Computer and Knowledge Engineering (ICCKE) (13–14 Oct. 2011, Mashhad, Iran), 2011, pp. 150–155. https://doi.org/10.1109/ICCKE.2011.6413342
  • S. Pourmohammad, R. Soosahabi, A. S. Maida. “An efficient character recognition scheme based on k-means clustering”, 2013 5th International Conference onModeling, Simulation and Applied Optimization (ICMSAO) (28–30 April 2013, Hammamet, Tunisia), 2013, pp. 1–6. https://doi.org/10.1109/ICMSAO.2013.6552640
  • B. Y. Li. “An experiment of k-means initialization strategies on handwritten digits dataset”, Intelligent Information Management, 10:2 (2018), pp. 43–48. https://doi.org/10.4236/iim.2018.102003
  • L. C. Munggaran, S. Widodo, A. M. Cipta, Nuryuliani. “Handwritten pattern recognition using Kohonen neural network based on pixel charactacter”, International J. of Advanced Computer Science and Applicatons, 5:11 (2014), 6 pp. https://doi.org/10.14569/hUIJtRtApLCs:S/A/t.h2e0s1a4i.0o5rg1/1D01ownloads/Volume5No11/Paper_1-Handwritten_Pattern_Recognition_Using_Kohonen_Neural_Network.pdf
  • A. Fahad, N. Alshatri, Z. Tari, A. Alamri, I. Khalil, A. Zomaya, S. Foufou, A. Bouras. “A survey of clustering algorithms for big data: taxonomy and empirical analysis”, IEEE Transactions on Emerging Topics in Computing, 2:3 (2014), pp. 267–279. https://doi.org/10.1109/TETC.2014.2330519
  • Y. Bi, P. Wang, X. Guo, Z. Wang, S. Cheng. “K-means clustering optimizing deep stacked sparse autoencoder”, Sensing and Imaging, 20:1 (2019), 6, 19 pp. https://doi.org/10.1007/s11220-019-0227-1
  • Y. Chen, C.-G. Chen, C. You. “Stochastic sparse subspace clustering”, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (13–19 June 2020, Seattle, WA, USA), 2020, pp. 4155–4164. https://doi.org/10.1109/CVPR42600.2020.00421
  • S. Zhang, C. You, R. Vida, C. G. Li. “Learning a self-expressive network for subspace clustering”, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (20–25 June 2021, Nashville, TN, USA), 2021, pp. 12393–12404. https://doi.org/10.1109/CVPR46437.2021.01221
  • D. Latypova, D. Tumakov. “Opredeleniye osnovnykh klasterov rukopisnykh tsifr”, Tsifrovaya obrabotka signalov i yee primeneniye. DSPA - 2020 (14–15 aprelya 2020 goda, Moskva, Rossiya), 2020, pp. 620–625.
  • D. Latypova, D. Tumakov. “Peculiarities of image recognition by the Hopfield neural network”, IEMAICLOUD 2021: International Conference on Intelligent Emerging Methods of Artificial Intelligence & Cloud Computing, Smart Innovation, Systems and Technologies, vol. 273, ed. Garcia Marquez F.P., Springer, Cham, 2021, ISBN 978-3-030-92904-6, pp. 34–47. https://doi.org/10.1007/978-3-030-92905-3_4
  • S. McConnell, R. Sturgeon, G. Henry, A. Mayne, R. Hurley. “Scalability of self-organizing maps on a GPU cluster using OpenCL and CUDA”, High Perfomance Computer Symposium, J. Physics: Conf. Series, 341 (2012), 012018, 10 pp. https://doi.org/10.1088/1742-6596/341/1/012018
  • Y. Xu, W. Zhang. “On a clustering method for handwritten digit recognition”, 2010 Third International Conference on Intelligent Networks and Intelligent Systems (01–03 November 2010, Shenyang, China), 2010, pp. 112–115. https://doi.org/10.1109/ICINIS.2010.130
  • G. Cohen, S. Afshar, J. Tapson, A. van Schaik. “EMNIST: extending MNIST to handwritten letters”, 2017 International Joint Conference on Neural Networks (IJCNN) (14–19 May 2017, Anchorage, AK, USA), 2017, pp. 2921–2926. https://doi.org/10.1109/IJCNN.2017.7966217
  • A. Baldominos, Y. Saez, P. Isasi. “A survey of handwritten character recognition with MNIST and EMNIST”, J. of Applied Science, 9:15 (2019), 3169, 16 pp. https://doi.org/10.3390/app9153169
  • A. F. Agarap, A.P. Azcarraga. “Improving k-means clustering performance with disentangled internal representations”, 2020 International Joint Conference on Neural Networks (IJCNN) (19–24 July 2020, Glasgow, UK), 2020, pp. 1–8. https://doi.org/10.1109/IJCNN48605.2020.9207192
  • K. Cheng, R. Tahir, L. C. Eric, M. Li. “An analysis of generative adversarial networks and variants for image synthesis on MNIST dataset”, J. Multimedia Tools and Applications, 79 (2020), pp. 13725–13752. https://doi.org/10.1007/s11042-019-08600-2
  • J. Kossen, S. Farquhar, Y. Gal, T. Rainforth. “Active testing: sample-efficient model evaluation”, Proc. of the 38th International Conf. on Machine Learning (18-24 July 2021, Virtual), Proceedings of Machine Learning Research, vol. 139, 2021, pp. 5753–5763. UhtRtpL://proceedings.mlr.press/v139/kossen21a/kossen21a.pdf
  • R. Zhang, P.-C. Chang. “Robustness against adversary models on MNIST by deep-Q reinforcement learning based parallel-GANs”, 2021 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) (14–17 December 2021, Tokyo, Japan), 2021, pp. 1590–1597. hUtRtpL://www.apsipa.org/proceedings/2021/pdfs/0001590.pdf
  • F. Murtagh, M. Hernandez-Pajares. “The Kohonen self-organizing map method: An assessment”, J. of Classification, 12 (1995), pp. 165–190. https://doi.org/10.1007/BF03040854
  • I. S. Sen’kovskaya, P. V. Sarayev. “Automatic clustering in data analysis based on Kohonen self-organizing maps”, Vestnik MGTU im. G.I. Nosova, 2011, no. 2(34), pp. 78–79 (in Russian). hUtRtpL://vestnik.magtu.ru/content/%D0%92%D0%B5%D1%81%D1%82%D0%BD%D0%B8%D0%BA%20%D0%9C%D0%93%D0%A2%D0%A3%20%D0%B7%D0%B0%202011%20%D0%B3%D0%BE%D0%B4,%20%D0%9D%D0%BE%D0%BC%D0%B5%D1%80%202.pdf
  • P. Y. Simard, D. Steinkraus, J. C. Platt. “Best practices for convolutional neural networks applied to visual document analysis”, Proc. Seventh International Conference on Document Analysis and Recognition (06 August 2003, Edinburgh, UK), 2003, ISBN 0-7695-1960-1, pp. 958–963. https://doi.org/10.1109/ICDAR.2003.1227801
  • D. Ciresan, U. Meier, J. Schmidhuber. “Multi-column deep neural networks for image classification”, 2012 IEEE Conference on Computer Vision and Pattern Recognition (16–21 June 2012, Providence, RI, USA), 2012, pp. 3642–3649. https://doi.org/10.1109/CVPR.2012.6248110
  • Z. Kayumov, D. Tumakov, S. Mosin. “Hierarchical convolutional neural network for handwritten digits recognition”, Proc. Computer Science, 171 (2020), pp. 1927–1934. https://doi.org/10.1016/j.procs.2020.04.206
  • Z. Kayumov, D. Tumakov, S. Mosin. “Combined convolutional and perceptron neural networks for handwritten digits recognition”, 2020 22th International Conference on Digital Signal Processing and its Applications (DSPA) (25–27 March 2020, Moscow, Russia), 2020, pp. 74, 5 pp. https://doi.org/10.1109/DSPA48919.2020.9213301
Еще
Статья научная
QR-код для установки приложения SciUp Наведите камеру и скачайте бесплатное приложение SciUp