Deep Learning Sign Language Recognition System Based on Wi-Fi CSI
Автор: Marwa R. M. Bastwesy, Nada M. El Shennawy, Mohamed T. Faheem Saidahmed
Журнал: International Journal of Intelligent Systems and Applications @ijisa
Статья в выпуске: 6 vol.12, 2020 года.
Бесплатный доступ
Many sensing gesture recognition systems based on Wi-Fi signals are introduced because of the commercial off-the-shelf Wi-Fi devices without any need for additional equipment. In this paper, a deep learning-based sign language recognition system is proposed. Wi-Fi CSI amplitude and phase information is used as input to the proposed model. The proposed model uses three types of deep learning: CNN, LSTM, and ABLSTM with a complete study of the impact of optimizers, the use of amplitude and phase of CSI, and preprocessing phase. Accuracy, F-score, Precision, and recall are used as performance metrics to evaluate the proposed model. The proposed model achieves 99.855%, 99.674%, 99.734%, and 93.84% average recognition accuracy for the lab, home, lab + home, and 5 different users in a lab environment, respectively. Experimental results show that the proposed model can effectively detect sign gestures in complex environments compared with some deep learning recognition models.
Wireless, Device-free sensing, Channel State Information, Sign Language Recognition, Deep Learning, WiFi Imaging
Короткий адрес: https://sciup.org/15017519
IDR: 15017519 | DOI: 10.5815/ijisa.2020.06.03
Список литературы Deep Learning Sign Language Recognition System Based on Wi-Fi CSI
- H. Abdelnasser, M. Youssef, and K. A. Harras,“Wigest A ubiquitous wifi-based gesture recognition system,” in 2015 IEEE Conference on Computer Communications (INFOCOM). IEEE, 2015, pp. 1472–1480.
- Y. Wang, K. Wu, and L. M. Ni, “Wifall: Device-free fall detection by wireless networks,” IEEE Transactions on Mobile Computing, vol. 16, no. 2, pp. 581–594, 2016.
- T. Z. Chowdhury, “Using wi-fi channel state information (csi) for human activity recognition and fall detection,” Ph.D. dissertation, University of British Columbia, 2018.
- S. Oprisescu, C. Rasche, and B. Su, “Automatic static hand gesture recognition using tof cameras,” in 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO). IEEE, 2012, pp.2748–2751.
- Z. Zhang, “Microsoft kinect sensor and its effect,” IEEE multimedia,vol. 19, no. 2, pp. 4–10, 201
- U. of Washington. (2016) Signaloud demo. [Online]. Available: http://www.washington.edu/news/2016/04/12/uw-undergraduate-teamwins-10000-lemelson-mit-student-prize-for-gloves-that-translate-signlanguage/
- M. Scholz, S. Sigg, H. R. Schmidtke, and M. Beigl, “Challenges for device-free radio-based activity recognition,” in Proceedings of the 3rd workshop on Context Systems, Design, Evaluation and Optimisation (CoSDEO 2011), in Conjunction with MobiQuitous, vol. 2011, 2011.
- M. Al-qaness and F. Li, “Wiger: Wifi-based gesture recognition system,” ISPRS International Journal of Geo-Information, vol. 5, no. 6, p. 92, 2016.
- Q. Pu, S. Gupta, S. Gollakota, and S. Patel, “Whole-home gesture recognition using wireless signals,” in Proceedings of the 19th annual international conference on Mobile computing & networking. ACM, 2013, pp. 27–38.
- Y. Ma, G. Zhou, S. Wang, H. Zhao, and W. Jung, “Signfi: Sign language recognition using wifi,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2, no. 1, p. 23, 2018.
- J. Wang, D. Vasisht, and D. Katabi, “Rf-idraw: virtual touch screen in the air using rf signals,” ACM SIGCOMM Computer Communication Review, vol. 44, no. 4, pp. 235–246, 2015.
- S. Sigg, S. Shi, F. Buesching, Y. Ji, and L. Wolf, “Leveraging rf-channel fluctuation for activity recognition: Active and passive systems, continuous and rssi-based signal features,” in Proceedings of International Conference on Advances in Mobile Computing & Multimedia, 2013, pp. 43–52.
- W. Wang, A. X. Liu, M. Shahzad, K. Ling, and S. Lu, “Understanding and modeling of wifi signal based human activity recognition,” in Proceedings of the 21st annual international conference on mobile computing and networking. ACM, 2015, pp. 65–76.
- W. He, K. Wu, Y. Zou, and Z. Ming, “Wig: Wifi-based gesture recognition system,” in 2015 24th International Conference on Computer Communication and Networks (ICCCN). IEEE, 2015, pp. 1–7.
- H. Li, W. Yang, J. Wang, Y. Xu, and L. Huang, “Wifinger: talk to your smart devices with finger-grained gesture,” in Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 2016, pp. 250–261.
- Z. Tian, J. Wang, X. Yang, and M. Zhou, “Wicatch: a wi-fi based hand gesture recognition system,” IEEE Access, vol. 6, pp. 16 911–16 923, 2018.
- C. Wang, S. Chen, Y. Yang, F. Hu, F. Liu, and J. Wu, “Literature review on wireless sensing-wi-fi signal-based recognition of human activities,” Tsinghua Science and Technology, vol. 23, no. 2, pp. 203–222, 2018.
- C. Feng, S. Arshad, and Y. Liu, “Mais: Multiple activity identification system using channel state information of wifi signals,” in International Conference on Wireless Algorithms, Systems, and Applications. Springer, 2017, pp. 419–432.
- Y. Xie, Z. Li, and M. Li, “Precise power delay profiling with commodity wi-fi,” IEEE Transactions on Mobile Computing, vol. 18, no. 6, pp. 1342–1355, 2018.
- S. Liu, Y. Zhao, F. Xue, B. Chen, and X. Chen, “Deepcount: Crowd counting with wifi via deep learning,” arXiv preprint arXiv:1903.05316, 2019.
- C. Wu, Z. Yang, Z. Zhou, K. Qian, Y. Liu, and M. Liu, “Phaseu: Realtime los identification with wifi,” in 2015 IEEE conference on computer communications (INFOCOM). IEEE, 2015, pp. 2038–2046.
- C. Gao, Y. Li, and X. Zhang, “Livetag: Sensing human object interaction through passive chipless wifi tags,” in 15th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 18), 2018, pp. 533–546.
- L. Bottou, “Stochastic gradient descent tricks,” in Neural networks: Tricks of the trade. Springer, 2012, pp. 421–436.
- B. Wei, K. Li, C. Luo, W. Xu, and J. Zhang, “No need of data preprocessing: A general framework for radio-based device-free context awareness,” arXiv preprint arXiv:1908.03398, 2019.
- S. Yousefi, H. Narui, S. Dayal, S. Ermon, and S. Valaee, “A survey on behavior recognition using wifi channel state information,” IEEE Communications Magazine, vol. 55, no. 10, pp. 98–104, 2017.
- Z. Chen, L. Zhang, C. Jiang, Z. Cao, and W. Cui, “Wifi csi based passive human activity recognition using attention based blstm,” IEEE Transactions on Mobile Computing, 2018.
- A. Gulli and S. Pal, Deep learning with Keras. Packt Publishing Ltd, 2017.
- D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.
- M. D. Zeiler, “Adadelta: an adaptive learning rate method,” arXiv preprint arXiv:1212.5701, 2012.
