Deep Learning Model for Automatic and Real-Time Detection and Identification of Faces from Digital Images

Mamza Godiya Jasini; Dr. Yusuf Musa Malgwi; Mingyi Charity Lazarus

doi:10.56201/ijcsmt.v10.no4.2024.pg21.52

📄 Download Paper

Deep Learning Model for Automatic and Real-Time Detection and Identification of Faces from Digital Images

Mamza Godiya Jasini, Dr. Yusuf Musa Malgwi, Mingyi Charity Lazarus

Abstract

The expression on a human's face is a crucial aspect of communication and plays a significant role in conveying emotions and information visually. Although recognizing emotions from facial expressions comes naturally to humans, it presents a substantial challenge for computer algorithms. Extracting features through various image processing techniques is essential for machines to interpret emotions from images or videos. The development of an algorithm that can detect, extract, and evaluate facial expressions would enable automatic recognition of human emotions in digital images and videos. This study introduces a real-time facial emotion recognition system that utilizes a Convolutional Neural Network (CNN) to process image frames, detect faces, and recognize emotions from facial expressions. By training the CNN model on a large dataset of facial images and emotions, the system achieves accurate and rapid emotion recognition performance. The datasets utilized included FER2013 and Cohn- Kanade (CK) obtained from the Kaggle dataset.The Precision, Recall, and F-score from the CK dataset were 83.6142%, 95.0822%, and 88.9955%, respectively, while those of the FER2013 dataset were 91.8986%, 98.3649%, and 95.0218%, respectively. The developed system has the ability to rapidly detect faces in cluttered backgrounds and accurately classify emotions in real-time. Various recognition techniques were compared, and a tradeoff between accuracy and speed was evaluated for each. The results indicate that the CNN-based approach is highly effective in accurately recognizing facial emotions, with significant potential for real- world applications of facial emotion detection. The real-time implementation of the system can be used for person identification and authentication. Additionally, it can be utilized by doctors to understand the intensity of pain or illness in deaf patients.In security systems, it can identify a person regardless of their pr

Keywords

Facial Emotion RecognitionConvolutional Neural Network (CNN)Real-time

References

Avots E., Sapi?ski T., Bachmann M., Kami?ska D. Audiovisual emotion recognition in wild. Machine Vision and Applications. 2019;30(5):975–985. Bagwan R., Dhapudkar K., Chintawar S., &Balamwar A. (2021). Face Emotion Recogniton using Convolutional Neural Network. Grenze International Journal of Engineering & Technology (GIJET), 7(1). Dahmane M., Meunier J. “Emotion recognition using grid-based HoG features”, Face and Gesture 2011, Santa Barbara, CA, 2011. Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4),384-392. Ekman R (1997) What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS). Oxford University Press, New York. EkmanP., RosenbergE. L. What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System, Oxford University Press, Oxford, UK, 2005. Fan X.,TjahjadiT. “A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences,” Pattern Recognition, vol. 48, no. 11, pp. 3407–3416, 2015. Fan X., Tjahjadi T. Fusing dynamic deep learned features and handcrafted features for facial expression recognition. Journal of Visual Communication and Image Representation. 2019;65. Gholami F., Fatemeh G. Facial Expression Recognition based on combination of the basic facial expressionInternational Journal of Computer Science and Network Security,Vol. 17, no. 4 (2017). Giannopoulos P., Perikos I., Hatzilygeroudis I. (2018) Deep learning approaches for facial emotion recognition: a case study on FER-2013. In: Hatzilygeroudis I, Palade V (eds) Advances in hybridization of intelligent methods. Springer, Berlin, pp 1–16. Huang Y. L., Chen S. H., Tseng H. H. Attachment avoidance and fearful prosodic emotion recognition predict depression maintenance. Psychiatry Research. 2019;272:649–654. Jain N., Kumar S., Kumar A., Shamsolmoali P., Zareapoor M. Hybrid deep neural networks for face emotion recognition. Pattern Recognition Letters. 2018;115:101–106. JanA., Ding H., MengH., ChenL., Li, H. (2018). “Accurate facial parts localization and deep learning for 3D facial expression recognition,” in Proceedings of the 13thIEEE International Conference on Automatic Face and Gesture Recognition. 466-472. KanadeT., CohnJ. F.,TianY. “Comprehensive database for facial expression analysis,” in Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG '00), pp. 46–53, Grenoble, France, March 2000. Kazemi V., Sullian J. “One millisecond face alignment with an ensemble of regression trees”, 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, Kittler J., Xiao-jun W., Yun-kun L. (2019) A deep learning network for face recognition. Journal of electronic imaging, vol. 28 Klaser A., MarszalekM., SchmidC. A spatio-temporal descriptor based on 3Dgradients, Proc. Brit. Mach. Vis. Conf. (BMVC), pp. 275, Sep. 2008 Kratzwald B., Ilic S., Kraus M., Feuerriegel S., Prendinger H. Deep learning for affective computing: text-based emotion recognition in decision support. Decision Support Systems. 2018;115:24–35. Kulkarni N., Kaur S. FERFM: An enhanced facial emotion recognition system using fine- tuned mobilenetv2 architecture. IETE journal of research, 2023. Kumar R. K., Garain J., Kisku D. R., Sanyal G. Estimating attention of faces due to its growing level of emotions. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW); 2018; Salt Lake City, UT, USA. Le Ngo A.C., Oh Y.H., Phan R.C., See J. “Eulerian emotion magnification for subtle expression recognition”, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, 2016. Liang D., Liang H., Yu H., Zhang Y. (2020). Deep convolutional BiLSTM fusion network for facial expression recognition. Vis. Comput. 36, 499-508. Liang X., Xu L., Liu J., et al. Patch attention layer of embedding handcrafted features in CNN for facial expression recognition. Sensors. 2021;21(3): p. 833. MehendaleN.“Facial emotion recognition using convolutional neural networks (FERC).” SN Appl. Sci. 2, 446 (2020). Mishra S., Prasada G. R. B., Kumar R. K., Sanyal G. Emotion recognition through facial gestures - a deep learning approach. Proceedings of the International Conference on Mining Intelligence and Knowledge Exploration; 2017; Cham: Springer. PalR., MukherjeeA., MitraP.,MukherjeeJ. Modelling visual saliency using degree centrality, IET Computer Vision, vol. 4, no. 3, pp. 218-229, 2010. Pantic M., RothkrantzL. J. M. Automatic analysis of facial expressions: The state of the art, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1424-1445, Dec. Rajesh K.M., Naveenkumar M. “A robust method for face recognition and face detection system using support vector machines”, 2016 International Conference on Electrical, Electronics, Communication, Computer and Optimization Techniques (ICEECCOT), Mysuru, 2016. ReddyV. G., Dharma S. C., Mukherjee S. Facial expression recognition in the wild, by fusion of deep learnt and hand-crafted features. Cognitive Systems Research. 2020;62:23–34. RussellJ. A., Fernandez-DolsJ. M., MandlerG. The Psychology of Facial Expression, Cambridge University Press, Cambridge, UK, 1997. ShanC., GongS., McOwanP. W., Facial expression recognition based on local binary patterns: A comprehensive study, Image Vis. Comput., vol. 27, no. 6, pp. 803-816, Swinkles W., Clasen L., Xiao F., Shen H. “SVM point-based real-time emotion detection”,2017 IEEE Conference on Dependable and SecUreComPuting, Taipei, ViolaP., JonesM.J. Robust Real-Time Face Detection. International Journal of Computer Vision 57, 137–154 (2004). Wang X.H., Liang Y. C.,MaX. C. (2020). Facial expression classification algorithm research based on ideology of inception. Opt. Technique 46, 94-100. Zhang Y., & Yan L. (2023). Face recognition algorithm based on particle swarm optimization and image feature compensation. SoftwareX, 22, 101305 Zhang F., Cai N., Wu J., Cen G., Wang H., Chen X. Image denoising method based on a deep convolution neural network. IET Image Processing. 2018;12(4):485–493. Zhang K., Huang Y., Du Y., Wang L. (2017). Facial emotion recognition based on deep evolutional spatial-temporal networks. IEEE Transactions on Image Processing. A Publication of the IEEE Signal Processing Society, 26(9):4193-4203. Zhang Z., Michael L., Michael S., Shigeru A. (1998). Comparison between geometry-based and gabor wavelet-based facial emotion recognition using multilayer perceptron.

Deep Learning Model for Automatic and Real-Time Detection and Identification of Faces from Digital Images

Abstract

Keywords

References

More Articles from INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND MATHEMATICAL THEORY