IIARD International Institute of Academic Research and Development

INTERNATIONAL JOURNAL OF COMPUTER SCIENCE AND MATHEMATICAL THEORY (IJCSMT )

E-ISSN 2545-5699
P-ISSN 2695-1924
VOL. 10 NO. 4 2024
DOI: 10.56201/ijcsmt.v10.no4.2024.pg21.52

---

Deep Learning Model for Automatic and Real-Time Detection and Identification of Faces from Digital Images

Mamza Godiya Jasini, Dr. Yusuf Musa Malgwi, Mingyi Charity Lazarus

---

Abstract

The expression on a human's face is a crucial aspect of communication and plays a significant role in conveying emotions and information visually. Although recognizing emotions from facial expressions comes naturally to humans, it presents a substantial challenge for computer algorithms. Extracting features through various image processing techniques is essential for machines to interpret emotions from images or videos. The development of an algorithm that can detect, extract, and evaluate facial expressions would enable automatic recognition of human emotions in digital images and videos. This study introduces a real-time facial emotion recognition system that utilizes a Convolutional Neural Network (CNN) to process image frames, detect faces, and recognize emotions from facial expressions. By training the CNN model on a large dataset of facial images and emotions, the system achieves accurate and rapid emotion recognition performance. The datasets utilized included FER2013 and Cohn- Kanade (CK) obtained from the Kaggle dataset.The Precision, Recall, and F-score from the CK dataset were 83.6142%, 95.0822%, and 88.9955%, respectively, while those of the FER2013 dataset were 91.8986%, 98.3649%, and 95.0218%, respectively. The developed system has the ability to rapidly detect faces in cluttered backgrounds and accurately classify emotions in real-time. Various recognition techniques were compared, and a tradeoff between accuracy and speed was evaluated for each. The results indicate that the CNN-based approach is highly effective in accurately recognizing facial emotions, with significant potential for real- world applications of facial emotion detection. The real-time implementation of the system can be used for person identification and authentication. Additionally, it can be utilized by doctors to understand the intensity of pain or illness in deaf patients.In security systems, it can identify a person regardless of their pr

keywords:

Facial Emotion Recognition, Convolutional Neural Network (CNN), Real-time,

---

References:

Avots E., Sapi?ski T., Bachmann M., Kami?ska D. Audiovisual emotion recognition in
wild. Machine Vision and Applications. 2019;30(5):975–985.

Bagwan R., Dhapudkar K., Chintawar S., &Balamwar A. (2021). Face Emotion Recogniton
using Convolutional Neural Network. Grenze International Journal of Engineering &
Technology (GIJET), 7(1).

Dahmane M., Meunier J. “Emotion recognition using grid-based HoG features”, Face and
Gesture 2011, Santa Barbara, CA, 2011.

Ekman, P. (1993). Facial expression and emotion. American psychologist, 48(4),384-392.

Ekman R (1997) What the face reveals: basic and applied studies of spontaneous expression
using the facial action coding system (FACS). Oxford University Press, New York.

EkmanP., RosenbergE. L. What the Face Reveals: Basic and Applied Studies of Spontaneous
Expression Using the Facial Action Coding System, Oxford University Press, Oxford,
UK, 2005.

Fan X.,TjahjadiT. “A spatial-temporal framework based on histogram of gradients and optical
flow for facial expression recognition in video sequences,” Pattern Recognition, vol.
48, no. 11, pp. 3407–3416, 2015.

Fan X., Tjahjadi T. Fusing dynamic deep learned features and handcrafted features for facial
expression recognition. Journal of Visual Communication and Image
Representation. 2019;65.

Gholami F., Fatemeh G. Facial Expression Recognition based on combination of the basic
facial expressionInternational Journal of Computer Science and Network
Security,Vol. 17, no. 4 (2017).

Giannopoulos P., Perikos I., Hatzilygeroudis I. (2018) Deep learning approaches for facial
emotion recognition: a case study on FER-2013. In: Hatzilygeroudis I, Palade V (eds)
Advances in hybridization of intelligent methods. Springer, Berlin, pp 1–16.

Huang Y. L., Chen S. H., Tseng H. H. Attachment avoidance and fearful prosodic emotion
recognition predict depression maintenance. Psychiatry Research. 2019;272:649–654.

Jain N., Kumar S., Kumar A., Shamsolmoali P., Zareapoor M. Hybrid deep neural networks
for face emotion recognition. Pattern Recognition Letters. 2018;115:101–106.

JanA., Ding H., MengH., ChenL., Li, H. (2018). “Accurate facial parts localization and deep
learning for 3D facial expression recognition,” in Proceedings of the 13thIEEE
International Conference on Automatic Face and Gesture Recognition. 466-472.

KanadeT., CohnJ. F.,TianY. “Comprehensive database for facial expression analysis,”
in Proceedings of the 4th IEEE International Conference on Automatic Face and
Gesture Recognition (FG '00), pp. 46–53, Grenoble, France, March 2000.

Kazemi V., Sullian J. “One millisecond face alignment with an ensemble of regression trees”,
2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH,
Kittler J., Xiao-jun W., Yun-kun L. (2019) A deep learning network for face recognition.
Journal of electronic imaging, vol. 28

Klaser A., MarszalekM., SchmidC. A spatio-temporal descriptor based on
3Dgradients, Proc. Brit. Mach. Vis. Conf. (BMVC), pp. 275, Sep. 2008

Kratzwald B., Ilic S., Kraus M., Feuerriegel S., Prendinger H. Deep learning for affective
computing: text-based emotion recognition in decision support. Decision Support
Systems. 2018;115:24–35.

Kulkarni N., Kaur S. FERFM: An enhanced facial emotion recognition system using fine-
tuned mobilenetv2 architecture. IETE journal of research, 2023.

Kumar R. K., Garain J., Kisku D. R., Sanyal G. Estimating attention of faces due to its
growing level of emotions. Proceedings of the IEEE/CVF Conference on Computer
Vision and Pattern Recognition Workshops (CVPRW); 2018; Salt Lake City, UT,
USA.

Le Ngo A.C., Oh Y.H., Phan R.C., See J. “Eulerian emotion magnification for subtle
expression recognition”, 2016 IEEE International Conference on Acoustics, Speech
and Signal Processing (ICASSP), Shanghai, 2016.

Liang D., Liang H., Yu H., Zhang Y. (2020). Deep convolutional BiLSTM fusion network for
facial expression recognition. Vis. Comput. 36, 499-508.

Liang X., Xu L., Liu J., et al. Patch attention layer of embedding handcrafted features in
CNN for facial expression recognition. Sensors. 2021;21(3): p. 833.

MehendaleN.“Facial emotion recognition using convolutional neural networks (FERC).” SN
Appl. Sci. 2, 446 (2020).

Mishra S., Prasada G. R. B., Kumar R. K., Sanyal G. Emotion recognition through facial
gestures - a deep learning approach. Proceedings of the International Conference on
Mining Intelligence and Knowledge Exploration; 2017; Cham: Springer.

PalR., MukherjeeA., MitraP.,MukherjeeJ. Modelling visual saliency using degree
centrality, IET Computer Vision, vol. 4, no. 3, pp. 218-229, 2010.

Pantic M., RothkrantzL. J. M. Automatic analysis of facial expressions: The state of the
art, IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, pp. 1424-1445, Dec.
Rajesh K.M., Naveenkumar M. “A robust method for face recognition and face detection
system using support vector machines”, 2016 International Conference on Electrical,
Electronics, Communication, Computer and Optimization Techniques (ICEECCOT),
Mysuru, 2016.

ReddyV. G., Dharma S. C., Mukherjee S. Facial expression recognition in the wild, by fusion
of deep learnt and hand-crafted features. Cognitive Systems Research. 2020;62:23–34.

RussellJ. A., Fernandez-DolsJ. M., MandlerG. The Psychology of Facial Expression,
Cambridge University Press, Cambridge, UK, 1997.

ShanC., GongS., McOwanP. W., Facial expression recognition based on local binary
patterns: A comprehensive study, Image Vis. Comput., vol. 27, no. 6, pp. 803-816,
Swinkles W., Clasen L., Xiao F., Shen H. “SVM point-based real-time emotion
detection”,2017 IEEE Conference on Dependable and SecUreComPuting, Taipei,
ViolaP., JonesM.J. Robust Real-Time Face Detection. International Journal of Computer
Vision 57, 137–154 (2004).

Wang X.H., Liang Y. C.,MaX. C. (2020). Facial expression classification algorithm research
based on ideology of inception. Opt. Technique 46, 94-100.

Zhang Y., & Yan L. (2023). Face recognition algorithm based on particle swarm optimization
and image feature compensation. SoftwareX, 22, 101305

Zhang F., Cai N., Wu J., Cen G., Wang H., Chen X. Image denoising method based on a deep
convolution neural network. IET Image Processing. 2018;12(4):485–493.

Zhang K., Huang Y., Du Y., Wang L. (2017). Facial emotion recognition based on deep
evolutional spatial-temporal networks. IEEE Transactions on Image Processing. A
Publication of the IEEE Signal Processing Society, 26(9):4193-4203.

Zhang Z., Michael L., Michael S., Shigeru A. (1998). Comparison between geometry-based
and gabor wavelet-based facial emotion recognition using multilayer perceptron.

DOWNLOAD PDF

---

Back