Recognition of facial expressions based on information from the areas of highest increase in luminance contrast

Recognition of facial expressions based on information from the areas of highest increase in luminance contrast

Автор: Vitali Babenko, Daria Alekseeva, Denis Yavna, Ekaterina Denisova, Ekaterina Kovsh, Pavel Ermakov

Журнал: International Journal of Cognitive Research in Science, Engineering and Education @ijcrsee

Рубрика: Original research

Статья в выпуске: 3 vol.10, 2022 года.

Бесплатный доступ

It is generally accepted that the use of the most informative areas of the input image significantly optimizes visual processing. Several authors agree that, the areas of spatial heterogeneity are the most interesting for the visual system and the degree of difference between those areas and their surroundings determine the saliency. The purpose of our study was to test the hypothesis that the most informative are the areas of the image of largest increase in total luminance contrast, and information from these areas is used in the process of categorization facial expressions. Using our own program that was developed to imitate the work of second-order visual mechanisms, we created stimuli from the initial photographic images of faces with 6 basic emotions and a neutral expression. These images consisted only of areas of highest increase in total luminance contrast. Initially, we determined the spatial frequency ranges in which the selected areas contain the most useful information for the recognition of each of the expressions. We then compared the expressions recognition accuracy in images of real faces and those synthe-sized from the areas of highest contrast increase. The obtained results indicate that the recognition of expressions in synthe-sized images is somewhat worse than in real ones (73% versus 83%). At the same time, the partial loss of information that occurs due to the replacing real and synthesized images does not disrupt the overall logic of the recognition. Possible ways to make up for the missing information in the synthesized images are suggested.

Еще

Expression recognition, saliency, total luminance contrast, second-order visual filters

Короткий адрес: https://sciup.org/170198686

IDR: 170198686   |   DOI: 10.23947/2334-8496-2022-10-3-37-51

Список литературы Recognition of facial expressions based on information from the areas of highest increase in luminance contrast

  • Açık, A., Onat, S., Schumann, F., Einhäuser, W., & König, P. (2009). Effects of luminance contrast and its modifications on fixation behavior during free viewing of images from different categories. Vision research, 49(12), 1541-1553. https:// doi.org/10.1016/j.visres.2009.03.011
  • Awasthi, B., Friedman, J., & Williams, M. A. (2011). Faster, stronger, lateralized: Low spatial frequency information supports face processing. Neuropsychologia, 49(13), 3583-3590. https://doi.org/10.1016/j.neuropsychologia.2011.08.027
  • Babenko, V. V., Ermakov, P. N., & Bozhinskaya, M. A. (2010). Relationship between the Spatial-Frequency Tunings of the First-and the Second-Order Visual Filters. Psikhologicheskii Zhurnal, 31(2), 48-57. Retrieved from https://www.elibrary.ru/ item.asp?id=14280688 (in Russ.)
  • Babenko, V. V., Yavna, D. V., Ermakov, P. N., & Anokhina, P. V. (2021). Nonlocal contrast calculated by the second order visual mechanisms and its significance in identifying facial emotions. F1000 Research, 10, 274. https://doi.org/10.12688/ f1000research.28396.1
  • Babenko, V., Yavna, D., Vorobeva, E., Denisova, E., Ermakov, P., & Kovsh, E. (2021). Relationship Between Facial Areas With the Greatest Increase in Non-local Contrast and Gaze Fixations in Recognizing Emotional Expressions. International Journal of Cognitive Research in Science, Engineering and Education (IJCRSEE), 9(3), 359–368. https://doi. org/10.23947/2334-8496-2021-9-3-359-368
  • Barabanshchikov, V. A. (2012). Ekspressii lits i ikh vospriyatiye [Facial expressions and their perception]. Moscow: Izdvo «IPRAN» [IPRAS Publishing House]. (in Russ.)
  • Barabanshchikov, V. A., Hoze E.G. (2013) Vospriyatiye ekspressiy spokoynogo litsa [Perception of expressions of a neutral face]. Mir psikhologii [World of Psychology], 1:203-223 Retrieved from https://www.elibrary.ru/item.asp?id=18907610 (in Russ.)
  • Becker, D. V., Neel, R., Srinivasan, N., Neufeld, S., Kumar, D., & Fouse, S. (2012). The vividness of happiness in dynamic facial displays of emotion. PLoS One, 7(1), e26551. https://doi.org/10.1371/annotation/f0519e8c-f347-4950-b7e8- 3e9cbc3ec2a9
  • Blais, C., Roy, C., Fiset, D., Arguin, M., & Gosselin, F. (2012). The eyes are not the window to basic emotions. Neuropsychologia, 50(12), 2830-2838. https://doi.org/10.1016/j.neuropsychologia.2012.08.010
  • Bombari, D., Schmid, P. C., Schmid Mast, M., Birri, S., Mast, F. W., & Lobmaier, J. S. (2013). Emotion recognition: The role of featural and configural face information. Quarterly Journal of Experimental Psychology, 66(12), 2426-2442. https://doi. org/10.1080/17470218.2013.789065
  • Bruce, N. D., & Tsotsos, J. K. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of vision, 9(3), 5-5. https://doi.org/10.1167/9.3.5
  • Butler, S., Blais, C., Gosselin, F., Bub, D., & Fiset, D. (2010). Recognizing famous people. Attention, Perception, & Psychophysics, 72(6), 1444-1449. https://doi.org/10.3758/APP.72.6.1444
  • Calder, A. J., Young, A. W., Keane, J., & Dean, M. (2000). Configural information in facial expression perception. Journal of Experimental Psychology: Human perception and performance, 26(2), 527. https://doi.org/10.1037/0096-1523.26.2.527
  • Calvo, M. G., & Beltrán, D. (2014). Brain lateralization of holistic versus analytic processing of emotional facial expressions. Neuroimage, 92, 237-247. https://doi.org/10.1016/j.neuroimage.2014.01.048
  • Cauchoix, M., Barragan-Jason, G., Serre, T., & Barbeau, E. J. (2014). The neural dynamics of face detection in the wild revealed by MVPA. Journal of Neuroscience, 34(3), 846-854. https://doi.org/10.1523/JNEUROSCI.3030-13.2014
  • Cheng, M. M., Mitra, N. J., Huang, X., Torr, P. H., & Hu, S. M. (2014). Global contrast based salient region detection. IEEE transactions on pattern analysis and machine intelligence, 37(3), 569-582. https://doi.org/10.1109/TPAMI.2014.2345401
  • Cheung, O. S., Richler, J. J., Palmeri, T. J., & Gauthier, I. (2008). Revisiting the role of spatial frequencies in the holistic processing of faces. Journal of Experimental Psychology: Human Perception and Performance, 34(6), 1327-1336. https://doi.org/10.1037/a0011752
  • Collin, C. A., Therrien, M., Martin, C., & Rainville, S. (2006). Spatial frequency thresholds for face recognition when comparison faces are filtered and unfiltered. Perception & psychophysics, 68(6), 879-889. https://doi.org/10.3758/BF03193351
  • Comfort, W. E., & Zana, Y. (2015). Face detection and individuation: Interactive and complementary stages of face processing. Psychology & Neuroscience, 8(4), 442. https://doi.org/10.1037/h0101278
  • Costen, N. P., Parker, D. M., & Craw, I. (1996). Effects of high-pass and low-pass spatial filtering on face identification. Perception & psychophysics, 58(4), 602-612. https://doi.org/10.3758/BF03213093
  • Deruelle, C., & Fagot, J. (2005). Categorizing facial identities, emotions, and genders: Attention to high-and low-spatial frequencies by children and adults. Journal of experimental child psychology, 90(2), 172-184. https://doi.org/10.1016/j. jecp.2004.09.001
  • Ekman, P. (1992). An argument for basic emotions. Cognition & emotion, 6(3-4), 169-200. https://doi. org/10.1080/02699939208411068
  • Ellemberg, D., Allen, H. A., & Hess, R. F. (2006). Second-order spatial frequency and orientation channels in human vision. Vision Research, 46(17), 2798-2803. https://doi.org/10.1016/j.visres.2006.01.028
  • Flevaris, A. V., & Robertson, L. C. (2016). Spatial frequency selection and integration of global and local information in visual processing: A selective review and tribute to Shlomo Bentin. Neuropsychologia, 83, 192-200. https://doi.org/10.1016/j. neuropsychologia.2015.10.024
  • Frey, H. P., König, P., & Einhäuser, W. (2007). The role of first-and second-order stimulus features for human overt attention. Perception & Psychophysics, 69(2), 153-161. https://doi.org/10.3758/BF03193738
  • Frischen, A., Eastwood, J. D., & Smilek, D. (2008). Visual search for faces with emotional expressions. Psychological bulletin, 134(5), 662-676. https://doi.org/10.1037/0033-2909.134.5.662
  • Gao, Z., & Bentin, S. (2011). Coarse-to-fine encoding of spatial frequency information into visual short-term memory for faces but impartial decay. Journal of Experimental Psychology: Human Perception and Performance, 37(4), 1051-1064. https://doi.org/10.1037/a0023091
  • Goffaux, V. (2009). Spatial interactions in upright and inverted faces: Re-exploration of spatial scale influence. Vision research, 49(7), 774-781. https://doi.org/10.1016/j.visres.2009.02.009
  • Goffaux, V., & Rossion, B. (2006). Faces are” spatial”--holistic face perception is supported by low spatial frequencies. Journal of Experimental Psychology: Human perception and performance, 32(4), 1023-1039. https://doi.org/10.1037/0096- 1523.32.4.1023
  • Goffaux, V., Peters, J., Haubrechts, J., Schiltz, C., Jansma, B., & Goebel, R. (2011). From coarse to fine? Spatial and temporal dynamics of cortical face processing. Cerebral Cortex, 21(2), 467-476. https://doi.org/10.1093/cercor/bhq112
  • Gold, J. M., Mundy, P. J., & Tjan, B. S. (2012). The perception of a face is no more than the sum of its parts. Psychological science, 23(4), 427-434. https://doi.org/10.1177/0956797611427407
  • Gold, J., Bennett, P. J., & Sekuler, A. B. (1999). Identification of band-pass filtered letters and faces by human and ideal observers. Vision research, 39(21), 3537-3560. https://doi.org/10.1016/S0042-6989(99)00080-2
  • Gosselin, F., & Schyns, P. G. (2001). Bubbles: a technique to reveal the use of information in recognition tasks. Vision research, 41(17), 2261-2271. https://doi.org/10.1016/S0042-6989(01)00097-9
  • Graham, N. V. (2011). Beyond multiple pattern analyzers modeled as linear filters (as classical V1 simple cells): Useful additions of the last 25 years. Vision research, 51(13), 1397-1430. https://doi.org/10.1016/j.visres.2011.02.007
  • Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat’s striate cortex. The Journal of physiology, 148(3), 574-591. https://doi.org/10.1113/jphysiol.1959.sp006308
  • Jack, R. E., Garrod, O. G., Yu, H., Caldara, R., & Schyns, P. G. (2012). Facial expressions of emotion are not culturally universal. Proceedings of the National Academy of Sciences, 109(19), 7241-7244. https://doi.org/10.1073/pnas.1200155109
  • Jennings, B. J., & Yu, Y. (2017). The role of spatial frequency in emotional face classification. Attention, Perception, & Psychophysics, 79(6), 1573-1577. https://doi.org/10.3758/s13414-017-1377-7
  • Kumar, D., & Srinivasan, N. (2011). Emotion perception is mediated by spatial frequency content. Emotion, 11(5), 1144-1151. https://doi.org/10.1037/a0025453
  • Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D. H., Hawk, S. T., & Van Knippenberg, A. D. (2010). Presentation and validation of the Radboud Faces Database. Cognition and emotion, 24(8), 1377-1388. https://doi.org/10.1080/02699930903485076
  • Leder, H., & Bruce, V. (2000). When inverted faces are recognized: The role of configural information in face recognition. The quarterly journal of experimental psychology Section A, 53(2), 513-536. https://doi.org/10.1080/713755889
  • Lee, H. S., & Kim, D. (2008). Expression-invariant face recognition by facial expression transformations. Pattern recognition letters, 29(13), 1797-1805. https://doi.org/10.1016/j.patrec.2008.05.012
  • Li, G., Yao, Z., Wang, Z., Yuan, N., Talebi, V., Tan, J., ... & Baker, C. L. (2014). Form-cue invariant second-order neuronal responses to contrast modulation in primate area V2. Journal of Neuroscience, 34(36), 12081-12092. https://doi. org/10.1523/JNEUROSCI.0211-14.2014
  • Liu, L., & Ioannides, A. A. (2010). Emotion separation is completed early and it depends on visual field presentation. PloS one, 5(3), e9790. https://doi.org/10.1371/journal.pone.0009790
  • Lobmaier, J. S., & Mast, F. W. (2007). Perception of novel faces: The parts have it!. Perception, 36(11), 1660-1673. https://doi. org/10.1068/p5642
  • Lundqvist, D., Flykt, A., & Öhman, A. (1998). The Karolinska directed emotional faces (KDEF). CD ROM from Department of Clinical Neuroscience, Psychology section, Karolinska Institutet, 91(630), 2-2. https://doi.org/10.1037/t27732-000
  • Marat, S., Rahman, A., Pellerin, D., Guyader, N., & Houzet, D. (2013). Improving visual saliency by adding ‘face feature map’and ‘center bias’. Cognitive Computation, 5(1), 63-75. https://doi.org/10.1007/s12559-012-9146-3
  • Maurer, D., Le Grand, R., & Mondloch, C. J. (2002). The many faces of configural processing. Trends in cognitive sciences, 6(6), 255-260. https://doi.org/10.1016/S1364-6613(02)01903-4
  • McKone, E. (2008). Configural processing and face viewpoint. Journal of Experimental Psychology: Human Perception and Performance, 34(2), 310-327. https://doi.org/10.1037/0096-1523.34.2.310
  • Morawetz, C., Baudewig, J., Treue, S., & Dechent, P. (2011). Effects of spatial frequency and location of fearful faces on human amygdala activity. Brain research, 1371, 87-99. https://doi.org/10.1016/j.brainres.2010.10.110
  • Näsänen, R. (1999). Spatial frequency bandwidth used in the recognition of facial images. Vision research, 39(23), 3824-3833. https://doi.org/10.1016/S0042-6989(99)00096-6
  • Oliva, A., & Schyns, P. G. (1997). Coarse blobs or fine edges? Evidence that information diagnosticity changes the perception of complex visual stimuli. Cognitive psychology, 34(1), 72-107. https://doi.org/10.1006/cogp.1997.0667
  • Olszanowski, M., Pochwatko, G., Kuklinski, K., Scibor-Rylski, M., Lewinski, P., & Ohme, R. K. (2015). Warsaw set of emotional facial expression pictures: a validation study of facial display photographs. Frontiers in psychology, 5, 1516. https://doi. org/10.3389/fpsyg.2014.01516
  • Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005, July). Web-based database for facial expression analysis. In 2005 IEEE international conference on multimedia and Expo (pp. 5-pp). IEEE. https://doi.org/10.1109/ICME.2005.1521424
  • Perazzi, F., Krähenbühl, P., Pritch, Y., & Hornung, A. (2012, June). Saliency filters: Contrast based filtering for salient region detection. In 2012 IEEE conference on computer vision and pattern recognition (pp. 733-740). IEEE. https://doi. org/10.1109/CVPR.2012.6247743
  • Peyrin, C., Michel, C. M., Schwartz, S., Thut, G., Seghier, M., Landis, T., ... & Vuilleumier, P. (2010). The neural substrates and timing of top–down processes during coarse-to-fine categorization of visual scenes: A combined fMRI and ERP study. Journal of cognitive neuroscience, 22(12), 2768-2780. https://doi.org/10.1162/jocn.2010.21424
  • Piepers, D. W., & Robbins, R. A. (2012). A review and clarification of the terms “holistic,”“configural,” and “relational” in the face perception literature. Frontiers in psychology, 3, 559. https://doi.org/10.3389/fpsyg.2012.00559
  • Pourtois, G., Dan, E. S., Grandjean, D., Sander, D., & Vuilleumier, P. (2005). Enhanced extrastriate visual response to bandpass spatial frequency filtered fearful faces: Time course and topographic evoked-potentials mapping. Human brain mapping, 26(1), 65-79. https://doi.org/10.1002/hbm.20130
  • Royer, J., Blais, C., Charbonneau, I., Déry, K., Tardif, J., Duchaine, B., ... & Fiset, D. (2018). Greater reliance on the eye region predicts better face recognition ability. Cognition, 181, 12-20. https://doi.org/10.1016/j.cognition.2018.08.004
  • Sakai, K., & Finkel, L. H. (1995). Characterization of the spatial-frequency spectrum in the perception of shape from texture. JOSA A, 12(6), 1208-1224. https://doi.org/10.1364/JOSAA.12.001208
  • Shaw, K., Lien, M. C., Ruthruff, E., & Allen, P. A. (2011). Electrophysiological evidence of emotion perception without central attention. Journal of Cognitive Psychology, 23(6), 695-708. https://doi.org/10.1080/20445911.2011.586624
  • Smith, F. W., & Schyns, P. G. (2009). Smile through your fear and sadness: Transmitting and identifying facial expression signals over a range of viewing distances. Psychological Science, 20(10), 1202-1208. https://doi.org/10.1111/j.1467- 9280.2009.02427.x
  • Smith, M. L., Cottrell, G. W., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions. Psychological science, 16(3), 184-189. https://doi.org/10.1111/j.0956-7976.2005.00801.x
  • Smith, M. L., Volna, B., & Ewing, L. (2016). Distinct information critically distinguishes judgments of face familiarity and identity. Journal of Experimental Psychology: Human Perception and Performance, 42(11), 1770-1779. https://doi.org/10.1037/ xhp0000243
  • Solomon, J.A., & Morgan, M.J. (2017). Orientation-defined boundaries are detected with low efficiency. Vision Research, 138, 66-70. https://doi.org/10.1016/j.visres.2017.06.009
  • Stein, T., Seymour, K., Hebart, M. N., & Sterzer, P. (2014). Rapid fear detection relies on high spatial frequencies. Psychological science, 25(2), 566-574. https://doi.org/10.1177/0956797613512509
  • Sun, P., & Schofield, A. J. (2011). The efficacy of local luminance amplitude in disambiguating the origin of luminance signals depends on carrier frequency: Further evidence for the active role of second-order vision in layer decomposition. Vision research, 51(5), 496-507. https://doi.org/10.1016/j.visres.2011.01.008
  • ‘t Hart, B.M., Schmidt, H.C.E.F., Roth, C., & Einhäuser, W. (2013). Fixations on objects in natural scenes: dissociating importance from saliency. Frontiers in Psychology, 4.- Article 455.- 9p. https://doi.org/10.3389/fpsyg.2013.00455
  • Tanaka, J. W., Kaiser, M. D., Butler, S., & Le Grand, R. (2012). Mixed emotions: Holistic and analytic perception of facial expressions. Cognition & Emotion, 26(6), 961-977. https://doi.org/10.1080/02699931.2011.630933
  • Tanskanen, T., Näsänen, R., Montez, T., Päällysaho, J., & Hari, R. (2005). Face recognition and cortical responses show similar sensitivity to noise spatial frequency. Cerebral Cortex, 15(5), 526-534. https://doi.org/10.1093/cercor/bhh152
  • Vlamings, P. H., Goffaux, V., & Kemner, C. (2009). Is the early modulation of brain activity by fearful facial expressions primarily mediated by coarse low spatial frequency information?. Journal of vision, 9(5), 1-13. https://doi.org/10.1167/9.5.12
  • Vuilleumier, P., & Pourtois, G. (2007). Distributed and interactive brain mechanisms during emotion face perception: evidence from functional neuroimaging. Neuropsychologia, 45(1), 174-194. https://doi.org/10.1016/j.neuropsychologia.2006.06.003
  • Vuilleumier, P., Armony, J. L., Driver, J., & Dolan, R. J. (2003). Distinct spatial frequency sensitivities for processing faces and emotional expressions. Nature neuroscience, 6(6), 624-631. https://doi.org/10.1038/nn1057
  • White, M. (2000). Parts and wholes in expression recognition. Cognition & Emotion, 14(1), 39-60. https://doi. org/10.1080/026999300378987
  • Willis, J., & Todorov, A. (2006). First impressions: Making up your mind after a 100-ms exposure to a face. Psychological science, 17(7), 592-598. https://doi.org/10.1111/j.1467-9280.2006.01750.x
  • Wu, J., Qi, F., Shi, G., & Lu, Y. (2012). Non-local spatial redundancy reduction for bottom-up saliency estimation. Journal of Visual Communication and Image Representation, 23(7), 1158-1166. https://doi.org/10.1016/j.jvcir.2012.07.010
  • Xia, C., Qi, F., Shi, G., & Wang, P. (2015). Nonlocal center–surround reconstruction-based bottom-up saliency estimation. Pattern Recognition, 48(4), 1337-1348. https://doi.org/10.1016/j.patcog.2014.10.007
  • Yarbus, A. L. (2013). Eye movements and vision. Springer. https://doi.org/10.1007/978-1-4899-5379-7
  • Yavna, D. V. (2012). Psikhofiziologicheskiye osobennosti zritel’nogo vospriyatiya prostranstvenno modulirovannykh priznako [Psychophysiological features of visual perception of spatially modulated features]. PhD Thesis. Rostov-on-Don (in Russ.)
Еще
Статья научная
SciUp 2024 © 2024 ©