Understanding emotional responses to text: a psycholinguistic investigation of visual influences

DOI:

Polycode texts are used in various forms of communication, particularly intensively in advertising and educational practices. It is commonly believed that adding color to educational literature helps avoid monotony and boredom. Illustrating texts, aside from its inherent decorative and “entertaining” function, also enhances the perception of verbal material by employing dual coding of information – both verbal and visual. The assertion that emotions rule the world has become commonplace in the 21st century. Everyone knows that if you cannot change the situation, you can change your attitude towards it. The ability to control one’s emotions is of great interest to many, and is often utilized by organizers of various courses and coaching programs, offering their products. However, there is another side to this issue – the ability to control the emotions of recipients through the creation of messages with specific parameters. The potential for manipulating public opinion is well-known to all. However, few understand exactly how this happens. One of the most covert methods of manipulation is the creolization of verbal text – its illustration. Readers, when introduced to verbal text, perceive illustrations uncritically, unaware that the image can alter the perception of the verbal text. This assertion is based on the interpretation of the results of several series of experiments, during which a change in the perception of verbal text under the influence of illustration was observed [Vashunina, 2007; Kreolizovannyy tekst..., 2020]. Thus, the purpose of this paper is to describe the influence of illustrations on the perception of verbal text and to explore how specific visual parameters affect emotional responses to textual content.

Theoretical framework

This study is grounded in the linguistic theory of emotions and the theory of perception of creolized text. The essence of the linguistic theory of emotions, developed by V.I. Shakhovsky, can be summarized as follows: humans reflect the world passionately and selectively, relying on their emotional states. Emotions also act as intermediaries between the objective world and its reflection in language, capturing the emotional significance of objects for humans. Emotional experiences expressed in words and statements are understood by all speakers of a specific language. This is because emotions are “sociologized and psychologized” [Shakhovsky, 2008, p. 5] within the linguistic community, serving not only as a tool for evaluating the surrounding environment but also constituting a significant component of this environment and the worldview. In this context, emotiveness, as a linguistic category, is an integral property of language for expressing psychological

(emotional) states and experiences of individuals through specific linguistic units and speech constructs – emotives . Emotives are further classified based on the extent of reflected emotions in their semantics, the modus of emotiveness, the type of emotive semantics, and the parameter of explicit or implicit expression.

At present, emotiology is being developed within a number of theoretical approaches in Russia (see: [Babenko, 2022; Zhelvis, 2001; Ionova, 2022; Krasavskiy, 2008; Lukyanova, 2015; Shakhovsky, 2010; etc.]) and internationally (see: [Absattar, Mambetova, Zhubay, 2022; Chentsova-Dutton, 2020; Jackson et al., 2019; Lindquist, 2021; Ponsonnet, 2022; Pritzke, Fenigsen, Wilce (eds.), 2020; Wierzbicka, Harkins, 2001; Wierzbicka, 2010; etc.]).

Emotional processes are fixed by means of a psycholinguistic mechanism, which reflects these processes in the semantics of certain words used to verbalize emotional states. The reflection of an emotional state in specific words manifests itself in the semantic structure of these words, which essentially forms the emotiveness of the word and its ability to reproduce specific emotional relationships depending on the context. The emotion in a word is preserved as an idea about it, which can “come to life,” reflecting the corresponding emotion experienced by a person at the moment. Emotions penetrate into words, embed themselves in them, persist, and, when necessary, manifest, express themselves, and are recognized through these words. Thus, certain words can have an inherent “emotional force,” determined by their meaning, associations, or usage in a specific context. The same principles apply to the dependence of the emotional impact of texts on the vertical context.

The second pivotal point of the study is the theory of polycode text perception. Through numerous experiments detailed in [Vashunina, 2007; Kreolizovannyy tekst..., 2020], it was established that illustration consistently alters the perception of a verbal text. This conclusion was drawn from the analysis of respondents’ evaluations given to verbal texts and texts with various illustrations (polycode texts). The experimental method employed was the semantic differential method, wherein participants were asked to evaluate objects using unipolar or bipolar scales of a specified dimension (usually five- or seven-point scales). Evaluations given by respondents to different objects were then compared. For instance, we compared the assessment of a purely verbal text (without illustration) with the assessment of the same verbal text but with the addition of illustrations. The change in the evaluation of the verbal text depends not merely on the fact of adding an illustration, but on the parameters of the image: content, composition, form, color. Varying individual parameters of the illustration in many cases allows us to determine which characteristic of the image specifically causes the change in the text perception and evaluation. If a change in evaluation, influenced by a specific parameter of the image, is identified for texts of different content and style, it seems logical to assume that by using this parameter, we can regulate the perception of the text along a particular scale.

Experiment description

In this research, participants evaluated the suggested objects using scales created on the basis of a preliminary associative experiment. During this experiment, subjects were asked to provide characteristics for both texts and images. The frequency characteristics became the names of the scales. Additionally, a standard semantic differential (SD) was employed. Since the SD is associated with connotative aspects of meaning, it offers the opportunity to apply the SD method to the study of visual objects and polycode texts. This is due to the fact that these objects predominantly appeal to the connotative, emotional reactions of communication participants. These methods allow for the construction of semantic spaces and the identification of perception criteria that may not be consciously recognized by individuals. They also help categorize the presented objects. The data used for this paper were obtained from a series of previously conducted experiments. These data were processed using Principal Component Analysis and Varimax with Kaiser Normalization programs.

This is a brief overview of the experiments that progressively explored the semantic perception of creolized texts: in the first stage, the perception of texts with a single verbal component but varying visual components was assessed; in the second stage, evaluations of different creolized texts with the same verbal component were compared; in the third stage, the image parameters that presumably caused the differing evaluations of the texts with identical verbal components were identified. The first series of experiments was fully described in [Vashunina, 2007]).

The experimental material consisted of 30 polycode texts in the Russian language on the topics “The Urals” and “A Snake” with various verbal components (in scientific-popular style and literary style: prose and poetry) and iconic components (two realistic images – graphic and pictorial, and three unrealistic ones – in natural, minor, and major color palettes), created according to specified parameters by students of an art school. Encyclopedic scientific-popular texts had to be selectively condensed (by removing excessive details and numerical data), while excerpts from fictional works were taken unchanged, within the volume allowed by the chosen format. Scientific-popular texts were compiled according to the requirements for educational materials, based on the processing of encyclopedic articles, and can be used in geography and zoology textbooks. The fictional texts include an original poem by the Russian poet I.A. Bunin (taken unchanged) and an excerpt from a story by T. Mayne Reid in Russian translation. Both texts can be used in literature and stylistics classes. Five illustrations were made for each of the six texts by students of an art school. Two of them were drawn in a realistic style. The other three are symbolic or contain symbolic elements. When creating these illustrations, the artists were not limited in their choice of styles, so illustrations of different styles were selected for different texts. Realistic illustrations consist of a graphic black-and-white drawing and a painted drawing (based on the graphic), created using natural colors. The basis of the three fantasy illustrations is also one drawing, executed in three variants: in black and white, in a cold color scheme, and in a warm color scheme.

The texts were evaluated using 50 semantic differential scales. The scales were the result of experimental selection. A group of subjects (totaling about two hundred people) was asked to characterize the verbal texts we selected using any definitions. Then the most frequently encountered characteristics were identified.

Using synonym dictionaries, the main member of the synonymic row was identified, which formed the basis of the scale. If an antonym was encountered during the characterization of the texts, a ready-made scale was obtained (for example, anxious – calm ). In the absence of an oppositional member in the subjects’ definitions, we resorted to the help of antonym dictionaries (Lvov, 1984). Since the scales were selected based on the definitions proposed by the subjects, some scales of classical SD were not included in the general list. It can be assumed that they are not relevant for the six selected texts. Approximately 1,500 native Russian-speaking students from various faculties at a pedagogical university (with at least 100 respondents for each text) were asked to read the texts and rate them using semantic differential scales.

The second series of experiments, the results of which are detailed in [Kreolizovannyy tekst..., 2020], involved four polycode texts, composed of images and verbal descriptions of of the featured images. A suprematist composition by K. Malevich and photographs of a city street, colored in minor (blue-lilac) and major (orange-pink) palettes, were used as iconic components in the polycode text. The verbal content is a compilative text composed by the experimenter from descriptions of the nonverbal part provided by the participants. The texts were evaluated using 34 semantic differential scales. The scales used by the participants to assess the content of the non-verbal part were taken from descriptions compiled by the participants themselves, relying on the images of their consciousness. The experiment involved 25 respondents for each text, residing in Moscow, aged 18–24 years old. The obtained total matrices were processed in the statistical package (SPSS) using factor analysis with subsequent rotation of factors to achieve a simple structure. During the experiment, the difference in semantic perception of the image, non-narrative text – description of this image, and creolized text composed of these two components was evaluated. The results can help creators of educational materials assess the pragmatic potential of illustrating descriptive texts.

The third series of experiments was described in detail in [Kreolizovannyy tekst..., 2020]). The experimental material included polycode texts (8), composed of four fictional prose verbal texts in Russian

(light-hearted and dark-sad in terms of emotional and semantic dominance) and two photographs of a street painted in blue-lilac (minor) and orange-yellow (major) tones. The verbal texts ranged from 270 to 300 characters and were excerpts from works of fiction. The texts were evaluated using 34 SD scales. The same scales as in series 2 were used in this series of experiments. This decision was made due to the need for a basis for comparison, which requires the presence of identical scales. The method of data processing and the number of participants were the same as in series 2. The results of the experiment can be used to predict changes in the perception of a literary text as a result of adding an illustration to it.

For this paper, the results connected with the emotional perception of the polycode texts are used. This material has not been previously analyzed or published. Some of the scales used in the experiments with semantic differentials assess the emotional perception of the received message. Several semantic differential scales coincide with the terms of emotions or emotional states (Anxiety – Anxious, Interest – Interesting, Sadness – Sad, Joy – Joyful), meaning that the evaluation on them fully reflects the recipients’ perception of the text in the corresponding emotional key. The remaining scales name specific aspects of emotions. For instance, studying the changes in perception of educational text in these aspects can help create an effective educational resource by arousing interest and joy, avoiding the escalation of sadness, and regulating arousal. Let us clarify the basis on which certain scales were considered as evaluating some emotional state. Different explanatory dictionaries were used to analyze the meanings of the lexical units, from which the definitions facilitating comparisons were selected.

Stress / stress is defined as “a state of increased nervous tension, overstrain caused by some strong impact” (Ozhegov, 2007). As evident from the definition, one of the semes is ‘tension,’ so the quality referred to as the tense scale correlates with the emotional state of stress. When evaluating on bipolar scales, an antonym is chosen for the quality, resulting in the tense – relaxed scale. Anxiety is considered the first stage of stress [Metody..., 2020], based on this, we will consider the quality anxious as correlating with stress. By adding the antonym, we get the anxious – calm scale.

Hatred / nenavist is “a feeling of strong enmity, malice” (Ozhegov, 2007), and the corresponding qualities correlating with it would be hostile and malicious. By adding antonyms, we obtain the scales hostile – friendly, malicious – kind .

Anger / gnev is “a feeling of strong disapproval, indignation; a state of irritation, resentment” (Ozhegov, 2007). The corresponding quality expressed by an adjective is angry (hence, we get the scale angry – kind ). Since anger is defined through other emotions, let us consider their definitions. Indignation / negodovanie is “outrage, extreme dissatisfaction” (Ushakov, 2008). Considering that a component of dissatisfaction is “negative attitude towards something” (Ushakov, 2008), we introduce the correlating quality negative , and thus, the scale negative – positive .

The state provoked by anger, including indignation, irritation, and resentment, when applied to a person, is hostility, which can escalate into aggression. On this basis, we include the quality hostile as correlating with the emotion of anger (thus, the scale hostile – friendly ), as well as the scale aggressive (which can be considered a peripheral correlate, as anger, while often, but not necessarily is associated with aggression).

Satisfaction / udovletvorenie is “the feeling of someone satisfied with the fulfillment of their aspirations, desires, needs” (Ozhegov, 2007). We believe that the evaluation of this fulfillment can be positive, and accordingly, the correlating scales will be negative – positive , bad – good .

Compassion / sostradanie is “a feeling of pity, sorrow, caused by someone’s misfortune, grief, experiencing someone else’s grief as one’s own misfortune” (Ozhegov, 2007). One of the meanings, according to the definition, is “someone else’s as one’s own,” which can be denoted by the adjective close (thus, the scale distant – close ). Also, based on the definition of the word kindness / dobrota (“responsiveness, a heartfelt disposition toward people, a desire to do good to others” (Ushakov, 2008) we relate compassion to kindness (the scale bad – good ).

One of the meanings of the word love / lubov’ – “a feeling of deep affection, selfless and sincere attachment” (Ushakov, 2008) – can be correlated with the characteristic close (the distant – close scale), based on the definition of attachment as a feeling of “closeness, based on loyalty, sympathy to someone or something” (Ushakov, 2008). “Deep emotional attraction, strong heartfelt feeling” (Ushakov, 2008) – another meaning of the word – is related to the type of love – romantic love, so the scale romantic is considered correlating.

Surprise / udivlenie is “a state caused by a strong impression of something unusual, unexpected, strange, incomprehensible; amazement” (Ushakov, 2008) – associated with the perception of the unusual and incomprehensible, so the scales ordinary – unusual , trivial – original , incomprehensible – understandable , natural , commonplace correlate with it.

Offense / obida is “unfairly inflicted sorrow, offense, as well as the feeling caused by it” (Ushakov, 2008), and one of the meanings is injustice , which is reflected in the scale unjust – just .

Contempt / prezrenie is “such an attitude towards someone or something that is caused by recognizing someone or something as unworthy, undeserving of respect, vile, morally low, insignificant” (Ushakov, 2008). We considered it possible to correlate such characteristics as primitive – “not deep enough, too simplified; morally undeveloped” (Ushakov, 2008) and unnecessary – “such that there is no need, necessity; superfluous, useless” (Ozhegov, 2007) with this emotion, as both of these adjectives are related to the concept insignificant .

Pleasure / udovol’stvie is “a feeling of joy from pleasant sensations, experiences, thoughts” (Lvov, 1 984). In accordance with this interpretation, the emotion of pleasure correlates with the qualities pleasant , attractive , beautiful (accordingly, the scales pleasant – unpleasant , beautiful – ugly , attractive – repulsive ).

Disgust / otvrashchenie is an “extremely unpleasant feeling caused by someone or something” (Lvov, 1984). Characteristics such as unpleasant (negative pole of the scale pleasant – unpleasant ), repulsive – “inspiring dislike, repulsion towards oneself” (Lvov, 1984), and ugly – “producing a repulsive, unpleasant impression with its appearance” (Lvov, 1984) are related to this emotion. Thus, the corresponding scales are unpleasant – pleasant , ugly – beautiful , repulsive – attractive .

The emotion of sadness / pechal’ has correlating characteristics sad and dark (one of the meanings is “sad, gloomy, joyless” (Lvov, 1984)), corresponding to the scales sad – happy , dark – bright .

The emotion of joy / radost’ is associated with characteristics joyful and bright (in the sense of “joyful, unclouded, pleasant” (Lvov, 1984)), with the scales joyful – sad , bright – dark .

If we consider the correlation of the evaluative scales with the emotional states, the following picture emerges (see Table 1). The left column lists the terms of emotions and emotional states, and the right column contains the scales that allow assessing the degree of expression of this emotion or state in the text during its perception by the recipient. In one part of the experiments, bipolar scales were used, while in another part, unipolar scales were used (they are italicized in the table). In bipolar scales, the first position lists the quality relevant to the corresponding emotion, and the second position lists its antonym. For example, for the emotion of pleasure, the scale looks like pleasant – unpleasant , and for the emotion of disgust, it looks like unpleasant – pleasant . Unipolar scales can represent both the quality relevant to the expression of the emotion and its antonym (antonyms are underlined).

Some bipolar scales can be considered as contrasting emotions and emotional states (see Table 2). Primarily, these are scales that coincide for opposing emotions in Table 1, where the actualization of different poles is implied. However, there is an exception – the scales picturesque and exquisite , related to the emotion Pleasure , do not correlate with the emotion Disgust , which is opposed to Pleasure , because non-picturesque and non-exquisite stimuli are unlikely to evoke disgust. This is explained by the fact that the opposition of emotions is somewhat conventional, as each emotion is a unique human reaction, and as a result, emotions are only to a limited extent subject to structured representation.

In addition to the oppositions presented in Table 2, some scales, as evident from Table 1, correlate with multiple emotions. In anxious – calm , calm relates to Stress and Anxiety . The hostile – friendly scale associates with Hatred , Anger . Negative – positive connects with Anger , Satisfaction . Distant – close corresponds to Compassion and Love .

Results

As seen from Table 1, the established correlations between emotions and emotional states with the semantic differential scales cover

Table 1. Correlation between semantic differential scales and emotions/emotional states

№	Emotion/ emotional state	Semantic differential scale
1	Stress	Exciting – Calming Anxious – Calm Calm
2	Anxiety	Anxious – Calm Calm
3	Hatred	Hostile – Friendly Angry – Kind
4	Anger	Hostile – Friendly Negative – Positive Angry – Kind Aggressive
5	Satisfaction	Positive – Negative Good – Bad
6	Compassion	Close – Distant Kind – Angry
7	Love	Close – Distant Romantic
8	Surprise	Unusual – Ordinary Original – Banal Natural Understandable Ordinary
9	Interest	Interesting – Uninteresting
10	Resentment	Unjust – Just
11	Contempt	Unnecessary – Necessary Primitive
12	Pleasure	Pleasant – Unpleasant Beautiful – Ugly Attractive – Repulsive Pleasant Exquisite Picturesque
13	Disgust	Unpleasant – Pleasant Ugly – Beautiful Repulsive – Attractive Pleasant
14	Sadness	Sad – Happy Dark – Bright
15	Joy	Happy – Sad Bright – Dark

Table 2. Opposition of emotions through semantic differential scales

Emotions/ emotional states	Scales
Hatred – Compassion	Hostile – Kind
Disgust – Pleasure	Unpleasant – Pleasant, Ugly – Beautiful, Repulsive – Attractive, Pleasant
Sorrow – Joy	Sad – Happy, Dark – Bright

various aspects of text content. Since visual characteristics have been identified during the experiments that influence the evaluation of the creolized text according to the listed scales with a certain probability, it is possible to forecast changes in the semantic perception of the text when using these visual characteristics.

Let us consider the possibilities of adjusting the emotional state of Stress by varying the image visual characteristics, which change the text evaluation on the scales exciting – calming and anxious – calm. The latter scale also regulates the perception of the emotional state of Anxiety.

Exciting – calming. The perception of the text as exciting is heightened by the brightness of the image – the brighter the illustration, the higher the assessment of the text as exciting is. Regarding the color palette, more often it involves colors from the major scale, although the use of saturated bright colors from the minor scale is possible. The calming quality is associated with the paleness of the image in the minor color palette.

The perception of the text as calm is associated with realistic illustrations and the clarity of the image. Complex identifiability of objects, ambiguity, and originality “cause anxiety” (as with everything unknown and unclear). However, objects do not necessarily have to be unclear. As the results of one experiment showed, the use of pale graphics (black-and-white graphic illustration) also complicates the immediate recognition of the objects in the illustration and leads to an increase in the evaluation of the text as anxious. Additionally, an increase in the evaluation of the text as calm occurs when using deep blue color and rounded geometric shapes, while angular sharp forms in the illustration elevate the evaluation of the text as anxious.

Let us consider the possibilities of altering the perception of the emotion Hatred by modifying the text evaluation on the scales of hostile – friendly , evil – kind . Sharp objects and forms are perceived as evil and hostile. Our analysis revealed colors and color combinations in illustrations that elevated the text evaluation as hostile and/or evil : purple with a dirty hue, burgundy spots on green, on blue-purple, black with yellow and blue. Similarly, in some cases, the black-and-white color scheme (graphic illustrations) had the same effect. Texts with unpleasant recipient images (in our analysis, these were images of snakes) were also assessed as more hostile. In other words, hostile and evil are associated with sharp, black, unpleasant. The increase in text evaluations as kind and friendly may be linked to colors of the major scale, as well as explained by the content of the image: a realistic depiction of an object that is perceived as kind and friendly .

A high rating on the aggressive scale was observed when using bright red color, as well as when the image conveyed a sense of attack (in our analysis, an image with a sensation of falling and pseudo-motion of a large object towards the viewer). Texts with illustrations in a realistic style and natural color scheme (without “aggressive” objects) were perceived as least aggressive. An interesting trend was discovered during the analysis: texts with illustrations in an unnatural color scheme were rated as more aggressive compared to texts in a natural scheme. We hypothesized that imposing one’s vision is perceived as aggression, a notion that should be further explored.

Compassion is associated with qualities of kind – evil (see above), close – distant . Warm colors create a sense of closeness, while cold colors evoke a sense of distance. Black-and-white (graphic) illustrations are also perceived as more distant . The visual representation of closeness is also linked to the presence of this quality in the depicted object itself, in which case the rating increases for texts with realistic illustrations in natural colors.

Love, in addition to the close – distant scale, correlates with the romantic scale, where the use of pink color increases the rating, while the suprematist composition as an illustration decreases it.

Surprise is associated with ratings on the scales unusual – ordinary , original – banal , natural , understandable , common .

Texts with unconventional illustrations (fantastical, in unnatural color schemes) are perceived as the most unusual and original , while texts with realistic illustrations (especially graphic ones) are considered ordinary and commonplace .

Texts with realistic illustrations are also seen as the most natural and understandable , whereas texts with fantastical illustrations receive the lowest ratings on these scales.

The emotion of Interest correlates directly with the interesting – uninteresting scale. In our analysis, texts with realistic images were often rated as more interesting , especially if the object depicted did not evoke aversion. Texts with unpleasant recipient images or with pale graphic illustrations tend to be rated as less interesting .

The emotion of Offense correlates with the unfair – fair scale. Ratings on this scale in approximately half of the cases can be explained based on image parameters. In other cases, correlations with image form are not evident, likely because fairness is a complex ethical concept weakly associated with external form. The use of color schemes that do not match the image content, as well as the complexity and some ambiguity of the illustration, may lead to higher ratings of text as unfair. Texts with realistic illustrations and illustrations whose color scheme matches the image content are often rated as more fair.

The emotion of Contempt/Disdain is related to the scales of unnecessary-necessary and primitive. Texts with images that evoke dislike, as well as those with graphic illustrations, are perceived as more unnecessary. Texts with realistic illustrations are often considered more necessary.

The texts with trivial (realistic) illustrations are rated as more primitive.

The emotions of Pleasure and Disgust are somewhat antonyms, therefore they correlate with different poles of the same scales: pleasant – unpleasant , beautiful – ugly , attractive – repulsive .

The rating of a text as pleasant increases when combined with a realistic image, provided the object does not evoke dislike. Conversely, the rating of a text as unpleasant increases, which also occurs in some cases with the use of an unnatural or disharmonious color palette, as well as dark, gloomy tones. In some instances, texts with unconventional images are rated as the most pleasant . Generally, texts with a dark-sad verbal component are perceived as less pleasant than texts with a light-cheerful verbal component. Regarding specific colors or color palettes, it is difficult to determine which coloristic solution will be more pleasant to recipients. This is related to the specificity of perceiving a creolized text: a fusion, a synthesis of components occurs. In our material, there were instances where a higher rating as pleasant was given to an illustration in a major color palette, while the most pleasant text was rated in combination with an illustration in a minor color palette.

The evaluation of texts as attractive did not show unequivocal results. In half of the cases, texts with realistic images were found most pleasant . Similarly, texts with illustrations in a major color scheme are often rated as more pleasant . The same image parameters increase the assessment of the text as beautiful . Texts with illustrations that cause aversion in content and form, as well as with black-and-white (graphic) illustrations, are often rated as more repelling (the latter parameter also increases the assessment of the text as ugly ).

The emotion of pleasure is also associated with the scales of exquisite and picturesque .

In the analysis of ratings on the exquisite scale, it was found that they increase with the addition of illustrations in a major color scheme. A decrease in ratings was observed when a suprematist composition by K. Malevich was used as an illustration. If a bipolar scale were composed (in our experiment the scale was unipolar), it would appear as exquisite – crude (Ozhegov, 2007). Simple geometric forms in the composition can be seen as crude , and it can be considered that the decrease in ratings on the exquisite scale occurs with the use of simple geometric forms in illustrations (likely only possible for abstract-style illustrations).

On the picturesque scale, the suprematist composition is also rated low. The understanding of picturesqueness is likely related to the perception of a plot or the experiencing of emotions.

According to the Ozhegov dictionary, picturesque is defined as “beautiful, worthy of an artist’s brush,” “vivid, imaginative, expressive” (Ozhegov, 2007). Texts with colorful realistic illustrations are rated high.

The emotions of Sadness and Joy are antonyms and correlate with different poles of the scales: sad – joyful and dark – light .

Texts with illustrations in a minor color scheme, as well as with black-and-white images, are perceived as more sad . The addition of a realistic image of an unpleasant object can raise the assessment of the text as sad . The enhancement of the text as joyful occurs with the addition of illustrations in a major color scheme, as well as realistic images of pleasant objects.

The opposition of dark – light is subject to clear representation. Adding a dark illustration increases the rating of the text as dark , while adding a light illustration increases the rating of the text as light .

Interpretation of experimental results

Thus, the sensation of Anxiety is associated with some ambiguity and originality of the image, the use of sharp angular forms, while deep blue color and rounded, especially amorphous geometric shapes reduce this sensation. These same parameters regulate the emotional state of Stress, with the stressogenic parameter of the image also being its brightness (especially bright red, orange, yellow colors), whereas pale minor tones of the image reduce the sensation of Stress.

The emotion of Hatred is associated with images of unpleasant objects, sharp forms, the use of black color, and certain color combinations (specific combinations are difficult to name because the impact potential can only be determined for a specific color stimulus), the main characteristics of which are the saturation of colors and the presence of dark colors. Hatred is weakened by the use of warm colors in a major scale and images of pleasant objects, which are perceived as kind.

The same image parameters regulate the perception of the emotion of Anger . To these, the use of unnatural color palettes, dark, gloomy tones, and large fragments of bright red color are added, as well as images depicting movement toward the viewer in the illustration. The weakening of the emotion occurs with the addition of an illustration in a realistic manner and natural colors, if its content does not evoke a sense of aggression.

The emotion of Satisfaction is enhanced by adding an illustration in a realistic style and natural colors (of course, the content of the image also plays a significant role), and is weakened by the use of unnatural color palettes, dark, gloomy tones.

The emotion of Love is associated with the use of warm tones in illustrations, primarily shades of pink. This emotion is weakened by adding geometric compositions (such as Suprematism) to the text, as well as by using cold colors in illustrations.

The emotion of Surprise is regulated by the degree of triviality of the illustration: non-trivial images evoke surprise, while expected realistic illustrations do not provoke this emotion.

The emotional state of Interest when combined with verbal text and a realistic illustration can be strengthened (in most cases) or weakened (if the illustration depicts an unpleasant object). Interest is also diminished by pale graphic illustrations.

The emotion of Contempt / Disdain is intensified by the presence of objects in the image that are unpleasant to the recipient and/or unaesthetic forms. As for the realism of the image, its impact can be bidirectional: this parameter can either weaken or intensify this emotion (if the image is perceived as primitive).

The emotion of Pleasure is enhanced by adding a realistic illustration, provided the object

Vashunina. Understanding Emotional Responses to Text does not evoke dislike in recipients. In some cases, this emotion can be intensified by using a major color palette in the image.

The emotion of Disgust can be intensified by adding a realistic image of an unpleasant object, as well as a graphic illustration.

The emotion of Sadness is intensified by adding a dark/black and white/minor illustration, as well as if the image object is unpleasant to the recipients.

The emotion of Joy correlates with light illustrations, a major color palette, and the realistic depiction of pleasant objects.

Conclusions

The correlations found between the visual parameters of illustrations and the emotions elicited by text prove that it is possible to adjust the emotional perception of text by varying certain image parameters.

The style of the illustration affects the emotions of the recipient. Some ambiguity and originality in the image increase Anxiety , Surprise , and Offense . The impact of a realistic illustration is not unidirectional. Depicting unpleasant objects increases Hatred , Anger , Contempt , Sadness , and decreases Interest . Depicting pleasant objects increases Interest , Pleasure , Joy , and decreases Hatred and Anger . Geometric compositions (in our case, Suprematism) decrease Love .

The composition of the image and geometric shapes play a specific role. Depicting motion towards the viewer increases Anger . Using sharp, angular forms increases Anxiety , Hatred , and Anger . Rounded, especially amorphous geometric shapes decrease Anxiety .

Color has a significant impact potential. An unnatural color palette increases Anger , Offense , Disgust , and decreases Satisfaction . A natural color palette increases Satisfaction and decreases Anger and Offense . Brightness increases Stress , while pale minor tones weaken this state. Dark tones increase Hatred , Anger , Sadness , and decrease Satisfaction , Love . A black-and-white color palette increases Hatred , Anger , Disgust , Sadness , and decreases Interest . Major colors increase Stress (bright), Love (especially shades of pink), Pleasure , Joy , and decrease Hatred, Anger (except for aggressive red). Deep blue color decreases Anxiety .

Based on the results of the experiments, it can be concluded that image parameters often regulate not just one emotion but several at once. This phenomenon must be considered when attempting to manage the perception of a verbal message through creolization.

In conclusion, a few words should be said about the implications of this and similar research: understanding how visual elements influence emotional responses to text can be informative for the development of emotion-aware text processing algorithms in NLP. By integrating visual analysis techniques with text processing models, NLP systems can better understand and respond to the emotional nuances of human communication. From a cognitive science perspective, the research provides insights into how visual stimuli interact with cognitive processes to shape emotional experiences. Understanding these mechanisms can contribute to theories of emotion processing and cognition, advancing our understanding of human perception and behavior.

NOTE

1 The publication was prepared as part of Research Project No. 071720-0-000, “Psychoneurolinguistics in Application to Methods for Enhancing Foreign Language Learning through Interdisciplinary Knowledge.”