Methodology for identifying and tracking social media misinformation in tweets about the impact of the COVID-19 pandemic on reproductive health

Золотарев Олег Васильевич

Санджив Кумар Джейн

PhD, доктор, доцент кафедры электротехники, Университет Меди-Кэпс, Индор, Мадхья-Прадеш. Автор двух научных работ, опубликованных в изданиях, индексируемых Scopus.

Санджай Каушал

PhD, доктор, доцент кафедры компьютерных наук, Университет Шарда, Нью-Дели, Автор восьми научных работ, опубликованных в изданиях, индексируемых Scopus.

Методология выявления и отслеживания дезинформации в социальных сетях ...

false information circulating on the internet [10]. Of particular concern is the rapid spread of false information on Twitter, with lies noted to spread faster than the truth [11].

Disinformation is defined as false, erroneous, or misleading information, and a type of claim that can be tested and confirmed to be false. Disinformation has an adverse effect on society as it causes anxiety, fear and influences public opinion [12].

Various misinformation about the COVID-19 vaccine has been circulating on social media. For example, misinformation through conspiracy, such as “coronavirus vaccine destroys female fertility and makes men impotent” can lead people to refuse vaccination [12]. We set out to understand what topics and concepts related to disinformation appear in Twitter discourse on the impact of coronavirus on the reproductive health of men and women.

The purpose of this study is twofold. First, it examines current scientific information about the potential impact of the coronavirus and the vaccine on male and female reproductive health. We a priori consider the results of recent research in this area to be true. Secondly, the study examines the opinion of Twitter users about the impact of the coronavirus and the vaccine on male and female infertility. If the focus of research matches the topic of discussion on Twitter, based on the results of study 1, we conclude that the opinions of Twitter users about the coronavirus and the vaccine in relation to infertility are true or false.

We use machine and expert methods in combination, which benefits the analysis. In this article, we aim to test the functionality of a combination of Twitter data analysis methods, to explore the discussed topics related to disinformation and related concepts. This study is limited to the study of the textual content of scientific articles, and in tweets we study texts and emoji.

Materials and Methods

Within the area of interest, the primary selection of the topic is determined by the expert choice, social networks (social inquiry). After choosing a frequent social topic, it is compared with scientific documents, a comparison of interests is carried out, an attempt is made to find out if there is disinformation.

We reviewed user queries on the topics: ‘Covid dementia’, ‘Covid fertility’, ‘Covidosis thrombosis’, ‘Covid neurological’ (Figure 1). The topic ‘Covid fertility’ was chosen as the leader throughout 2021.

Figure 1. Request frequency for the terms ‘Covid dementia’, ‘Covid fertility’, ‘Covid thrombosis’, ‘Covid neurological’ in English (worldwide)

A search was performed in the PMC for articles related to the study of coronavirus and fertility. The search was conducted for keywords in the abstracts filtered by publication date on 10 May 2022. The content of the information request was as follows: (“sars-cov-2”[MeSH Terms] OR “sars-cov-2”[All Fields] OR “covid”[All Fields] OR “covid-19”[MeSH Terms] OR “covid-19”[All Fields]) AND (“fertility”[MeSH Terms] OR “fertility”[All Fields]) AND (“2022/01/01”[PubDate] : “2022/12/31”[PubDate]). 998 articles were found, published for the period from 2022/01/01 to 2022/12/31. 167 articles more relevant to the subject of study (human fertility) were selected from a total collection of 998 articles. The relevance check included the control of the simultaneous occurrence in the abstracts of the terms COVID (Coronavirus) and Fertility. The significant terms have been extracted from the abstracts. For extracting terms, the special program was used (described below).

Data for research was downloaded from Twitter in May 2022 (from 22/04/2022 to 30/04/2022). The upload was carried out using the analytical mechanisms of Twitter (Vicini-tas) [Vicinitas. ], which allows uploading hashtags, user accounts and keywords. This research contains English tweets. The tweet collection was built based on keyword or hashtag queries “Covid, fertility”, “Covid, infertility”, “Coronavirus, fertility”, “Covid, sterility”, “Vaccine covid, fertility”. The collection included 9436 tweets. For the further analysis we used only 2560 original tweets. For extracting terms, the special program was used (described below).

The original terms’ extraction program was developed with the improvement of the classical approach for automatic extraction of named entities from full-text messages [13]. The improvement was as follows: to define named entities in the text as stable word-combinations, the main noun with determiners was retrieved out.

The standard set of python stop-words included only conjunctions, interjections, etc., which were excluded from consideration. The set of stop words specifically for processing medical texts has been significantly expanded, considering the results of the previous expert analysis of the vocabulary of scientific publications on biomedical topics. About 800 elements were added to the set of stop words (including words of such categories as geographical names, general medical terms, general scientific terms). This expanded set allows to reduce the amount of “noise” when extracting medical terms through automatic processing of scientific documents.

The set of stop words specifically for processing tweets has been expanded. According to the results of expert analysis, there were added several categories of stop words (geographical names, dates, days of the week, names of months, proper names, names of institutions, job titles, etc.). So, we significantly reduce the “noise” when extracting terms from tweets through automatic processing.

From the compiled collections of scientific articles and tweets, significant terms are extracted and analyzed (using an extended list of stop words). Terms related to the field of coronavirus and fertility were manually cross-checked.

We proceed from the assumption that scientific statements are true now. To analyze truthful statements, we applied contextual analysis of scientific texts. In the context, we singled out the effect of coronavirus (COVID) on an organ (parameter) of the object of study, indicating the mechanism of action (or without it). At the same time, the degree of evidence of the impact was especially noted.

Методология выявления и отслеживания дезинформации в социальных сетях ...

Results

Scientific terminology (PubMed)

Terms related to the field of coronavirus and fertility were divided by experts into thematic categories [14]. Eleven thematic categories were identified:

• -    Genetics and molecular biology,

• -    COVID and other diseases,

• -    Coronavirus,

• -    Infertility (without gender),

• -    Female reproductive health (FRH),

• -    Male reproductive health (MRH),

• -    COVID vaccination,

• -    Research objects,

• -    Methods,

• -    Relationships,

• -    Marker terms (sentiment words).

The minor categories were: 1) relationships (couple relationship, sexual behavior, sexual, etc.); 2) methods (antioxidant, drug, ultrasound, anti-inflammatory, etc.); 3) marker terms (anxiety, emotion(al), death, psychological, etc.).

A total of 388 terms were identified in 11 groups. The frequency of the most common term in each category is taken as 100 %. Top-5 terms for the major categories of scientific terminology related to “Coronavirus and Fertility”are listed below:

• -    “Coronavirus” (COVID-19, 100 %; SARS CoV-2, 55 %; pandemic, 45 %; SARS-CoV-2 infection, 38 %; COVID-19 pandemic, 34 %);

• -    “COVID and other diseases” (stress(or), 100 %; immune, 73 %; diabetes, 32 %; endocrine, 27 %; inflammation, 27 %);

• -    “Female reproductive health” (pregnancy, 100 %; pregnancy rate, 35 %; assisted reproduction technology (ART), 33 %; oocyte, 33 %; embryo, 25 %);

• -    “Male reproductive health” (male fertility, 100 %; Semen, 56 %; testicular, 50 %; testosterone (T), 44 %; semen parameters, 44 %);

• -    “Infertility (without gender)” (fertility, 100 %; reproductive, 47 %; infertility, 48 %; fertility treatment, 26 %; fertility preservation (FP), 24 %);

• -    “COVID vaccination” (COVID-19 vaccination, 100 %; vaccine, 73 %; vaccination, 65 %; miR-371a-3p, 19 %; side effects, 16 %);

• -    “Genetics and molecular biology” (express(ion), 100 %; angiotensin converting enzyme (ACE2), 67 %; hub genes, 53 %; luteinizing hormone (LH), 47 %; protein, 47 %);

• -    “Research objects” (woman, 100 %; child(ren), 57 %; animal (rabbit, rat, mice), 45 %; pregnant, 41 %; male, 39 %).

Social network terminology (Twitter)

Terms related to the field of coronavirus and fertility were divided by experts into thematic categories. Nine thematic categories were identified:

• -    COVID and other diseases, Coronavirus,

• -    Infertility (without gender),

• -    Female reproductive health (FRH),

• -    Male reproductive health (MRH),

• -    COVID vaccination,

• -    Discussion objects,

• -    Marker terms (sentiment words),

• -    Conspiracy theory.

The minor category was the Conspiracy theory (poison, depopulation, Microchips, etc.).

A total of 220 terms were identified in 9 groups. The frequency of the most common term in each category is taken as 100 %. Top-5 terms for the major categories tweets related to Coronavirus and Fertility are listed below:

• -    “Coronavirus” (COVID(-19), 100 %; get covid, 10 %; Long Covid, 7 %; virus, 7 %; COV-ID(-19) infection, 5 %);

• -    “COVID and other diseases” (lung, 100 %; cancer, 67 %; heart attack, 67 %; myocarditis, 67 %; heart, 67 %);

• -    “Female reproductive health” (pregnancy, 100 %; baby, 43 %; birth, 29 %; pregnant women, 21 %; in vitro fertilization (IVF), 14 %);

• -    “Male reproductive health” (male infertility, 100 %; erectile dysfunction, 62 %; sperm, 37 %; sperm count, 37 %; sex, 37 %);

• -    “Infertility (without gender)” (fertility, 100 %; infertility, 52 %; sterility, 12 %; fertility issues, 11 %; fertility rate, 11 %);

• -    “COVID vaccination” (COVID(-19) vaccine, 100 %; vaccine, 98 %; vaccinate, 38 %; Covid(-19) Vaccination, 26 %; vax, 18 %);

• -    “Markers” (risk, 100 %; death, 62 %; damage, 56 %; recover, 50 %; safe, 37 %);

• -    “Discussion objects” (people, 100 %; men, 90 %; women, 75 %; children, 75 %; population, 70 %).

Comparison of scientific and social network terminologies

For matching categories in scientific and social network terminologies, a comparison was made by frequency terms. Terminological usage (frequency) indicates how often the terms of a given category are used in a discussion (Figure 2).

Figure 2. Terminological frequency. Comparison of coinciding thematic categories by the frequency of terms (in % of the total number of terms).

Методология выявления и отслеживания дезинформации в социальных сетях ...

Detection of true and false judgments

Revealing scientific judgments (PubMed)

In the previous sections, we found that the interests of the scientific community and the community of Internet users coincide in many ways. Our next task was to find out how the opinions of Twitter users differ from scientific judgments. The following topics were chosen for the study:

• -    Male reproductive health and coronavirus,

• -    Male reproductive health and the vaccine,

• -    Female reproductive health and coronavirus,

• -    Female reproductive health and the vaccine.

The contextual analysis described in the Methods section was used.

We applied contextual analysis of scientific texts. The contextual analysis included determining the evidence of the effect of coronavirus (COVID) on a reproductive system (organ, parameter) of the object of study, indicating the mechanism of action (or without it). Examples of results are shown in the Table 1.

Table 1

Some results of contextual analysis of scientific texts related to “Reproductive system and Coronavirus”

Influence/ impac t	Proved/not	Mechanism of action	Target, parameter	Object/ period
Male reproductive system and Coronavirus
effect of SARS-CoV-2 [15]	Is significantly reduced	Immunopathological damage	Testicles, semen index	After infection
Covid-19 [16]	Decreased, reduced	Orchitis development	Sperm quality, sperm count, sperm motility	Patients with COVID-19
Covid-19 [17]	Downregulation	Semenogelin 1 and prosaposin	Male fertility	COVID-19-recovered patients
Detection of SARS-cov-2 virus [18]	Remains scarce, Has been reported	Testicular damage and dysregulation of gonadotropins	Testis	Males
COVID-19 Infection [19]	Induce	miR-371a-3p Upregulation	Fertility	Males
SARS-CoV-2 infection [20]	Direct effects	Presence of viral entry receptors (ACE2 and/ or CD147)	Testicular cells, such as spermatocytes, Sertoli cells, Leydig cells
Impacts of COVID-19 [21]	Dysfunctions	The induction of systemic inflammatory responses and oxidative stress	Reproduction	Males
Impact of SARS-cov-2 [22]	Very limited evidence	Impact on fertility parameters	Male fertility and sexual health, Reproductive hormones, etc.	Males
COVID-19 [23]	Negative impact	Distribution of ace2 and transmembrane protease serine 2	Male fertility, sperm quality	autopsy

Ending Table 1

Female reproductive system and Coronavirus
SARS-CoV-2 infection [24]	May interfere		Mice’s fertility, lower pregnancy rate	Infected pregnant mice
Severe acute respiratory syndrome coronavirus type 2 infections [25]	Have been associated	Adverse perinatal outcomes	Maternal morbidity	Pregnancy
SARS-CoV-2 infection [26]	Does not affect		Oocyte yield, fertilization and maturation rate, number of good quality embryos, etc.	Woman
SARS-CoV-2 infection [27]	Increased risk	Spontaneous abortion	Complications	Pregnant women
COVID-19 [28]	Was not significantly affected		Ovarian reserve	Patients recovering
SARS-CoV-2 [29]	Impair	Mitochondrial hijacking	Fertility	Female

For topics “Male reproductive health and the vaccine”, “Female reproductive health and the vaccine” (Table 2) we show only few results without mechanism of action because they are similar mainly.

Table 2

Some results of contextual analysis of scientific texts related to “Male/Female reproductive system and Vaccine”

Influence/ impact	Proved/not	Target, parameter	Object/ period
Vaccine [30]	Does not seem to affect	Sperm parameters	Male
COVID-19 vaccination [31]	Did not affect	Men’s reproductive health, sperm quality and fertilization capacity	Men
Vaccines against SARS-cov-2 [17]	No evidence	Spermatogenesis or male reproductive health	Male
Vaccine [32]	No negative impacts	Fertility, the course of pregnancy, or fetal development	Woman
Vaccination [33]	No difference	Clinical pregnancy rates, fertilization rate and transferred embryos’ quality	Vaccinated and unvaccinated patients (woman)
COVID-19 vaccination [34]	Do not appear to adversely affect	Assisted reproductive technology and pregnancy, gametes, embryos	Woman

We divided statements about the confirmation and denial of the impact of the coronavi-rus/vaccine on the reproductive health of men and women. Confirmatory markers include the following: “is significantly reduced”, “negative impact”, “can affect”, “decreased”, “reduced”, “downregulation”, “has been reported”, “observed”, “participates in”, “strong association”, “dys-regulation”, “damage”, “direct effects”, etc. Negative markers include the following: “very little evidence”, “no viral RNA was detected”, “unclear”, “many unresolved questions”, “no evidence”,

Методология выявления и отслеживания дезинформации в социальных сетях ...

“limited evidence”, “remains unknown”, “does not seem to affect”, “no significant changes”, “did not affect”, “does not impair”, etc.

We then calculated the ratio of confirmatory (“yes”) and negative (“no”) markers in scientific documents by topics “Male reproductive health and coronavirus” (77,59 % for “yes” and 22,41 % for “no”), “Male reproductive health and the vaccine” (100 % for “no”),“Female reproductive health and coronavirus” (66,67 % for “yes” and 33,33 % for “no”), “Female reproductive health and the vaccine” (8,33 % for “yes” and 91,67 % for “no”).

Revealing user judgments (Twitter)

The following topics were chosen for the study:

• -    Male reproductive health and coronavirus,

• -    Male reproductive health and the vaccine,

• -    Female reproductive health and coronavirus,

• -    Female reproductive health and the vaccine.

We applied contextual analysis of tweets. The contextual analysis included determining the evidence of the effect of coronavirus (COVID) on a reproductive system (organ, parameter) of the object of discussion. Examples of results are shown in the Table 3.

Table 3

Some results of contextual analysis of tweets related to “Male and female reproductive system and Coronavirus (Vaccine)”

Influence/ impact	Proved/not	Target, parameter	Object/ period
Covid	Reduces/impacts/can mess/ potentially negative effect/ diminished/ exponentially worse/ temporarily reduces/ may be messing	Fertility	Men
Covid	Causes/can cause	Infertility	Men
Covid	May be messing	Fertility, lower sperm count and motility	After infection male
Covid	Might effect	Fertility	Children
COVID-19 infection	May lead	Fertility problem	Men
Covid	Is causing	Erectile dysfunction, infertility	Men
Covid	Affected	Decreased sperm production and deformed sperm	Guy
Covid-19	Might lead to	Infertility, testicular abnormalities, Testicular pathology	Covid-19 survivors
Long Covid	Possible/ would lead	Infertility	Male
Long Covid effects	Effect	Reproduction, sperm count	Male
Long Covid effects	Some studies have shown	Sex drive loss, fertility	Male
Covid	Probably	Delayed ovulation, period	Women
Long covid	Effect	Reproductive fertility	Girls
Long-term	Side effects’	Miscarriage, fetal abnormalities, fertility	Woman

Ending Table 3

Covid vaccines	Cause	Sterility	Men
Covid vaccine	Destroys/ It hasn’t been tested/ may be a factor	Fertility	Men
COVID‐19 vaccine	Linking/ adverse reaction	Infertility	Male
Covid vaccines	May cause	Sterility	Man
Covid vaccine	Most common side effect	Sterility, or dysmenorrhea	Woman
COVID-19 vaccines	No evidence	Fertility problems	Anygender
Covidvaccine	There was no link	Infertility	Both men and woman
COVID vaccines	No information	Fertility	Female
Vaccines	No indication	Infertility	Either sex
COVID vaccination	The effect is small and temporary	Menstrual cycle timing	Women
COVID-19 vaccine	Does not harm	Wombs	Women
Covid vaccine	Affects is a lie	Uteruses, fertility	Women
COVID19 vaccination	No evidence	Clinical outcomes in ivf, Fertility	Women
COVID-19 vaccinations	Misinformation	Pregnancy, fertility and breastfeeding	Women
Vaccination	No evidence	Affect fertility	Women trying to become pregnant
Vaccinated	Poison, damages	Fertility	All
Vaccinated	Baseless fearmongering	Fertility	Pregnant women
Vax	Control of the population	Fertility	Men
Covid jabs	Destroy	Fertility	Civilization
Covid shot	Is effecting/ messing	Fertility	Men(and female)
Vax	Wreck the immune system	Fertility	European women
Covid vaxx	Causes	Infertility	Women
Moderna COVID vaccine	Damaging	Unborn child, fertility	Women

We divided statements about the confirmation and denial of the impact of the coronavirus/ vaccine on the reproductive health of men and women. Confirmatory markers include the following: “may be messing”, “reduces”, “possible”, “impacts”, “causes”, “effect”, “would lead”, “potentially negative effect”, “is messing”, “alters”, “it seriously affected”, “affected”, “destroy”, “diminished”, “impaired”, etc. Negative markers include: “no information”, “does not impact”, “no effects”, “do not cause”, “no evidence”, “there was no link”, “the effect is small and temporary”, “has no impact”, “misinformation”, “affects is a lie”, etc.

We then calculated the ratio of confirmatory (“yes”) and negative (“no”) markers in tweets by topics “Male reproductive health and coronavirus” (100 % for “yes”),“Male reproductive health and the vaccine” (77,27 % for “yes” and 22,73 % for “no”), “Female reproductive health and coronavirus”, (100 % for “yes”),“Female reproductive health and the vaccine” (22,22 % for “yes” and 77,78 % for “no”).

Методология выявления и отслеживания дезинформации в социальных сетях ...

Discussion

In scientific articles, much attention is paid to the genetic and molecular biological aspects of the impact of coronavirus and vaccines on fertility. Naturally, research and diagnostic methods are mentioned. Many categories of people were subjected to the study (women, animals, men, pregnant women, patients with coronavirus, recovered from coronavirus, etc.). Often fertility is mentioned without specifying gender.

Among the thematic categories, the category coronavirus is the most mentioned, both in terms of the number of terms extracted and the number of terms used (frequency). The category ‘Female reproductive health’ is in second place in the number of terms and in third place in frequency. The category ‘Research objects’ is in third place in the number of terms and in second place in frequency.

In terminological diversity in tweets, ‘COVID and other diseases’ is the leader, ‘Discussion objects’ are in second place, and ‘Female reproductive health’ is in third place. ‘Coronavirus’ is in first place, ‘COVID vaccination’ is in second, and ‘Infertility (without gender)’ is in third place in frequency of use of terms.

A comparative analysis of terminological diversity of the thematic categories showed that terms related to ‘Coronavirus’, ‘Female reproductive health’, and ‘Research objects’ are leading in scientific papers. There are more different terms in tweets when discussing diseases/compli-cations of COVID (COVID and other diseases), Objects, Coronavirus and different markers are used to indicate the user’s attitude to the subject.

A comparison of the frequency of the most used terms by thematic categories showed that in scientific articles there are more frequent terms related to’ Coronavirus’, ‘Research objects’, ‘Female reproductive health’. In tweets, we noted a greater number of terms used in the categories ‘Coronavirus’, ‘COVID vaccination’, ‘Infertility (without gender)’.

It can be concluded that the focus of researchers and social network users are the same thematic categories related to coronavirus and fertility, such as ‘COVID and other diseases’, ‘COVID vaccination’, ‘Male reproductive health’, ‘Female reproductive health’, ‘Infertility (without gender)’, ‘Coronavirus’. Only the ratio of categories in terms of variety and frequency of terms changes.

A contextual analysis of scientific articles to determine the impact of coronavirus and covid on the reproductive system of men and women made it possible to draw the following conclusions, which we a priori consider correct and truthful.

Scientific articles prove that:

• 1)    the vaccine does not affect male reproductive health (100 % negative statements);

• 2)    the vaccine has practically no effect on women’s reproductive health (91.6 7% of negative statements);

• 3)    coronavirus greatly affects male reproductive health (77.59 % of positive statements);

• 4)    coronavirus greatly affects women’s reproductive health (66.67 % of positive statements).

A contextual analysis of tweets to determine the impact of coronavirus and covid on the reproductive system of men and women made it possible to draw the following conclusions:

• 1)    the vaccine strongly influenced male reproductive health (77,27 % positive statements);

• 2)    the vaccine little effects on women’s reproductive health (22,22 % of positive statements);

• 3)    coronavirus absolutely affects male reproductive health (100.00 % of positive statements);

• 4)    coronavirus absolutely affects women’s reproductive health (100.00 % of positive statements).

  

Figure 3. Comparison of scientific conclusions and opinions of Twitter users by topics about the impact of coronavirus COVID on reproductive health

Scientific articles and tweets are actively discussing the impact of coronavirus on the reproductive health of men and women. However, on social media, users are convinced of the full and absolute impact of coronavirus on the reproductive health of men and women, while in scientific articles the figure 3 were 77.59 and 66.67 %, respectively. Consequently, the conclusions about the impact of coronavirus on the reproductive health of men and women are exaggerated by 1.29 times for men, 1.50 times for women.

The greatest discrepancies are observed in the impact of the vaccine on the reproductive health of men and women. In scientific articles, there are 8.33 % of positive statements about the impact of the vaccine on women’s reproductive health, and in tweets – 22.22 %, that is, 2.67 times higher. Quite disastrous is the judgment of Twitter users about the impact of the vaccine on male reproductive health. 100 % of users are convinced of the presence of such an influence, while scientific articles deny such an influence.

The methodology we proposed is shown in Figure 4.

The methodology proposed includes, as an initial stage of the analysis, checking the coincidence of interests of the scientific community and users of social networks. In the case of coincidence of interests (and in our case they practically coincide), the second stage of the methodology is implemented. In the second step, by comparing judgments from scientific sources and social networks, we find out how they differ. With a strong difference, it can be concluded that false information is being disseminated on social networks.

Методология выявления и отслеживания дезинформации в социальных сетях ...

Figure 4. Flow chart of the methodology

Acknowledgement

The authors are grateful to I. Zatzman for his advice in preparing the manuscript.