Threats to objectivity in the social science research

DOI:

Before considering objectivity, the basic principle of scientific knowledge, it is worth stressing the theoretical component of science as a special and one of the forms of social consciousness. Since it is not easy to choose a generally accepted definition of science, it seems appropriate to consider some principles of scientific knowledge since each of them emphasizes one or more features. Science is, above all, “knowledge that is general and systematic, that is, all specific attitudes are deduced from a small number of general principles within its framework’’ (Nagel, 1974: 213). A more precise definition of science describes it as “an objective, logical, precise, verifiable and systematized method of collecting, describing, classifying, defining, measuring, experimenting, generalizing, predicting, controlling and evaluating experiential facts’’ (Šušnjić, 1973: 24). Moreover, there is even more comprehensive definition:

“Science represents the complete sum of systematized human knowledge and experience of nature, society, knowledge and opinion in their historical development, where by systematized knowledge we mean facts, categories, principles, laws, theories, and systems that, thanks to a unique way of thinking and knowing, in harmony with each other, tested and proven and which form a single logical whole’’ (Simić, 2002: 16).

The scientific method distinguishes science from all other forms of knowledge. However, this does not mean that science should be equated with method, even when this term means “research, examination, direction and method of research’’ (Šušnjić, 1973: 19). It is not possible to talk and think about science without clearly distinguishing among its three constitutive elements: objects, methods and cognitive goals (Nikolić, 2010: 19). Each science has “its own special subject, scientific methods, a certain amount of proven and reliable scientific knowledge, conceptual and categorical apparatus, classifications and typologies, scientific laws, hypotheses, paradigms and theories’’ (Milosavljević, 2013: 53). Scientific knowledge is based on the correct determination of the subject and goal of knowledge, also on the methodological support represented by the valid choice and the application of research techniques (Dašić, 2023a). Finally, scientific community has a shared system of communication in order to disseminate gained knowledge. Scientific knowledge is, in fact, based on “written communication and a highly structured format for presenting arguments’’ (Piršl, 2016: 473).

Science has two faces – static and dynamic. The static state of science is reflected in the fact that this form of knowledge represents a system of collected (accumulated) knowledge and experience from previous epochs. The collected facts are checked or reproduced. On the other hand, the dynamic state of science is characterized by the fact that the existing sum of scientific facts (knowledge) is subject to change, therefore, it is constantly evolving and increasing. Research as an activity of the human spirit leads to the advancement of science, ie scientific knowledge emerges as a “resultant or product of research’’ (Simić, 2002: 16).

The dynamic nature of science implies that there are no absolute truths. In his quest for understanding the world, a man pushes the boundaries of knowledge, turning sometimes dogmatized truths into relative ones based on new scientific evidence. This constant search for truth is the “first generic and distancing element’’ of science, primarily thanks to the application of the scientific method (Milosavljević, 2013: 48). Thus, this causes the basic requirement for the objectivity of the scientific approach. The focus of the analysis in this paper is on the aforementioned principles of scientific knowledge and on the obstacles with their application in both the quantitative and qualitative research in social sciences.

Theoretical considerations of the problem

The dynamic process of research takes place within a stable structure made of rules. They are, in fact, a kind of barrier to intuitive or commonsense knowledge, although it should not be neglected. Universal scientific principles (basic principles or constitutive principles of science) refer to objectivity, reliability, generality, precision and systematicity . Milosavljević and Radosavljević think that science requires “certain necessary properties” such as “objectivity, precision, systematicity, verifiability, criticality, method, as well as relative constancy, relative variability and development, confirmation and proof, intersubjectivity and communicativeness” (2006: 172). Certain authors, while trying to point out the specificity of social phenomena, add new principles to those already mentioned. Recently, Humpreys et al., for instance, have identified five commitments for all communication research; validity, transparency, ethics, reflexivity and collaboration (2021).

The basic principles of scientific knowledge - or the constitutive principles of science - make it easy to distinguish science from other forms of knowledge (for example, ideologies, religions, etc.). What makes scientific knowledge more objective than other belief systems can be expressed by answering two questions:

• (a)    Whether the belief system is normative (or evaluative), or whether it is estimating and predicting what should and what must happen;

• (b)    Whether the assurance system is empirical , focused on facts concerning objective reality.

Science is above all a system of non-evaluative beliefs directed towards the empirical world. Ideology as a separate system of beliefs is, like science, empirical. However, what separates every type of ideology from science is reflected in the fact that it determines what should happen in reality - for example, on what principles inequalities in society should be based. Religious belief systems also state what should happen, but the religious concepts are not empirical. Finally, there are forms of knowledge for which the answer to both questions (a) and (b) is no - it is a logical system (for example, mathematics).

There are many problems in contemporary science caused by the lack of objectivity, in particular in social sciences (Dašić, et al., 2023b). For a long time, the issue of objectivity has been considered in the methodological debates dealing with the best research practices for social phenomena (Ilić, 2012; 2013). Although is not possible to achieve the complete objectivity, it remains a crucial condition for any scientific research not only for social sciences, but also for science in general. This aim is more difficult to be achieved for social sciences, for variety of reasons such as, for instance, influence of contextual factors, plurality of values among social groups, and individual preferences. The respondents, participating in a survey, often create a number of problems, including reluctance, misunderstanding of items, opportunistic behavior in answering the questions, etc. Thus, special attention should be paid when considering the objectivity problem in social sciences. Otherwise, ignoring objectivity would be the undermining of scientific character of any conducted research.

The current theories of objectivity, mostly developed within discipline of philosophy of science, do not provide researchers with useful concepts that can help them to deal with this issue. This paper does not focus on the philosophical notions of objectivity. We are interested in an issue of how the conceptualization of objectivity may enhance the methodological quality of science; for example, lack of bias, replicability, reproducibility, etc. We believe that the best way to enhance objectivity is to deal more carefully with rigorous methodological procedures.

Objectivity

The principle of objectivity is usually regarded as the golden rule of science, that is, the most important principle of science. Objectivity usually means that scientific knowledge is true and impartial. The truth about objective reality is reached not only empirically, i.e. by observations at the phenomenal level (by observing manifest variables, but also by looking at deeper structures (latent variables). Generally speaking, objectivity is “the independence of the results from the researcher who obtained them’’ (Milas, 2009: 501). Furthermore, objectivity is closely tied to scientific principle of precision (Ilić, 2014).

The principle of objectivity is based on the epistemological postulate of the reality existing independently of the principle itself and its foundations. The objecti-ve/subjective dichotomy can be considered ontologically and epistemologically. A statement is objective in the ontological sense if it is autonomous; for example: water is a compound of formula H2O. A person is epistemologically objective if he/she accepts the statement based on evidence (in this case, chemical research) that water is a compound composed of oxygen and hydrogen (Brown, 2001).

Science is usually seen as a method for producing reliable knowledge by testing falsifiable claims on the basis of objective evidence. An objective observation is, according to Mascolo, typically “understood as one that is (a) based upon publically observable phenomena (i.e. overt behavior); (b) unbiased, in the sense that it records only what is observed, without adding or taking away from the observation, and (c) provides an accurate representation how the world as it truly is’’ (2016: 544).Similarly, Reiss and Sprenger suggest that objectivity can be considered as (a) a faithfulness to facts, (b) free from value commitments, and, (c) as being free from scientists’ personal biases (2017). These three main ways of conceptualizing of objectivity include some of mentioned individual notions; for instance, the procedural objectivity, value neutral objectivity, value free objectivity (Wright, 2018).

Although many papers have been published dealing with subjectivity in scientific research, there is no significant evidence to support such an approach. Armstrong’s original review of the empirical evidence, for instance, led him to conclude that procedures used by many scientists are too subjective (1979). It is also important to distinguish between the conduct of the research and the reporting of obtained results. Cotton called for more objectivity in reporting (1982). Despite a paramount importance of objectivity, unfortunately, a conceptualization of scientific objectivity has not yet proposed (van Dongen and Sikorski, 2021).

The principle of objectivity has two aspects: (1) the attitude of a researcher towards the objective reality, and, (2) the basic formal properties of scientific knowledge (Milić, 1966: 184). The first aspect implies two knowledge requirements: (a) impartiality in observing reality, and (b) gathering all relevant empirical facts. The second aspect, unlike the first one, is easier to control.

First aspect of objectivity

The first aspect involves two cognitive requirements. The first requirement implies impartiality, i.e. the absence of personal, group, class and other interests, personal emotions and prejudices in all phases of research. Thus, objectivity might be considered as what remains in the absence of certain factors, including prejudices. The sources of prejudice are numerous:

“First, scientists generally have a theory they would like to be able to confirm, so they often stick to old interpretations for a long time, even though they should be rejected or changed. Second, science is expensive. The one who pays the bills decides what kind of knowledge the research should come to. Third, political programs and ideologies often determine what type of research is desirable to conduct, especially when it comes to specific areas of government funded research’’ (Turner, 2009: 60).

The activity of scientists, obviously, takes place largely under the influence of various subjective factors. The third source of prejudices, noticed by Turner, is reported as ’funding bias’ (Nelson, 2014). Objectivity threatens the very act of perceiving reality; namely, “there is no objective but only subjective perception’’ (Bešić, 2019: 20). However, most attention is paid to the influence of researchers’ value judgments on their activity. In that sense, Škorić remarks: “These are value choices of research problems, resources and motives of a certain society in which the scientist works, etc. The methodology of science is far more objective, but it is also influenced by some trends in the scientific community’’ (2010: 36). Value judgement of a researcher reflects on (1) selection of research topic, (2) determination of conclusion’s content, (3) identification of facts, and, (4) assessment of evidence (Nejgel, 1974: 431). The difficulty of achieving objectivity is also caused by the two additional influences of (a) ethical dilemma and (b) complexity of social phenomena. Ethical issues may refer to the relation with source of data, the relation with funding agencies, relations with participants in a survey etc.

The objectivity of scientific knowledge implies ethical neutrality, that is, as Max Weber advocated, science should be “value-neutral’’. Methodology, however suitable for theories of different scope and depth, obviously cannot be immune to the influence of social and cultural values in structuring knowledge. This is clearly indicated by the conflict of value orientations within which the perceived facts are interpreted. This was also a challenge for Weber when he conducted a survey on the position of farmers in East Germany, authorized by the Association for Social Policy (Bendix, 1960). In 1892, he published the obtained results in a study which contained more than 800 pages. The key question was: can social phenomena be studied independently of political considerations, i.e. value orientations? Weber noted that the fact of replacing German workers with Polish workers on large agricultural lands east of the Elbe can be regarded both from a purely economic point of view and from the point of view of national interests:

“In the first case, the solution is obvious: since Polish labor is cheaper than German, economic reasons imply that Polish workers be hired. But in the latter case, the solution is even more obvious: as the recruitment of Polish workers strengthens Slavic penetration into German areas, the engagement of Polish labor harms national interests’’ (Đurić, 1987: 19-20).

Scientists, like all other people, adopt and nurture different values and attitudes. The adopted value system, however, must not affect the study of the very subject of research. Although Mannheim's idea of “free-floating intellectuals’’ is also a challenge for scientists, it is not easy to eliminate the influence of various social factors on the research process (Brdar, 2005). It is well known that many scientific truths about social reality have been discovered by ideologically clearly defined intellectuals. However, this does not jeopardize the objectivity of scientific research. As Helen Longino points out, it can be maintained by understanding scientific research as a social rather than an individual process. In this sense, based on the agreement of methodologists and scientists in the field of social sciences, the concept of “contextual empiricism’’ was proposed, which was demonstrated at phenomena and processes in the field of feminism (Longino, 1991). Moreover, researchers are required to explicitly state their beliefs about the subject of the research, and to present the obtained empirical results, according to the procedure that corresponds to the used research method and technique (Tufford and Newman, 2012).

The principle of objectivity, therefore, does not exclude the influence of the value system of certain cultures and social groups. As Duarte et al. explain:

“Scientific knowledge does not need to abdicate objectivity in order to substantiate ethical and political positions in the face of the problems that affect humanity today. The fallible and historically situated character of scientific knowledge does not diminish its importance in understanding natural and social reality and in developing plans for transformative action. The limits, gaps, in consistencies, and contradictions present in scientific knowledge are sumounted in the historical process of production, dissemination, and incorporation of this knowledge into people’s thinking, practice, and life. Believing that it is possible for science to progress in the production of objecttive knowledge does not in any way imply that science possesses the absolute, definitive, and unquestionable truth’’ (2021: 3).

Therefore, value-free objectivity requires a more general value-free ideal, and, consequently, any research should not be influenced by non-epistemic values such as, for instance, ‘equality’ or ‘fairness’ (Betz, 2013). The issue of value-free objectivity has not yet been resolved. In light of the conflicting opinions Douglas, for example, argues that is not possible to follow value-free ideal (2009).

The second cognitive requirement, governed by the principle of objectivity in relation to reality, is connected to the constant search for new empirical facts. Namely, the abundance of the collected experiential material enables the acquisition of a more complete and clearer picture of the research question in order to draw valid conclusions. It is about the openness towards reality and new experiences. It is about, as Manić notes, “that all available relevant experiential data is taken into account when considering a problem, as well as about the search for new information if assumed that it could be relevant for the study’’ (2017: 67).

The omission of some relevant data, consciously or by accident, does not lead to objective research. Such research remains accurate, but incomplete. The researcher, based on the analysis of the collected sources, determines the facts, which he then describes, systematizes and interprets. The ultimate goal is to explain “the content of facts, its connections and influences with other facts in time and space and their consequences’’ (Pejanović, 2017: 28-29).

Second aspect of objectivity

The second aspect of the principle of objectivity – the basic formal properties of scientific knowledge – is focused on the verifiability of the obtained results. In that sense, rules have been set for verifying both empirical material (collected for research purposes) and derived attitudes and conclusions about the studied phenomenon or process. Therefore, the second aspect of the objectivity refers to intersubjective verifiability, publicity and constant control. Intersubjective verification implies that competent individuals (respected experts in a certain scientific field) verify the collected original empirical facts (Fajgelj, 2010: 29).

The public, another procedural rule, obliges researchers to provide the scientific public with the insight into the theoretical and methodological foundations of research, data sources, the way data are arranged and classified, techniques used for analyzing the results, etc. The research can be repeated only if the researcher explicitly presents all phases of his work in the selected journal. The available space for publishing research results is, unfortunately, limiting for achieving the desired level of publicity in all parts of the research. The rule of publicity applies equally to (1) the publicity of experiential data and (2) the publicity of the research process.

Finally, the constant control of every empirical data and formulated scientific attitudes is necessary (Milić, 1966: 190-191). Sometimes, there should be will to change or reject attitudes in the light of new facts. Also, the initial assumptions are not possible to prove due to variety of reasons. Failure to verify a hypothesis may be caused by: (1) inappropriate sampling, (2) measurement error, (3) an inappropriate test, (4) an incorrect hypothesis, or (5) a combination of the above (Mascelo, 2016: 207).

Sometimes it is hard to distinguish between objectivity and verifiability of scientific knowledge. Fajgelj believes that there are epistemological differences between these two concepts (2010). The principle of objectivity is based on the intersubjective verification not only of the results of empirical research but also of scientific statements, that is, hypotheses, postulates and theories. Verifiability, on the other hand, is easier to be achieved when certain methodology is applied in a particular study. The strict application of certain procedural rules, which are being related to each method, technique or research procedure, lead to greater objectivity of scientific knowledge.

Threats to replicability and reproducibility

In order to achieve objectivity of any study, it is inevitable to take care about underlying concepts such as replicability and reproducibility. Replicability of many studies in the filed of social sciences is often a highly contested issue. Replicability implies that a result “can be obtained with other random samples drawn from a multidimensional space that captures the most important facets of the research design’’ (Asendorpf et al., 2013: 109). Reproducibility, on the other hand, means that “Researcher B [. . .] obtains exactly the same results (e.g., statistics and parameter estimates) that were originally reported by Researcher A [. . .] from A’s data when following the same [data analysis]’’ (Asendorpf et al., 2013: 109).

There are numerous factors that affect replicability of social science research. Recently, Dorothy Bishop has described various threats to reproducibility, recognized but unaddressed for decades (2019: 435). Fortunately, they might be overcome. As Bishop notices:

“Yet many researchers persist in working in a way almost guaranteed not to deliver meaningful results. They ride with what I refer to as the four horsemen of the reproducibility apocalypse: publication bias, low statistical power, P-value hacking and HARKing (hypothesizing after results are known). My generation and the one before us have done little to rein these in’’ (Bishop, 2019: 435).

Following Bishop’s identification of threats to objectivity of research, Dienlin et al. have proposed a four-fold classification, and, it includes the following causes of low replicability: (1) questionable research practices (2) publication bias, (3) low statistical power; and (4) human errors (Dienlin et al . , 2021: 4).

Table 1. Two distant modes of research

Exploratory research	Confirmatory research
New hypotheses are generated.	A priori formulated hypotheses are tested.
Predictions are based on obtained	Postdictions may help develop
empirical support.	or update theories.

The QRP known as p-hacking refers to the researchers’ efforts to deliberately search only those paths that will lead to statistically significant results (Simmons, Nelson and Simonsohn, 2011). To illustrate, imagine a researcher who continues to collect data until significant results are obtained.

The second threat to replicability deals with scientific journals which rely on engagement of editors, board members and reviewers. It has been noticed for a long time that a desire for novel and statistically significant results creates a publication bias. It is also obvious that some theories or concepts deserve more attention than the other ones. This is especially true in the case of theories developed by well/known scientists that confirm conventional beliefs. It is very difficult, on the other hand, for present theories and concepts that challenge common assumptions, especially those created by younger and unknown researchers. Management theory, for instance, as Boal and Willis contend, is one of the scientific fields that is more involved in undergoing critical self-searching, disputing the legitimacy of used methods and approaches in the development of fruitful research programs (1983: 203). Finally, researchers may experience difficulties due to their institutional connections. It was noticed, almost four decades ago, that acceptance of papers is influenced by the institutional affiliation of the author (Peters and Ceci, 1982).

On the basis of strong publication bias, follows the third threat to replicability of research. Namely, in many sciences, the tendency to examine small effects using small samples, leads to low statistical power. It refers to the probability to notice a real effect. In general, large effects require small samples, while for small effects, large samples are needed. Researchers often investigate a small effect using a small sample, producing, thus, underpowered analyses. There are at least two reasons why these analyses are questionable. First, as Funder and Ozer state, they reduce the researcher’s ability to detect effects that actually exist, and, second, they overestimate the size of identified effects (2019). As a result, low power leads to erroneous results that cause the problem tied to replication.

At last, the vast majority of studies contain human errors, as an inevitable part of research practice. After all, all researchers, as other professionals, make various mistakes. The researcher, for instance, may report incorrectly the obtained statistical results. Nujiten et al., conducting an analysis of more than 250,000 papers published in the field of psychology (1985-2013), noticed that half of the papers reporting significance tests contained at least one p-value inconsistent with its test statistic or degrees of freedom (2016). Yet, many of them are not willing to share their data in order to easily detect and remove noticed mistakes.

Therefore, the researchers find themselves often in a situation to choose between the advancement of scientists and scientific advancement, the dichotomy coined by Boal and Willis. The vast majority of scientists adopt, according to gathered empirical evidence, the so-called Author’s Formula (Armstrong, 1982: 197). It states that the researchers who are fully dedicated to career development should not (1) choose an important issue, (2) challenge common beliefs, (3) obtain surprising results, (4) use simple methods, (5) provide full disclosure (description of used procedures) and (6) write clearly. This opportunistic approach, mainly, leads to low replicability of many even empirical studies within social science applied disciplines as well. Thus, this general lack of replicability of scientific findings has become recognized as the replication crisis (Harris, 2017). Moreover, the general public has become aware of these shortcomings (Anvari and Lakens, 2018).

One way to increase replicability in the research process could be an acceptance of new practices, recently recognized as so-called Open Science. This term refers to a broad movement among scholars calling for changes in the way scientific research is conducted. Open science practices include various efforts such as open access (OA) publication, open sharing data, or open methods. Despite such initiatives, there were no OA articles published in, for instance, The Sport Psychologist , and, only four articles had been published OA in the Journal of Applied Sport Psychology , as reported by Tamminen and Poucher (2017).

Conclusion

The issue of objectivity in scientific research has been widely considered from the standpoint of philosophy of science. This discipline tends to explain what the principle of objectivity means and how it is used in both the quantitative and qualitative studies. In this paper, however, an attempt has been made to offer a practical guidance to cope with various threats to objectivity of any scientific research.

The best way to overcome the problem of objectivity, as it is explained in this paper, is to follow strict procedures proposed for each step in the whole research process. The suggested approach is, therefore, the acceptance of objectivity as methodological resilience. This stance is more appropriate to quantitative methodology and study of numerous phenomena within social sciences.

The issue of objectivity has been central to the methodology of qualitative research in many social sciences. A sociologist, for instance, who investigates various phenomena related to sport people and sport organizations, in order to ensure objectivity, should avoid calling for a specific value, and, consequently, his/her focus should be on “what is” instead of “what ought to be”. On the other hand, it is not possible to neglect influence of various values in theory building. In line with this, one can conclude that objectivity should not be viewed as a sole criterion in evaluating studies of social phenomena.

Acknowledgements

This paper was prepared with the financial support of the Ministry of Science, Technological Dvelopment and Innovations of Republic of Serbia, within the funding of the scientific research work at the University of Belgrade, Technical Faculty in Bor, according to the contract with registration number 451-03-47/2023-01/200131.

Conflict of interests

The authors declare no conflict of interest.

Author Contributions