What kind of teacher does the “school of the future” need? Possibility of using John Hattie's approach in Russian education

Introduction. Substantiation of efficiency markers in relation to social policy in general and to education management in particular remains relevant for departmental structures [1, pp. 58-95]. As a consequence, the academic community finds solutions to this problem, but they are often being disputed because they are usually formed ad hoc , taking into consideration the experience of other countries, but in isolation from the prevailing cultural traditions, and without considering the opinion of the educational community and the need for preliminary testing on the example of the model region (organization). As a result, some of the new phenomena turn into real institutional traps for all participants in educational relations. Suffice it to recall the Unified State Examination, which is now approved by only a third of school teachers [15].

It is well known that education today is a field in which tradition and innovation meet, which often leads to a direct confrontation. One side of this conflict is traditional education and the other is experimental education or the “school of the future”.

Traditions in teaching (traditional didactics), formed over the centuries, are revealed in the works of J.A. Komensky (17th century), A. Diesterweg (19th century). The traditional educational concept rests on “five pillars”: 1) knowledge paradigm (orientation of pedagogical process on the formation of knowledge); 2) central role of the teacher (belief in the authority of the teacher as a central part of the educational process); 3) class-and-lesson system (belief in the positive impact of the team on the final results of education); 4) discipline (providing a link between training and education); 5) B. Bloom’s taxonomy of educational objectives (strategy of skills development at certain levels: knowledge/ comprehension/application/analysis/synthesis/ evaluation) [2, pp. 20-29].

A.A. Verbitsky believes that the classical educational paradigm conceptualizes a person as “a simple system, limiting the range of the student’s mental functions included in the work” [3, pp. 3-6], and also “does not provide for the activity at the level of thinking and personality of the student, his/ her creative consciousness” [3, pp. 3-6]. G.M. Nurmukhamedov points out that a significant drawback of the traditional education system consists in the lack of “a diagnostic goal focused on the meaning of human life”, which in turn is expressed in work and creativity [2, pp. 20-29].

With the development of progress (technological, in particular), the model of traditional education was often criticized, which was facilitated by the preservation of school orders in post-war Europe and the United States, and the lack of real educational innovations against the background of improving technologies [4, pp. 23-27]. J. Goodlad in 1980 wrote that “the ability of schools to perform their main and usual role (promoting literacy and elimination of ignorance) raises more and more doubts, and for a reason” [5, p. 10]. However, in the 1970s, the impact of demographic pressures on Western schools increased due to the increase in the number of divorces. As a result, relations in the heterogeneous class became increasingly alienated and depersonalized [5, pp. 23-27]. In addition, in science there emerges a new nonlinear idea of the educational process, proceeding not from the universal system, the same for all, but from the individual goals and strategies of each student, the theory of metacognition (Marzano taxonomy) [6, pp. 5-17].

As a consequence, in the second half of the 20th century in Europe there emerged a movement of “the new school”, which on the basis of the development of the concept of “natural education” by J.-J. Rousseau proclaimed a departure from institutional to non-institutional forms of education. In particular, the fathers of the “new school”: educators R. Cousinet and C. Freinet wrote about the need for a radical reorientation of the education system to suit the needs of the child, and about the importance of using new forms of teaching, such as work in small groups (instead of classes), learning through play, organizing school self-government, and abandonment of grading and textbooks [7]. Teachers in this model already cease to play a leading role in the educational process. Instead of mentoring and transferring knowledge, they are intended to monitor the “natural development of students and advise them on challenging issues” [8, pp. 121-122].

French teachers-innovators probably did not even suspect that their rather cautious ideas would be later developed in a more radical theory of unschooling – education of children in complete separation from school. Within the framework of this trend, the Austrian theoretical philosopher I. Illich developed the thesis about “the liberation of society from the monopoly of the school” [9, p. 16]. Inspired by the ideas of Illich, American teacher J. Holt put forward a theory that the main cause of academic failure in the modern school is the school system itself, because it is interested in the formation of a “producer rather than a thinker” [10, p. 8]. In his opinion, the child’s qualities are developed best of all in “natural conditions”, outside the walls of an educational organization (“the best place for learning is the thick of life”) [10, p. 447].

The ideas of unschooling, which were repeated many times in foreign pedagogical science (R. Moore, P. McLaren, D. Goodlad, K. Robinson, A. Helmke, etc.), influenced a number of pedagogical concepts (Wal-dorf education, Montessori education, homeschooling) [11, pp. 213-219]. However, this concept did not have a significant impact on the formation of the educational paradigm in Western countries; the concept received the status of a revolutionary, but at the same time underground and marginal ideological direction1. Already in the 1980s, against the background of the decline of industrial production, revolutionary ideas in Western pedagogy receded into the background, and educational traditions began to revive. In particular, the system of classical education regained popularity thanks to the article by D. Sayers “The Lost Tools of Learning”, in which the “standards” of the medieval school (grammar, dialectics, rhetoric) were adapted for the modern world [11, pp. 213-219].

However, being on the second positions, the ideas of the “new school” show themselves indirectly as an ideology of modern educational policy. A striking example of this can be found in the monograph of New Zealand scientist Jonh Hattie, which he called “Visible Learning” (2009). The book boldly claims not only theoretical and philosophical, but also empirical justification of the factors that help the school to become effective (in relation to educational performance). In this regard, Hattie’s work proved to be advanced for its time and gained considerable authority in the educational community of Western countries, and in some cases became the ideological and methodological basis for reforming the education system. The reason for this is simple: the scientist offers a fairly ordinary solution to educational problems, which by its accessibility may attract managers at all levels. This solution is reflected in the fact that schools are implementing a monitoring system for the indicators that he highlighted.

At the same time, Hattie’s approach emphasizes the professional activity of teachers, who according to their duties are closer to the moderators of the student’s educational path than to professionals (which is in tune with the thoughts of the theorists of the “new school”).

Currently, the wave of popularity of “Visible Learning” has reached Russia. In 2017, this book was translated and published by “Natsional’noe obrazovanie” publishing house, which is part of group of companies “Prosveshchenie”. In our study, we asked the question: to what extent is Hattie’s concept applicable in the Russian school and in teachers’ work in the light of future development prospects? These prospects, largely identified by the national project “Education”, guide the teacher along the path of continuous updating of their knowledge (including digital and technological), as well as forming a situation of success for each student. Last but not least, we are interested in the feasibility of integrating Hattie’s ideas into educational policies at the local, regional and national levels.

The goal of our paper is to analyze, based on the study of the research literature and own calculations, the scientific validity of Hattie’s arguments and the results of his metaanalysis for the subsequent assessment of the applicability of his approach in Russian educational policy.

Methodology. When formulating the goal of the study we use the following methods: 1) theoretical and methodological overview of the strengths and weaknesses of Hattie’s ideas used a discursive analysis of his concept, and also criticism of his book “Visible Learning” (the bulk of which is presented exclusively in foreign languages); 2) to verify the stability of Hattie’s assessment model, we calculated the coefficients of variation (CV) of the statistical population of medium size effects highlighted by the researcher in the course of meta-analysis.

Hattie’s concept . Hattie’s study is not the first attempt to generalize educational effects in meta-analysis. This method, proposed by D. Glass in 1976, involves the synthesis of already created empirical works on a given topic not in the traditional (review) way, but with the help of mathematical evaluation tools [12]. This method is more common in economics and medicine. In 2000 R. Marzano in the book “A New Era of School Reform” for the first time applied meta-analysis to education. He summarized 4,057 dimensions of effects and ultimately identified five levels of school performance: 1) a safe and orderly environment that supports interaction and collaboration; 2) an educational structure that supports effective learning in each classroom; 3) a guaranteed and viable curriculum focused on improving learning outcomes; 4) a standards-based reporting system for student performance; and 5) a knowledge control system that ensures that students acquire the knowledge and skills they are taught at school. However, according to experts, Marzano’s system is noticeably limited in the choice of school-related and structural factors [14].

Hattie essentially supplemented the scientific tradition of studying the education system, because he applied meta-meta-analysis (or mega-analysis) rather than meta-analysis. This approach is different, because it synthesizes not the empirical works themselves, but the very meta-analyses devoted to the evaluation of these works. Hattie’s book analyzes more than 800 meta-analyses that were conducted on the basis of studies of educational performance of schoolchildren in three countries (USA, Australia and New Zealand) during the 1980s–2000s [14].

The goal of such a large-scale generalization is to assess the so-called “school effects” (the term is borrowed from Marzano). These are external and internal factors that have a different vector of orientation in relation to the process of forming the student’s academic performance. At the same time, Hattie understands effective education as a multi-faceted process, including “successful learning” on the part of children and “successful teaching” on the part of teachers [14, pp. 54-59]. In a fundamental sense, Hattie sets a task to find a universal recipe for a successful educational reform.

Hattie’s methodology is based on the following sequence of research steps:

• 1.    Identification of environmental sources that affect the academic performance of the child. Hattie identified six such sources: family, school (as a special environment and administrative apparatus), teachers, curricula, teaching strategies and methods, and the student.

• 2.    For each source, a set of estimated variables are determined , Hattie called them factors. In total, he identified 138 factors ranging from the stages of development of the child’s intelligence according to J. Piaget to the summer holidays.

• 3.    Synthesis of meta-analyses to assess the impact of factors on the success of students. At this stage, as part of the generalization of scientific papers, variables were calculated for each factor; the variables are called the “average effect size” (d) . The size of the effect was determined in the range from -2 to 2.

• 4.    Interpretation of the results of calculations for which Hattie developed a “barometer of influence” with zones of negative, low, medium and high effect.

Calculations carried out by Hattie on the basis of synthesis of meta-analyses show that the work of the teacher has the greatest influence on the achievements of schoolchildren (d=0.49) . However, this conclusion will not be complete without taking into account the influence of various factors (Tab. 1) . Thus, it was found that micro- training (pedagogical practice) is extremely important (d=0.88) for the effectiveness of the educational process, and the stock of pedagogical knowledge and teacher qualification are the least important (d=0.1). Such conclusions led Hattie’s critics to question the correctness of his conclusions.

Oddly enough, the least influence on the knowledge and skills of students, according to Hattie, is exerted by the school itself as a social institution and educational organization (d=0.23) . In matters of socialization, the researcher clearly gives priority to the family rather than to education. Here we find obvious parallels with the ideas of the classics unschooling (Illich and Holt), who wrote that it is best for the teacher to work outside institutional structures. Hattie himself is not very enthusiastic about school administration, believing that it is concerned with petty economic problems (school uniform, inventory purchases, accounting, etc.), which do not seem to belong to education itself [14]. At the same time, he says nothing about the fact that the funding regulated by the school administration is directed to the creation of important conditions for the educational

Table 1. The size of the effects of “school factors” in J. Hattie’s meta-analysis

Source of influence	Average size of the effect (d)	Minimum size of the effect		Maximum size of the effect
		Factor	Value	Factor	Value
Teacher	0.49	Subject knowledge, education	0.1	Micro-education	0.88
Curricula	0.45	Whole text method	0.06	Advanced vocabulary programs	0.67
Student	0.40	Nutrition and diet	0.12	Students idea of their level of knowledge	1.44
Teaching and learning approaches	0.42	Control of the student over extracurricular factors	0.04	Formative evaluation	0.90
Home	0.31	Watching TV	-0.18	Home environment	0.57
School	0.23	Moving between schools	-0.34	Accelerated learning	0.88

Compiled with the use of: Hattie J. Visible Learning . Moscow: Natsional’noye obrazovanie, 2017. 496 p.

process: teachers’ salaries, infrastructure development, buildings, heating, sewerage, etc. In Russia, for example, financing issues are rightfully considered the “cornerstone” of educational policy. Thus, according to the results of the monitoring of the economic situation and social well-being of teachers of the Vologda Oblast, in 2017, 45% of teachers were most concerned about the level of school funding in terms of further prospects for the development of Russian education [15].

It should be noted that the controversial aspects of the results of Hattie’s calculations are largely reflected in the scientific methodology he uses. The sample of “school factors” taken into account by one or another source of influence is not the same. For example, Hattie used 29 different factors to assess the impact of the school, and only 7 and 10, respectively, for the impact of the family and the school [14]. This is due to the limitations of the problems that have been addressed in the writings that formed the basis of the metaanalysis of “Visible Learning”.

To interpret the calculations, Hattie developed an evaluation scale (“barometer of influence”), the basis of which is the so-called “central point” (h) . According to the author, it should determine the typical size of the effect, equal to 0.40. This is a kind of “benchmark for assessing the pedagogical impact” [14, pp. 36-37]. After the release of Hattie’s work, his critics often discussed the appropriateness of choosing this “typical size”. In accordance with the understanding that the “h point” is a conditional watershed of the desired and unwanted results, Hattie formed a “barometer of influence”, which includes the following zones:

• 1.    Negative effect zone (d<0) . Factors with negative effect values are concentrated here; they are the source of “destructive behavior of schoolchildren” [14, p. 34-35]. Hattie included five different factors in the “negative effect zone”: summer vacation, retention (holding students back), moving between schools (source

  – school); family on welfare/state aid, watching television by the student (source – family).

• 2.    Development effect zone (d=0-0.15) . According to Hattie, it includes the teacher’s education (0.11), teacher subject matter knowledge (0.09) and other factors. As Hattie himself says, “similar results can be easily achieved outside the school” [14, pp. 38-40].

• 3.    Low and moderate effect zone (d=0.15-0.4). This included average teacher effect (0.32), use of calculators (0.27), class size (0.21), family structure (0.17), etc. The Influence of these factors leads to the results that the student can achieve in a year of studying at school [14, pp. 38-40].

• 4.    Desired effect zone (d>0.4) . It includes the quality of teaching at school (0.44), teacher expectations of the learning process and its effectiveness (0.43) – average effect; microeducation of teachers (0.88), clarity of teaching (0.75), relationship between teacher and student (0.72), professional development (0.62), inclusion of students to the group of “children with special educational needs” (0.61) – high effect. These factors, according to Hattie, “have the greatest impact on the student’s progress” and are formed in the course of many years of professional activity of the teacher [14, pp. 38-40].

It should be noted how little attention the author of “Visible Learning” pays to the educational competencies of the teacher. In essence, he says that it is important for a teacher not so much to have strong subject knowledge, but rather to teach a subject in a non-boring and accessible way (in all likelihood, this conclusion ignores the relationship between educational training and the quality of professional activity). Hattie thinks that the impact of pedagogical education programs (implemented in universities) on the performance of the school is doubtful, since “the low quality of teacher training is the main obstacle to pedagogical education”, a small effect of which is fairly compensated by the experience acquired on the job [14, pp. 160-161].

According to Hattie, a special place in the system of pedagogical skills belongs to the ability to establish verbal and mental contact with children (“active learning strategy”) [14, p. 339]. He argues that the most important task for the teacher is to form a situation of “visible learning” built on the type of feedback (“teachers see the educational process through the eyes of students – the students see themselves as their own teachers”) [14, pp. 328329]. Maintaining such a situation, according to Hattie, requires constant monitoring. He believes that “the starting point in addressing educational problems should be neither textbooks, nor the usual lesson plans, but the desired learning outcomes – success criteria that correspond to educational intentions” [14, p. 329]. As such criteria, Hattie proposes indicators of the “desired effect zone” [14, p. 39]. “The main thing is for the pedagogical community to acquire professional maturity and move from opinions to evidence, from subjective assessments to critical ones” [14, p. 358].

On the basis of the review of Hattie’s ideas, we will try to answer the question: what kind of teacher, in his opinion, is necessary for the “school of the future”? The place of the teacher, in all probability, remains very significant; but it should be the teachermoderator of the educational path of the child, rather than a professional teacher. The school as an institution in the paradigm of “Visible Learning” is presented purely as a bureaucratic machine – a controller of teacher performance in accordance with a system of pre-selected indicators. Such a position has been criticized by the scientific and pedagogical community of Western countries, although it has managed to take root in educational policy.

The scientific tradition of criticism of Hattie’s ideas. We should say that after Hattie’s book was published in 2009, it immediately acquired the status of the “Holy Grail of education” [16, pp. 425-438]. I. Snook and others write that the concept of “visible learning” led to a great debate in society and attracted the attention of politicians [17, pp. 93-106]. S. Eacott compares “visible learning” with the myth about “a great man who can save education” [18, pp. 413-426]. N. Brown believes Hattie’s efforts are “enormous and commendable”, although he notes that he disagrees in many ways with his methodology2. I. Arnold, in turn, calls Hattie’s work quite convincing, and, in his opinion, most teachers will agree with the conclusion that “effective learning cannot take place without proper feedback from teacher to student” [19, pp. 219-221]. According to P. DeWitt, the author of “Visible Learning” is “open and honest about the lessons he has learned from life”3.

Moreover, the influence of this work on the educational policy of a number of foreign countries is very great, which is especially noticeable in Australia (Hattie in 2011 became Director of the Melbourne Education Research Institute at the University of Melbourne). S. Eacott writes: “Hattie’s work is everywhere in contemporary Australian school leadership” [18, pp. 413-426]. In this country, the “quality teaching model” (QTM) has been adopted at the legislative level, which is largely based on the postulates of “Visible Learning”. QTM is a document that is essentially a “theoretical framework for evaluating teacher performance and behavior in the classroom” [20, pp. 340-344]. Thus, in Australian education, “managerial rhetoric has concentrated around the idea of rationality” [18, pp. 413-426]. The concept of “visible learning” is well established in the United States, where Hattie’s idea of the advantage of small classes for the performance of schools was received well and was adopted by the social movement “Class size matters”. It actively lobbies for the maximum reduction of

the number of pupils in classes in government circles and at the local level4. H. Knudsen in his article notes that the book “Visible Learning” had a great influence on Danish schools [21, pp. 253-261].

Let us emphasize at the same time that Hattie’s work received a warm welcome in Russia. His ideas are actively discussed at webinars for teachers. Based on the analysis of his ideas, Russian researchers come to the conclusion that it is necessary to use meta-analysis to “check the productivity of pedagogical innovations” [22, pp. 7990]. E.A. Sokolova finds the connection of Hattie’s research with the provisions of the Federal State Educational Standards (FSES) concerning the formation of critical thinking in schoolchildren [23, pp. 6-14]. N.A. Borisenko calls the publication of “Visible Learning” one of the main events in the field of publishing translated pedagogical literature in recent years. She notes that so far it is the only scientific work “in which the most important factors affecting the educational achievements of schoolchildren are evaluated” [24, pp. 257-265].

At the same time, Hattie’s research results and conclusions are highly criticized by many modern scientists (Tab. 2). For instance, A. Kamenetz considers the theory of “visible learning” one of the “big ideas in education that do not work”⁵. What is the essence of these doubts? First of all, the scientific methodology Hattie uses has been heavily criticized. In his mega-analysis, Siebert J. Myburgh and his colleagues identify six “chronic problems”:

• 1)    “trash in the trash” : in his analysis, Hattie indiscriminately included data from poorly designed and poorly planned studies (critics call them “extreme”);

• 2)    “displacement of publications” : Hattie-relies on a set of already published data, which

  date back to the 1980s–1990s and are no longer relevant (this point of view is supported by N.A. Borisenko);

• 3)    “comparing apples to oranges” : the comparison of initially disparate studies, to which Hattie resorts, evokes the “subjectivity, reproducibility and generalizability” of his findings;

• 4)    “incorrect use of effect size” : averaging the values of variables into a single indicator, as Hattie did, is incorrect; in this regard, the size of the effect “cannot be causal” and be a panacea for the educational community;

• 5)    “empirical bias”: in his analysis, Hattie uses not all indicators, but only empirically achievable ones: for example, he ignores the results of qualitative research;

• 6)    “limitations of application” : Hattie’s conclusions are not related to the conceptual understanding of educational reality, because they do not take into account the factor of social interaction, which is very important for education [25].

While much of criticism of Hattie’s ideas has been focused on mathematical missteps in the application of meta-analysis mechanisms, L. McKnight and B. Whitburn in their work “Seven reasons to question the hegemony of visible learning” presented a cultural assessment of his work. As a result, they found a close connection between Hattie’s policy statements and some controversial practices of neoliberalism (in particular, it concerns discrimination of schoolchildren on the level of development of abilities and “underprofessionalization” of teachers) [26, pp. 1-13]. According to L. McKnight and B. Whitburn, Hattie’s ideas entail “political baggage”, and also introduce into education the concept of “new administrative panopticism” (“everyone watches everyone”) [26, pp. 1-13]. To do this, the teacher will need to move from internal reflection (characteristic of the teaching profession) to external accountability. In this, the authors see the reactivity of Hattie’s ideas, which they compare even with “educational fascism emanating from poststructural and postmodern doubts about knowledge” [26, pp. 1-13]. They treat what is written in the book as a rational “male view” that undermines the female monopoly on education, which worries the opponents of “visible learning”.

Summing up the generalization of criticism, we note that in the Western scientific tradition (as opposed to Russian) there was an opinion about the incorrectness of Hattie’s evidence. In “Visible Learning”, its research prerequisites are incorrectly defined, therefore the obtained scientific results “cannot be automatically applied in practice without hard work on their unification with personal beliefs, values and experience” [25, p. 18]. It seemed to most opponents that the choice of indicators of meta-meta-analysis was not due to anything (except for the scattered information that was in the hands of the researcher). As a result, the conclusions of his book are questioned.

We can assume that such criticism would not have arisen at all, if some of Hattie’s ideas did not seem too controversial (as, for example, the idea of the weak importance of subject matter knowledge for the work of the teacher, or the unimportance of the school as an institution). At the same time, the indicators given in the evaluation system do not always reflect the educational realities. Scientists are surprised why “the list of considered factors does not include the school textbook (or its analogue) as the main means of education” [24, pp. 257-265].

Hence the conclusion: “The book is perceived as a panacea for the educational community, but as a result of taking this drug, the school will experience a short-term placebo effect” [25, p. 13].

However, the question concerning the validity of the doubts of Hattie’s opponents is not so unambiguous. Rather, the arguments of the current controversy can only be called hypotheses, because they are often strictly theoretical and emotionally colored. So far, the majority of critics have not tried to test the strength of Hattie’s model mathematically (although Hattie uses calculated data). We will try to fill this gap.

Testing Hattie’s evaluation model. To test Hattie’s evaluation model, we use the method of exact distribution of variation coefficients – a statistical characteristic used in the analysis of measurements of random variables (like the data of the meta-analysis under consideration) [27, pp. 166-171].

The calculation of the coefficient of variation is usually necessary to substantiate the reliability of the selected variables by estimating the homogeneity of the samples and comparing the spread of random parameters. It is often used to check the safety of machines and structures (in mechanics and engineering), as well as to compare the dispersion of values relative to the expected value (in the social sciences) [27, pp. 166-171].

To assess the quality of the meta-analysis model, we calculated the coefficient of variation (V σ ) as a percentage of the average deviation of the sizes of variables (effect sizes) to their average value in accordance with the formula :

• V- = - x 100%, x

where V σ is the coefficient of variation of the variables,

σ is the average deviation of the size effects, x is the average size of the effects.

Negative values were not taken into account during the testing.

According to the results of the calculations, the variation of the variables used by Hattie exceeds 50%, which is higher than the statistical threshold of sample homogeneity (33%). The same conclusion can be drawn with respect to individual sources of influence – the home (39%), the school (66%), the student (41%), the teacher, teaching and learning approaches (51% each), and the curricula (42%; Tab. 3 ).

Table 3. Variation coefficient of the variables used in J. Hattie’s meta-analysis, in %

Source of influence	Number of observations taken into account	Variation coefficient (V σ )	Deviation from the optimal value (V σ = 33%), +/-	Nature of the data set
All indicators	133	56.87	-23.87	Extremely uneven
According to individual sources of influence
Home	5	38.57	-5.57	Not even enough
School	25	65.97	-32.97	Extremely uneven
Student	19	71.24	-38.24	Extremely uneven
Teacher	10	51.01	-18.01	Extremely uneven
Teaching and learning approaches	49	50.65	-17.65	Extremely uneven
Curricula	25	41.95	-8.95	Extremely uneven
Calculated with the use of: Hattie J. Visible Learning . Moscow: Natsional’noye obrazovanie, 2017. 496 p. Note. Only positive values (133 out of 138) were taken into account in the calculations.

To interpret the calculated data, we turned to the recommendations of experts, according to which the coefficients of variation with values less than 17% indicate an absolutely homogeneous set of data, in the range from 17 to 33% – sufficiently homogeneous; in the range from 35 to 40% – insufficiently homogeneous, and the coefficients of variation more than 40% indicate a high oscillation of the feature in the aggregate⁶. The aggregate used by Hattie belongs to the latter group ( V σ = 56.87%) .

Thus, the verification of the evaluation model under consideration allows us to talk about different degrees of heterogeneity of the data set used in Hattie’s study. This is least noticeable in the case of family factors, most of all-in the case of the influence of the student and the school. In turn, this means that determining the strength of any effects in such a model, due to its instability, will likely not lead to the formation of objective conclusions. Thus, the hypothesis of Hattie’s opponents about the absence of any control over the calculations during the meta-analysis is confirmed.

Conclusion. Summarizing the above, we note the strengths and weaknesses of Hattie’s meta-analysis. On the one hand, we cannot but agree with some of his conclusions. The life of teachers within the new paradigm of education and the challenges of digitalization is significantly changing, and their situation in society continues to deteriorate. According to the All-Russian Center for Public Opinion Research (VTsIOM), in 2018, only 16% of Russians considered the work of a teacher prestigious7. This result proves that the status of a teacher in the Russian Federation is insufficiently high, while the situation is opposite in other countries. Thus, according to the results of the TALIS-2013 study, in Asian countries (United Arab Emirates, Korea, Malaysia and Singapore) every second school teacher out of three is confident in the social significance of their profession [28, pp. 7-11]. In this regard, Hattie’s contribution is certainly high. In his work, indeed, he expresses a noble desire to substantiate scientifically the importance of the work of the teacher for the education and moral development of children. This very desire has received the most positive feedback in Russia8. However, can the importance of the teacher for the school become the subject of scientific substantiation? Is it not self-evident?

On the other hand, we can highlight several aspects that cause distrust in Hattie’s scientific argument. First of all, he made a number of methodological errors in the meta-analysis, one of which is the use of inhomogeneous medium-sized effects. When accumulating them into a common indicator for individual sources of influence, data can be obtained that cannot be trusted (as evidenced by the results of our calculations). Apparently, Hattie ignored the data validation phase, which included the effects that should ideally have been rejected.

We also share the opinion of Hattie’s opponents that it is extremely difficult (and is it necessary?) to measure the processes taking place in educational systems using any evaluation methodology. At school, there are important phenomena that are not amenable to understanding from the point of view of standardized approaches and statistics: communicating, values, mutual understanding, etc. The problem of taking them into account is solved in the framework of the qualitative rather than quantitative (accounting) approach that Hattie uses.

Let us also emphasize the problem of “underprofessionalization of teachers” in Hattie’s concept (to which L. McKnight and B. Whitburn pay attention). According to Hattie, the teacher is important, but not as a professional, but as a moderator of the free development of the child. This thesis was borrowed by “Visible Learning” from the tradition of unschooling. However, the idea that “everyone can become a teacher” can be found in the amendments adopted in 2016 to the current professional standard “Teacher” in the territory of the Russian Federation9. It seems that the “underprofessionalization” of teachers can only reduce the social importance of this profession and negatively affect the formation of vocation among young professionals. Today, vocation is one of the few aspects of teachers’ work that helps them survive the difficulties of educational reform and the risks of “burnout”, as evidenced by the results of sociological studies [15].

Thus, we can conclude that the implementation of Hattie’s ideas within the framework of Russian educational policy is inapplicable, since it can cause negative consequences and extend the range of new problems (in particular, the increase in the bureaucratic functionality of the teaching profession in the pursuit of “school leadership”).

For example, the education system of Australia already feels such effects (“a tragedy in the Australian educational leadership” – this is how S. Eacott calls the policy on “continuous production of data”, which became common for schools in this country after the first attempts to implement Hattie’s ideas in practice) [18, p. 422]. Here it is necessary to point out that a similar problem concerns modern Russian education as well. According to the data of the all-Russian monitoring of the Russian Academy of National Economy and Public Administration (RANEPA), 79% of teachers of secondary schools pointed out that in 2018 the reporting component noticeably increased in their professional activity; as a result, their work “is becoming more difficult” [29, p. 25]. A number of experts already call the continuous growth of document circulation an “organizational pathology” of the education system10. At the same time, according to a regional study, more than a quarter of school teachers who wish to find a new job as soon as possible are concerned about the bureaucratization of their profession (even the educational reforms carried out at the federal level were not such a significant factor in the formation of such plans) [15]. Thus, coupled with the low prestige of teaching in society, even a slight increase in “bureaucratic pressure” (inevitable in the framework of the concept of “visible learning”) can become a catalyst for protest activity of teachers of schools in the form of mass “withdrawal from the profession”. Of course, it is necessary to search for and use new solutions in Russian educational policy, but it requires scientific substantiation, taking into account the adaptive capabilities of the agents of the institutional system and the established traditions. Otherwise, the consequences of implementing such solutions can be catastrophic.

Hattie’s work, in our humble opinion, warns practitioners about the need for a strict selection of solutions to educational issues (from updating curricula to finding the means suitable for the development of human resources of educational institutions). Indeed, in Hattie’s work, you can find simple answers to rather complex questions: what works at school? what doesn’t work there? what you need to invest money in, and what to save? R. Slavin, Director of the Center for Research and Reform in Education at Johns Hopkins University writes: “How wonderful to have every known variable reviewed and evaluated!”11. In education, however, there is little that “works” and little that “does not work”. The correct question is: “Under what conditions will this work in school?” But the task of finding these conditions concerns not so much the evaluation of the activities of educational organizations as the effectiveness of public administration.