Journal rankings: theory, methodology, bibliometry
Автор: Rubinstein A.Y., Burakov N.A.
Журнал: Economic and Social Changes: Facts, Trends, Forecast @volnc-esc-en
Рубрика: Theoretical and methodological issues
Статья в выпуске: 6 т.16, 2023 года.
Бесплатный доступ
The article presents the findings of a scientometric study of publication activity and ranking of journals registered with the RSCI on the subject “Economics. Economic sciences”. The analysis of a new methodology proposed by the RSCI for determining journal rankings has revealed its serious shortcomings. The article formulates three main critical postulates to prove that the journal ranking results derived according to this methodology are incorrect. One of its most vulnerable aspects is the unsubstantiated choice of journals carried out by an anonymous group of experts for the purpose of determining the composition of the RSCI core. The situation is complicated by the fact that this subjective choice leads to the discrimination of other journals, whose citation rate is recorded in the RSCI only if the article is cited in a journal that is part of the core. The choice of bibliometric indicators determining the aggregate rating of journals is unsatisfactory, and first of all this applies to the indicator “length of the article text”, which, strictly speaking, has no relation to scientometric indicators whatsoever. The formula for determining the aggregate rating itself, in which the weights of bibliometric indicators are “adjusted” to comply with a certain expert rating from five years ago, cannot be called justified. The constructive criticism provided in the article is accompanied by a description of an alternative approach with the elimination of the identified shortcomings of the new RSCI methodology. We are talking about a fundamentally different way of allocating the journals core: it does not use subjective assessments of any expert groups; instead, it uses certain formal criteria related to the production of knowledge and its dissemination in the form of publication activity of academic institutes of the Russian Academy of Sciences and leading universities that are founders of economic journals. The article presents a new approach to the methodology for determining a journal rating based on the application of MW analysis (Multiway data analysis), which is a generalization of factor analysis applied to a multidimensional matrix. As a result, we put forward another set of journals included in the core; this ranking is called Yadro.RU; we also propose a more reasonable version for the ranking of journals registered with the RSCI on the subject “Economics. Economic sciences”.
Journals, citation, impact factor, hirsch index, herfindahl index, aggregate rating, ranking, rsci core, yadro.ru, mw analysis
Короткий адрес: https://sciup.org/147242465
IDR: 147242465 | DOI: 10.15838/esc.2023.6.90.10
Список литературы Journal rankings: theory, methodology, bibliometry
- Adler R., Ewing J., Taylor P (2011). Citation statistics. In: Igra v tsyfir', ili kak teper' otsenivayut trud uchenogo: sbornik statei o bibliometrike [The Game of Figures, or How a Scholar’s Work is Evaluated Now: Collection of Articles on Bibliometrics. Moscow: ICNMO (in Russian).
- Aleskerov F.T., Badgaeva D.N., Pislyakov V.V., Sterligov I.A., Shvydun S.V. (2016). An importance of Russian and international economic journals: A network approach. Zhurnal Novoi ekonomicheskoi assotsiatsii=Journal of the New Economic Association, 2(30), 193–205 (in Russian).
- Arrow K. (1962). The economic implications of learning by doing. The Review of Economic Studies, 29(3), 609–626.
- Brin S., Page L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ICDN Systems, 30, 107–117.
- Burakov N., Bukhvald E., Kolchugina A. (2019). Regional index of economic development and ranking of the subjects of the Russian Federation. Federalizm=Federalism, 3, 149–171 (in Russian).
- Burakov N., Rubinstein A. (2020). Theoretical and applied aspects of measuring the economic growth potential of Russian regions. Prostranstvennaya ekonomika=Spatial Economics, 1, 24–50 (in Russian).
- Burakov N.A. (2021). Intangible assets and economic growth in the cultural sector. Voprosy teoreticheskoi ekonomiki=Issues of Economic Theory, 4, 92–104 (in Russian).
- David P., Foray D. (1995). Accessing and expanding the science and technology knowledge base. STI Review, 16.
- Foray D. (2006). The Economics of Knowledge. The MIT Press.
- Foray D., Mairesse J. (1998). Innovations et Performances: Trois Experiences de Collaborations Interdisciplinaires. P.: Edition de l’EHESS.
- Garfield Eu. (1955). Citation indexes for science. Science, 122(3159), 108–111.
- Hayek F.A. (1945). The use of knowledge in society. The American Economic Review, 35(4), 519–530.
- Idei i chisla. Osnovaniya i kriterii otsenki rezul'tativnosti filosofskikh i sotsiogumanitarnykh issledovanii [Ideas and Numbers. Grounds and Criteria for Evaluating the Effectiveness of Philosophical and Socio-Humanitarian Research] (2016). Moscow: Progress-Traditsiya.
- Ivanova N.I. (2002). Natsional'nye innovatsionnye sistemy [National Innovation Systems]. Moscow: Nauka.
- Kozyr-Kowalski (1967). Max Weber a Karol Marks. W-wa
- Machlup F. (1962). The Production and Distribution of Knowledge in the United States. Princeton, N.J.: Princeton University Press.
- Machlup F. (1984). The Economics of Information and Human Capital. Princeton, N.J.: Princeton University Press.
- Machlup M. (1966). Proizvodstvo i rasprostranenie znanii v SShA [The Production and Distribution of Knowledge in the United States]. Moscow: Progress.
- Makarov V.L., Kleiner G.B. (2007). Mikroekonomika znanii [Microeconomics of Knowledge]. Moscow: Ekonomika.
- Maunoury J.-L. (1972). Economie du savoir Collection U. Published by Armand Colin.
- Nazarenko M.A. (2013). Scientometric indicators of the rating of the Russian Science Citation Index. Uspekhi sovremennogo estestvoznaniya=Advances in Current Natural Science, 7, 178–180 (in Russian).
- Rubinstein A.Y. (2023). About scientometric rankings and journal BACchanalia. Ekonomicheskii zhurnal VShE=HSE Economic Journal, 27(2), 290–305 (in Russian).
- Rubinstein A.Y., Slutskin L.N. (2018). “Multiway data analysis” and the general problem of journals’ ranking. Prikladnaya ekonometrika=Applied Economietrics, 50, 90–113 (in Russian).
- Seglen P. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ, 314(7079), 498–502.
- Waltman L. (2016). A review of the literature on citation impact indicators. Journal of Informetrics, 10(2), 365–391.
