Methodological tools for assessing the effectiveness of implementation of the state social and economic development policy of Russia's regions

The achievement of goals and obtainment of desired results within the framework of implementation of the state social and economic policy is an important condition for sustainable socio-economic growth of the region. The improvement of the public administration efficiency is stimulated by the introduction of mechanisms and procedures of policy assessment, which are in constant dynamic change.

During the development of the author’s approach to the evaluation, we relied on the works of domestic and foreign researchers, as well as on the current state mechanisms of policy evaluation. Numerous approaches to assessment can be grouped as follows: by the function of the area of assessment [1; 6; 8]; evaluation objects (policies, programs, projects); used evaluation methods; assessment target; realization levels (local, regional, national); policy implementation stages (pre-assessment, accompanying assessment, summative assessment) [8]; assessment subject [7].

According to the function of the area of assessment, the scientists single out approaches focused on the evaluation of the results of comprehensive policy, carried out, for example, in the region [8; 14; 16]. The project SIRENA, developed by the Novosibirsk researchers helps simulate socio-economic development of the region, including in the short-and longterm perspective, and assess the impact of management decisions of the center on the development of individual regions or their combination by given socio-economic indicators. The information system for monitoring and evaluation of regional development (RAUNIS), used abroad, is analogous to SIRENA [13]. The assessment of efficiency of enforcement authorities’ activities in the RF subjects is an example of the integrated policy assessment mechanism adopted at the state level [8]. At the same time, there are mechanisms that focus on the estimation of individual areas of socio-economic development: in the field of evaluation of education policy [17], healthcare and employment [4].

Most frequently used methods are the following:

– method of integral estimates [8] with the use of statistical indicators and data of sociological research;

– method to compare the actually achieved rate and the planned one [12];

– analysis of “cost-outcome” (costeffectiveness analysis, analysis of “costbenefit”) [6; 11-13];

– methods of subjective evaluations (based on surveys) [2; 5];

– methods to evaluate factors and risks [5; 10].

We can consider an assessment target, such as result of the executive bodies’ activities in a Russian Federation subject [7]; socio-economic implications of management decisions [10]; efficiency of government spending [2; 11; 12]. At the present time there is no common view on content aspects of the categories “effectiveness” and “performance efficiency” of a policy that leads to ambiguity and complexity of evaluation. The concept “effectiveness of implementation of state policy” is usually interpreted as the degree of objectives achievement, reflected in the improvement of socio-economic indicators of regional development within the planned timeframe. Due to the proximity of the concepts “effectiveness” and” of a policy, policy efficiency is seen as an integral part of the effectiveness. Traditionally, the concept “effectiveness” is interpreted as the ratio of outputs and costs and is quite narrow. In the broad sense effective management should be viewed as a complex category, which, on the one hand, is understood as the “state of economy, when a certain ratio of output and costs to achieve this output gives the subject the opportunity to obtain maximum results due to the use of resources of a certain value or leads to the production of goods of a certain value at the lowest costs of resources” [11]. On the other hand, it is a broad understanding of management effectiveness that provides an assessment of economic, social and technical effectiveness. Depending on the subject of the evaluation we can differentiate external (on the part of higher authorities or third party organizations [2; 5; 9; 10]), internal or self-assessment [11; 12].

The majority of methods to estimate the authorities’ activity are characterized by the lack of complexity of the used systems of indicators, insufficient elaboration of the evaluation criteria and concentration mainly on the analysis of quantitative indicators. It is necessary to take into account regional specifics of policy implementation – in our case, the so-called special northern conditions. The author’s methodological tools to assess the effectiveness of implementation of the state social and economic policy aimed at developing northern regions of Russia helps evaluate the effectiveness of implementation of the policy goals and objectives in the systematic relationship with institutional aspects and policy implementation conditions. The methodological tools are based on the principles of balance and complexity. The balance principle is focused on ensuring the balance between short-term and long-term goals of regional development, the balance of domestic interests in the region’s development and interests of the state, the compliance of indicators of upper levels of the hierarchy with lower levels. The complexity principle is based on the inclusion of equivalent aspects of the policy effectiveness assessment: effectiveness of goals and objectives achievement; quality of state regional development programs; conditions for the implementation of socio-economic policy in the northern regions.

The algorithm to evaluate the implementation of socio-economic development policy in the northern regions of Russia includes (figure) :

Stage 1. Assessment of the socio-economic situation in the region

It includes characteristics of the geographic location, the level of economic assimilation of the territory, the sectoral structure of economic complex, the investment activity of the territory, features of the demographic situation in the region, the level and quality of life, the financial condition of the territory.

The study of the socio-economic situation in the region is necessary to identify potential opportunities and threats for the region’s development of that require management decisions.

Schematic diagram of the methodical approach to evaluating the effectiveness of implementation of the state social and economic policy in Russia’s Northern regions

System of indicators to assess the effectiveness of achieved goals and objectives of the regional socio-economic policy

Goal	Tasks	Quantitative indicators	Quality indicators
Improvement of the territorial organization of economy	Formation of the stable settlement system, creation of clusters, “growth points”	Population density, persons/km2 Level of economic assimilation, GRP/km2	Missing
Ensuring high and sustainable economic growth rates	Development of foreign economic and interregional relations in the interests of the region’s economic development	Share of foreign trade turnover of the region in the foreign trade turnover of the country, %	Missing
	Diversification of the industrial production and the structure of the economy	Share of manufacturing industries in GRP, % Share of the employed in manufacturing industries, %	Missing
	Increasing the efficiency of regional economy	Specific residential electricity consumption, kW/h/person Electricity intensity of GDP, kW/h/thousand rubles	Missing
	Modernization and development of the innovative activity	Volume of innovative products (works, services), million rubles Internal costs on R&D to GRP,%	Missing
	Improvement of the socioeconomic effectiveness of natural resources use	Share of revenues of the region’s consolidated budget from the use of natural resources, %	Missing
	Development of small and medium business	Number of enterprises of small and average business, units/10000 persons Share of the employed at enterprises of small and average business, %	Entrepreneurial activity level Satisfaction with business environment
	Tourism development	Number of places in hotels, units/1000 persons Number of places in food courts, units/1000 persons	Satisfaction with created conditions for tourism development in the region Assessment of the region’s attractiveness for tourism development
Strengthening the financial state of the region	Increasing investment inflow to the region and implementation of the investment projects	Volume of investment in fixed capital per capita, million rubles/person Rating of investment attractiveness of the region, place	Missing
	Improving interbudgetary relations relations (refuse of subsidies)	Share of own tax and non-tax) revenues in the consolidated budget of the region, % Own revenue of the consolidated budget per person, rubles	Missing

Continuation the table

Stage 2. Justification of strategic goals and tactical objectives of the region’s development and list of quantitative (objective) and qualitative (subjective) indicators reflecting the efficiency of their achievement.

It contains a comprehensive analysis of the strategic documents defining socioeconomic development of Russia’s Northern regions and results of comprehensive assessment of the socio-economic situation in these regions. The objectives of the state socio-economic policy are grouped into 7 blocks within the identified strategic objectives of policy. For each task we substantiate the use of the indicator, which dynamics of changes would indicate the effectiveness of its implementation. The evaluation system includes quantitative and qualitative indicators (table) . The quantitative indicators of performance efficiency assessment are based on statistical data and the qualitative ones – on results of the sociological survey of population and experts.

Stage 3. Comprehensive evaluation of the state socio-economic policy implementation in the Northern territories

It is carried out according to the method, including 3 blocks: assessment of the impact of implementation of the socio-economic policy objectives and tasks; assessment of the conditions of state policy realization; assessment of the quality of drafting of the regional government programs.

Block 1. Method to assess the performance efficiency of implemented goals and objectives of the state socio-economic policy in the Northern territories

The strategic goals, tactical objectives and outcomes should be consistent with the final significant effect and the choice of indicators [11, 12]. In accordance with this approach, we developed the system of quantitative and qualitative indicators that reflect the effectiveness of implemented goals and objectives of the state socioeconomic policy development in the Northern territories.

The total integral indicator to assess achieved goals and objectives of the state socio-economic policy in the Northern territories is a sum of the policy efficiency indices on the basis of quantitative and qualitative indicators, adjusted by a weighting factor:

TII = 0.8 x EI + 0.2 X CI

To justify the weighting factor value we used the method to assess the performance of the executive authorities in the RF subjects, developed by the Ministry of Regional Development of the Russian Federation [7]. The advantage of this approach lies in the possibility of obtaining more objective evaluations of the performance of the executive authorities due to the adjustment of the Federal State Statistics Service data by sociological surveys. The proposed performance indices help evaluate the efficiency of policy implementation in the Northern regions of the Russian Federation and identify “weak points” by calculating indices by blocks of tasks.

Block 2. Method to assess the impact of conditions on the implementation of the state social and economic policy of Northern territories development

The problems of socio-economic development and the practical measures of state bodies to regulate regional development should be systemically considered by analyzing the conditions that affect the formation and implementation of regional socio-economic policy and taking them into consideration when developing program and strategic documents. The developed method includes the typology of conditions, the system to characterize their indicators and the algorithm to evaluate the effect of conditions on policy performance. It takes into account the following groups of conditions: resource and climate; geographical; economic; social; demographic; financial; institutional.

The first group describes climatic conditions and resource potential of the territory. The demand and the development features determine the main vector of the Northern region development. This group is assessed by bioclimatic index of the weather mode severity; integrated index of the availability of natural resources and the potential sustainability of natural ecosystems, expressed in points.

The second group describes geographic and geo-economic features of the Northern region and possibility of its use for socioeconomic development. This group of conditions is evaluated by the following indicators: GDP per unit area (rubles/ km2); population density (persons/km2); density of paved public roads (km/ thousand km2); coefficient of energy supply in the region; cost of a fixed set of consumer goods and services (% of the national average); share of foreign trade turnover of the region in the foreign trade turnover of the country (%).

The third group of conditions is used to evaluate the level of accumulated economic potential and competitiveness of the region. The assessment indicators are the following: volume of GDP per capita (rubles/person); share of GRP in the total GRP of the Russian Federation (%); cost of fixed assets (rubles/person); GRP per employee in the region’s economy (rubles/person); volume of innovative goods, works, services (million rubles).

The fourth group characterizes the level of social development of the territory and is associated with the development of living standard. These conditions are evaluated by the following indicators: number of economically active population (thousand persons); share of economically active population of the Russian Federation subject in the economically active population of the country (%); purchasing power of money incomes of the population (times); ratio of purchasing power of money incomes in the region and the national average (%); retail trade turnover (million rubles); share of retail trade turnover of the region in the total retail trade turnover of the country (%).

The fifth group describes demographic potential of the population and characteristics of the reproductive processes based on the study of natural and mechanical movement of the population through the assessment of indicators of the region’s population (thousand persons), natural and migration growth (‰).

The sixth group helps assess the financial condition of the territory and is estimated by the following indicators: share of own revenues in the consolidated budget of the region (%); level of expenditures of the region’s consolidated budget per capita (rubles/person).

The seventh group characterizes the degree of the region’s involvement in the program regulation of socio-economic development and the level of budget subsidization through the use of indicators: share of investment from the federal and regional budgets in the total amount of investment in fixed capital of the region (%) and proportion of uncompensated receipts to the region’s consolidated budget (%).

The algorithm to evaluate the effect of conditions on the state policy efficiency includes the following steps:

• 1.    Definition of the main groups of conditions to implement the state social and economic policy of Northern development and the metrics.

• 2.    Construction of the correlation matrix to determine the tightness of relationship between conditions and identification of conditions maximally independent from each other.

• 3.    Construction of regression models to determine the degree of correlation between indicators, reflecting the policy implementation conditions, and indicators revealing the efficiency of solving problems.

• 4.    Construction of the model to assess the impact of policy implementation conditions on the effectiveness of goals and

objectives of the state policy to develop the Northern regions by using the method of equal intervals.

The use of this method helps evaluate the intensity of the impact of certain conditions on the implementation of specific tasks of socio-economic policy.

Block 3. Assessment of the quality of drafting of the regional government programs

The well-elaborated program document raises the possibility of its use as an effective tool for the socio-economic policy implementation and leads to more effective expenditure of budgetary funds. Based on the analysis of state documents [1, 2] we formed 10 indicators for evaluating the quality of drafting of state programs at the regional level. To quantify them, we use numerical methods, comparing various indicators of the quality of state programs drafting and find a composite index, which reduces all indicators to a single value – points.

The indicators disclosing the quality of drafting of regional programs are:

• 1.    Goal, tasks and priorities identified in the state program should take into account the provisions of strategic documents approved by the RF President and the RF Government, provisions of priority national projects and individual decisions of the RF President and the RF Government.

• 2.    Compliance of the tasks formulated in the regional program to the outcomes of its implementation.

• 3.    Compliance of the indicators of the regional state program to the outcomes stated in it.

• 4.    Compliance of the tasks stated in the regional state program to the set of basic measures of the program.

• 5.    Compliance of the state of the sphere of regional state program implementation to the problems.

• 6.    Compliance of the tasks formulated in the state regional program to the highlighted problems.

• 7.    Presence of justified demand in financial resources for its implementation in the regional state program.

• 8.    Accounting and risk analysis in the regional state program.

• 9.    Presence of risk management measures in the regional state program.

• 10.    Availability of the method to assess the effectiveness of the regional state program.

For evaluation of the considered indicator we calculated the ratio of the targets agreed in federal documents and the regional program and the total number of targets identified at the federal level.

To assess compliance of the tasks to the outcomes we calculated the ratio of the number of tasks that correspond to the final results and the total number of tasks stipulated by the state program.

To estimate compliance of the indicators and the outcomes we calculated the ratio of the number of outcomes that correspond to the adequate performance of their measurement and the total number of outcomes.

The assessment of the compliance of the tasks stated in the regional state program to the set of basic measures of the program involves the calculation of the ratio of the number of tasks that correspond to the activities and the total number of tasks.

To evaluate it we calculated the ratio of the number of problems identified through the analysis of statistical data and agreed with the problems stipulated by the state program and the total number of problems identified on the basis of statistical data analysis.

The assessment of this indicator involves calculation of the ratio of the number of problem and the total number of problems stipulated by the regional state program. The first six indicators are ranked, the points are calculated.

The points by criteria 7–10 are assigned as follows: if the program has necessary financial resources for its implementation, methods of assessing effectiveness, risk analysis and risk management measures, it is assigned 1 point. Otherwise, the program is assigned 0 points.

The identification of the role of each criterion by calculating weighting factors is the next stage of assessment. The weighting factors of the criteria for the quality of drafting of the regional government programs are calculated by the method of expert assessment. According to the direct evaluation method, the coefficients are given to each criterion: the larger the contribution of the criterion, the higher the weighting factor. Then the weighting factors are averaged by averaging arithmetic.

The last stage of the drafting of regional state programs concerns calculation of the final index of the quality of drafting of the regional programs, which represents the sum of scores by 10 quality criteria with weighting factors.

Stage 4. Development of the proposals on improvement of the mechanisms and tools to implement the state social and economic policy development of the Northern territories

On the basis of comprehensive assessment of the effectiveness of government policies we developed recommendations for public authorities. The consideration of different aspects of the assessment allows us to fully characterize the process of achieving these goals and objectives from the perspective of socio-economic results, intensity of the impact of conditions and quality of the state programs development. It helps develop specific recommendations aimed at optimizing the development of individual components of large blocks: socioeconomic, environmental and institutional.

The advantages of the proposed methodological tool are the following: possibility to consider regional specificity, including that of the Northern regions, in the formulation and implementation of goals and objectives of the policy. The problems to take into account the specifics of the North are stated in many strategic and policy documents; however, they are not methodologically supported. The methodical approach helps, on the one hand, consider the problem of the policy effectiveness comprehensively and, on the other hand, develop targeted recommendations for the regulation of various spheres of socioeconomic development and identify reasons for failing to achieve the set goals and objectives.