The impact of Slovenian qualifications framework: stakeholders' perspective

The role of international organisations (IOs) in global and/or European education policy-making has received considerable attention in the scientific community (Caspersen and Frølich, 2017; Jakobi, 2009 ; Kleibrink, 2011 ; Portnoi, 2016 ), particularly their role in the establishment of the European qualifications framework (EQF), a key European policy instrument for lifelong learning (LLL) ( Elken, 2015 ), and the national qualifications frameworks (NQFs) whose establishment have also been strongly supported by various IOs around the globe ( Chakroun, 2010 ; Raffe, 2013 ). Nevertheless, there is still much to be learned about their impact on national education systems, policies and practices ( Portnoi, 2016 ) and the application of the policies from IOs to the national level ( Crossley, 2019 ; Jakobi, 2009 ).

To address this research gap, the current study aimed to contribute to the better understanding of the impact of the EQF-influenced NQFs in Europe. This study explored the impact of the Slovenian qualifications framework (SQF) on the country’s education and training system. We measured the impact indirectly by focusing on the views of the stakeholders, whom we named ‘key stakeholders’, i.e. members of the professional public who have been, by their occupational position, involved in the SQF’s implementation or usage. Furthermore, we looked into the key stakeholders’ views regarding the extent to which the SQF objectives were met and investigated their familiarity with the SQF’s benefits and whether their views differed regarding the SQF’s usage.

In what follows, we briefly introduce the role of IOs in fostering policy transfer and the instruments used in the EQF policy transfer, elaborate on the issue of measuring the impact of the current SQF, outline the methodological approach employed in this work and present the results of empirical study. In the final section, we discuss the EQF’s impacts on the SQF and that of SQF on the education and training system.

© 2021 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license .

International organisations and instruments used to facilitate transfer

Due to globalisation, IOs as agenda-setters, such as the European Union (EU), the Organisation for Economic Co-operation and Development (OECD) and the United Nations Educational, Scientific and Cultural Organisation (UNESCO), among others, play an increasingly crucial role in the formation of the global and/or European education policies. They have been identified as ‘central nodes for policy diffusion’ ( Jakobi, 2012, p. 391 ) that are able to transfer policies between countries and promote their own policies, although their formal competencies are generally limited. Despite the fact that the IOs generate different policy outputs and differ in focus and scope, connections and policy convergence among the IOs’ policies exist ( Jakobi, 2009 ). The case of the NQFs is a good example of such a convergence ( Raffe, 2013 ). As influential actors framing education and contemporary LLL policies, IOs promote policy transfer towards evidence-based educational practices, the measurement of the effectiveness of education and the achievement of goals relating to competitiveness and employability ( Mikulec, 2017 ) through new instruments and practices of governance based on knowledge and data generation, peer learning, benchmarks, indicators, monitoring, evaluation and funding ( Ioannidou, 2014 ; Lawn and Grek, 2012 ).

Therefore, the IOs stimulate a kind of transnational educational policy transfer that encompasses ideas, ideology, practices and institutions, involving multiple actors ( Crossley, 2019, p. 4 ). The process of policy transfer should be understood as a continuum that can incorporate various influences, from voluntary ones to more coercive ones (i.e. policies that are forcibly imposed on countries or those that countries must adopt due to political pressure) ( Portnoi, 2016 ). Furthermore, as Jakobi (2009, pp. 34–36) pointed out, IOs promote policy transfer and influence national policy development through the following identifiable soft instruments: (a) discursive dissemination or the establishment of ideas and goals for national agendas; (b) standard setting or the conventions and recommendations as well as benchmarking, explicit aims and rules to which states should comply; (c) financial means aimed at eliciting specific behaviour (establishing programmes or institutions); (d) coordinative functions or the instruments of surveillance monitoring progress toward common policy aims; and (e) technical assistance enabling states to achieve set policy aims.

European education policy and instruments used in the EQF policy transfer

The EU is one of the key agencies contributing to the formation of European education policy. The adoption of the Lisbon Strategy in 2000 marked the starting point in establishing a European education policy defined by common goals, implementation tools and financial resources, although the EU formal competencies in the field of education were limited by the subsidiarity rule ( Mikulec, 2017 ). For the purpose of formulating and maintaining the European education policy, the European Commission (EC) adopted an open method of coordination (OMC) to improve the effectiveness, coordination and measurability of the outcomes of various LLL policies. The OMC established a new form of multilevel educational governance that (a) is exercised in the form of ‘soft law’ (e.g. recommendations, guidelines, resolutions, conclusions, etc.), (b) is based on soft law voluntary implementation from the member states, (c) and strives to establish monitoring mechanisms through benchmarks and indicators in order to measure and compare the progress of member states ( Ioannidou, 2014 ; Lawn and Grek, 2012 ).

The EQF is one of the numerous soft law recommendations produced by the EU. Whilst the initiatives leading to the development of the EQF can be traced back to 2001 ( Elken, 2015 ), it was not before 2008 that European Parliament and Council issued the EQF recommendations—a ‘translation grid’ with eight-level descriptors—and recommended that member states use it as a common reference tool to compare qualification levels, relate qualifications systems or NQFs to the EQF and use an learning outcome-based approach when defining qualifications. In 2017, the Council (2017) issued slightly modified version of these recommendations to address the problematic issue regarding the concept of ‘competence’, amongst others (cf. Bohlinger, 2019, p. 400 ). Meanwhile, data from the latest Cedefop (2018, pp. 12–13) survey indicated the following: (1) a total of 39 European countries (all 28 member states plus another 11 countries) had adopted recommendations and either established or were in the process of establishing an NQF, (2) most countries had proposed or adopted the EQF’s eight-level structure and established comprehensive frameworks, and (3) 34 countries had formally linked their NQFs to the EQF, whilst 29 countries linked theirs to the Bologna framework. Despite the non-binding nature of the EQF recommendations, the development of EQF-influenced NQFs in Europe in the last decade has been remarkable, notwithstanding the evidence indicating the NQFs’ inability to fulfil the broader set of objectives and purposes they claim ( Mikulec, 2017 ; Bohlinger, 2019 ).

Following Jakobi (2009), the main instruments used by the EU when promoting the EQF policy transfer and influencing national NQF policy development can be identified. On the level of (a) discursive dissemination, the following ideas and goals can be identified: the EQF is supposed to facilitate LLL and employability, promote the mobility and social integration of European citizens and enable greater transparency, comparability and portability of qualifications (Cedefop, 2018; Council, 2017). On the level of (b) standard setting, as the EU issued the EQF recommendations, it must set clear timelines and benchmarks referring to the criteria and procedures for referencing SQF to the EQF, the quality assurance principles linked to the learning outcome-based approach, the outcome-based standards and the monitoring and credit system principles based on the learning outcome-based approach linked to quality assurance and validation of prior learning (Council, 2017). To achieve the EQF policy aims, the EU supported candidate countries and member states through (c) financial means, i.e. actions funded by the EU programmes (Council, 2017) (in case of Slovenia, the SQF development and implementation was supported with European social funds (ESF) in the years 2009–2014 and with Commission grants to the EQF national coordination points (NCP) from 2011 onwards). On the level of (d) coordinative functions that explore the progress made towards common policy aims, the EQF has been governed by the OMC and monitored through the EQF Advisory Group (EQF AG) and other networks (i.e. EQF national coordination points and EQF peer learning activities), whilst the implementation of the EQF and established NQF is one of the indicators in the two ‘flagship initiatives’ for the Europe 2020 Strategy (Mikulec, 2017). Finally, technical assistance is provided in terms of expert advice, because at least two international experts should be included in the EQF referencing process (Council, 2017) (in the case of Slovenia, three international experts were included).

Measuring the impacts of NQFs

Contrary to the numerous publications issued by the EC and contracting agencies in the last 12 years, which argued in favour of objectives the EQF and NQFs should bring – among others improved transparency of national qualification systems, reinforced use of learning outcomes approaches, enabled validation of non-formal learning and mobility of learners, enhanced quality of learning and greater stakeholders engagement ( Cedefop, 2018 ; cf. Raffe, 2013, p. 147 ; Werquin, 2007, pp. 466–468 ; Young, 2007, pp. 449–500 ) –, critical scholars emphasised that NQFs are, in fact, unable to fulfil the broader set of objectives and purposes they claim and even problematised the idea of measuring the impacts of the EQF and NQFs. More than ten years after the EQF implementation, we still ‘know little about its actual impact’ ( Bohlinger, 2019, p. 403 ), and member states adopted the EQF ‘on paper while decoupling it from Member States’ de-facto implementation’ ( Bohlinger, 2019, p. 395 ). Raffe (2013) pointed out that the impacts of NQFs are difficult to investigate, as they are not yet fully implemented and/or not functioning for long enough to measure such impacts. Similarly, Allais (2010, 2011) emphasised that the NQF’s impact on the national education and training system may be difficult to measure, because ‘the concepts and categories used to measure performance may be changed by the NQF itself’ ( Allais, 2010, p. 91 ). She added that the NQF cannot be researched and evaluated as ‘policy mechanism[s] in their own right’ ( Allais, 2017, p. 769 ), but should rather be examined in a broader context of (education and labour market) systems and institutions, the relationship between them and the NQF and other ongoing reforms in the country ( Allais, 2017, p. 775 ). Pilcher, Fernie and Smith (2017, p. 9) critically reminded that studying the impacts of NQFs is almost impossible due to the semantic vagueness of the term ‘impact’, the differences in NQFs’ philosophies and objectives and the methodological complexities; thus, such a task is ‘a dream for which it is impossible to identify a suitable yardstick to measure’. Yet, a clear definition of the term ‘impact’ and the conduct of independent research focusing on fundamental questions can help raise the value of such research.

The current study’ research design takes these methodological limitations into consideration. The SQF’s overall impact is studied 10 years after it came to existence, eight years after its referencing to the EQF and five years after the SQF Act was adopted. Moreover, the study is limited to the SQF’s impact on the education system (excluding the labour market), by exploring the extent to which SQF objectives have influenced changes in education system: (1) improved recognisability and understanding of qualifications, (2) improved transparency of qualifications and qualifications subsystems, and (3) reinforced LLL support. The study examined the SQF in the context of wider education reforms undertaken before the SQF development as well. Finally, the impact is examined from the perspective of those stakeholders, whose occupational positions make them the primary users or implementers of the framework: they are the ones expected to implement it into various official and curricular documents and use it for different types of career, education and learning guidance and counselling activities. In that regard, they can be considered a representative sample.

The case of the SQF

Context and settings

After gaining independence in 1991, Slovenia reformed its entire educational system in the first half of 1990s based on European values regarding the principles of human rights and justice, the autonomy of students and teachers, quality provision and comparable European educational standards. Furthermore, it introduced the outcome orientation of the curricula and standards of knowledge as the basis for national external testing of knowledge, the essential purposes of which were to monitor the quality of educational institutions and to strengthen the autonomy of teachers ( Ermenc, 2014 ; Mikulec and Ermenc, 2016 ).

In 1998, Slovenia started to prepare for its formal accession into the EU and later became a member in 2004. This triggered a second wave of educational reforms, especially in vocational, higher and adult education subsystems, which can also be attributed to the extensive financial support from European project funds. Slovenia was one of the 29 countries that signed the Bologna Declaration in 1999 and, in the coming decade, reformed its higher education system in line with Bologna requirements leading to the following: (a) a three-cycle degree structure, (b) the use of the European Credit Transfer and Accumulation System (ECTS) and learning outcome-based approach in curriculum development, (c) the use of the European Standards and Guidelines for Quality Assurance (ESG), (d) the establishment of the NQF in line with the EQF and framework of qualifications for the European higher education area (FQ EHEA) and (e) the establishment of recognition procedures in line with the Lisbon Recognition Convention ( Ermenc and Mikulec, 2020 ). Furthermore, with the Lisbon Strategy set in 2000, Slovenia participated in its implementation and fulfilled the Education and Training 2010 programme in vocational education and training (VET) and adult education from 2002 on, in addition to monitoring and reporting on already achieved objectives. The reforms focused on (a) quality assurance, (b) transparency of (vocational) qualifications based on learning outcomes, (c) implementation of decentralised competencebased curricula, (d) development of flexible and individualised learning paths, (e) development of the LLL strategy, (f) implementation of key competences and validation of informal learning and (g) implementation of European transparency tools (SQF, Credit Transfer System for VET (ECVET), Europass) ( Ermenc, 2014 ).

Therefore, the implementation of the Bologna and Lisbon processes in the first decade of the 21 ^st century triggered a variety of educational reforms, with which the economic goals of education came to the fore, and the SQF’s development, which started in 2009 and was one of the last initiatives to be implemented from the Lisbon period in Slovenia and built on previously undertaken reforms.

SQF features

The SQF development in Slovenia started in 2009, when the ‘Slovenian Qualifications Framework’ project was lunched and other supporting bodies were set, including the EQF national coordination point, the Interdepartmental Working Group established by the Government involving representatives of key stakeholders, and the SQF expert group. First, the SQF draft proposal was prepared at the end of 2010, and a comprehensive consultation process with stakeholders took place in 2011 and in the beginning of 2012. Then, the SQF’s first referencing to the EQF and its self-certification to the FQ EHEA were prepared, discussed with the stakeholders, presented to the EQF AG in 2013 and adopted in 2014 by the EQF AG. Finally, at the end of 2015, the special SQF Act was formally adopted by the Slovenian government ( CPI, 2014 ; ZSOK, 2015 ).

The SQF is comprehensive framework, which includes qualifications from all subsystems of initial and further education and training. The SQF includes three types of qualifications: (1) formal educational qualifications; (2) vocational qualifications that can also be obtained outside the formal education system and consist of (a) national vocational qualifications (NVQs) and (b) qualifications obtained under continuing vocational training or study programmes for continuing education; and (3) supplementary qualifications, i.e. qualifications that supplement an individual’s competences in a specific professional field and are tied to the labour market’s needs. Following the example of the EQF from 2008, the SQF level descriptors are described in terms of knowledge, skills and competences, although its definitions are adapted to the existing national system ( CPI, 2014 ; ZSOK, 2015 ). The framework, which contains 10 levels, can be categorised as a ‘communication framework’ ( Raffe, 2011, pp. 283–284 ; Raffe, 2013, p. 148 ), as it is mainly based on existing education legislation and established educational practices. Its purpose is to improve the existing qualifications system’s transparency, and the form of learning outcomes is not prescribed uniformly for all qualifications in different educations subsystems by the SQF ( Mikulec and Ermenc, 2016, pp. 6–7 ). However, the SQF is also designed as a tool of reform. This is because, for the first time, it introduced new type of qualifications (i.e. supplementary qualifications)

into an existing qualifications system and gave those qualifications coming from the labour market (i.e. employers) state recognised visibility. Nonetheless, this type of qualifications is not equivalent to the educational or vocational qualifications and there is no horizontal progression between supplementary qualifications and educational or vocational qualifications. Furthermore, the SQF also recognised new vocational qualifications (i.e. qualifications obtained under continuing vocational training programmes or study programmes for continuing education), which support LLL opportunities in higher education and foster the greater integration of the labour market and education in continuing VET.

By making a distinction between the SQF’s communication and reform roles, we can better understand its main objectives. The first two, i.e. (a) ‘to improve transparency, accessibility and quality of qualifications’ and (b) ‘to connect and coordinate Slovenian qualifications subsystems’, are related to the SQF’s communication role, whilst the third one, i.e. (c) ‘to support LLL’ and to integrate the labour market needs with education and support mobility, are related to the SQF’s reform role ( CPI, 2014, p. 30 ). For the purpose of the empirical investigation, we developed the following three categories related to the abovementioned SQF objectives: (1) recognisability and understanding of qualifications, (2) transparency of qualifications and coordination of qualifications subsystems and (3) LLL support.

Materials and Methods

A quantitative research approach was employed in order to contribute to the better understanding of the impact of the EQF-influenced SQF on its education and training system. We examined the achievement of the objectives at a common level, that is, at the level of all stakeholders included in the survey, and also looked into the extent to which the SQF’s objectives were met from the perspectives of those stakeholders who used the SQF at their work compared to those who do not. We also compared the perspectives of those stakeholders who were informed about the SQF’s benefits (or assessed that the benefits were presented to them) with those who were not informed. The following research questions have been designed:

RQ1. What are the key stakeholders’ views regarding the extent to which the SQF objectives have been met?

RQ2. Are there any differences in the stakeholders’ views on the extent to which the SQF objectives have been met depending on their usage of the SQF?

RQ3. Are there any differences in the stakeholders’ views on the extent to which the SQF objectives have been met depending on whether they are familiar with the SQF’s benefits?

Survey sample

In the first phase, all relevant institutions were identified and requested to participate: including ministries responsible for education, higher education, labour market and economy; education institutions (vocational schools and colleagues, secondary general schools and universities); adult education colleges and institutes; students’ organisations and employment service. The institutions were asked to identify those employees who best fit the description of the key stakeholder, so that the survey sample can be the best expert sample for the study. In other words, we were interested in those who were supposed to implement and use the SQF for professional (not personal) reasons, but were who not involved in the establishment or design of the framework. There were 50 key stakeholders included in the survey. Although the sample is of a medium size, it is representative for the studied population. The largest number of the sample included faculty representatives (44%), followed by educational institution representatives (18%) and career advisers (12%). The remaining categories—representatives of student organisations (4,0%), ministries (2,0%), adult education (4,0%) and vocational colleges (4,0%)—were represented to a much lesser extent.

Instrument

A questionnaire was designed to collect the data from the identified stakeholders. It consisted of three sets of questions, which measured the opinions of the stakeholders on (1) the recognisability and understanding of qualifications (5 items), (2) contribution of SQF to the transparency of qualifications and coordination of qualifications subsystems (6 items) and (3) the LLL support of SQF (4 items), respectively. The items were originally measured on a 5-point scale ranging from ‘disagree’ (grade 1) to ‘fully agree’ (grade 5). However, due to the small survey sample size, the grades were merged into a 3-point scale wherein grade 1 indicated disagreement, grade 2 indicated undecided respondents and grade 3 indicated their agreement.

In addition, two additional dichotomous questions, which considered the use of the SQF at work and being informed about the SQF’s benefits, were asked. The respondents were given the opportunity to answer with ‘I do not know’ at each item, in case they did not have enough information to express their opinion.

Procedure

The study was conducted using an online questionnaire administered in February 2019. The collected data were analysed using quantitative analysis methods, such as basic descriptive statistics (frequency distribution). The existence of statistically significant differences regarding the SQF’s benefits between users and non-users of SQF and between informed and non-informed stakeholders were examined with the Likelihood ratio test. The respondents who provided the ‘I do not know’ answer were omitted from the analysis.

Results

Recognisability and understanding of qualifications

Table 1.

Frequency distributions for items measuring recognisability and understanding of qualifications with the results of the likelihood ratio test regarding the use of the SQF and being informed about the SQF’s benefits

	n	Disagree	Undecided	Agree	Likelihood ratio test
					LR	df	p
Familiarity with the SQF	50	12.0%	8.0%	80.0%
Users	30	0.0%	33%	96.7%	14.608	2	0.001
Non-users	18	27.8%	16.7%	55.6%
Informed about the SQF's benefits	17	0.0%	0.0%	100.0%	8.090	2	0.018
Not informed about the SQF's benefits	30	13.3%	13.3%	73.3%
Awareness of the purposes of the SQF	49	16.3%	20.4%	63.3%
Users	30	6.7%	20.0%	73.3%	4.323	2	0.115
Non-users	18	27 8%	22.2%	50.0%
Informed about the SQF's benefits	17	0.0%	11 8%	88.2%	8.562		0.014
Not informed about the SQF's benefits	30	20.0%	26.7%	53.3%		2
Availability of the materials and other SQF-related information needed	46	13.0%	26.1%	60.9%
Users	30	3.3%	20.0%	76.7%	9.370	2	0.009
Non-users	15	26 7%	40 0%	33.3%
Informed about the SQF's benefits	17	0.0%	0 0%	100.0%	23 082		0.000
Not informed about the SQF's benefits	29	20.7%	41.4%	37.9%		2
Familiarity with the EQF	47	12.8%	19.1%	68.1%
Users	30	3.3%	16.7%	80.0%	6.082	2	0.048
Non-users	16	25.0%	25.0%	50.0%
Informed about the SQF's benefits	17	0.0%	11.8%	88.2%	6.832	9	0.033
Not informed about the SQF's benefits	29	172%	24.1%	58.6%
Contribution of the SQF to a better understanding of qualifications	42	4.8%	33.3%	61.9%
Users	27	3.7%	25.9%	70.4%	2.277	2	0.320
Non-users	15	6.7%	46.7%	46.7%
Informed about the SQF's benefits	15	0.0%	0.0%	100.0%	19.322		0.000
Not informed about the SQF's benefits	27	7.4%	51.9%	40.7%		2

Table 1 shows that the recognisability and understanding of qualifications amongst the stakeholders is highest in terms of familiarity with the SQF, which is typical for SQF users (96, 7%) and for those who are informed about the SQF’s benefits (100%). Familiarity with the EQF amongst the stakeholders is lower than familiarity with the SQF, and this finding holds for those who use the SQF at their work and those who are informed about the benefits of the SQF.

About two-thirds of the stakeholders state that they are familiar with the SQF’s purpose, and a slightly smaller proportion state that they have access to the materials and other information on the SQF that they need.

Being informed of the SQF’s benefits has a significant impact on the recognisability and understanding of qualifications amongst the stakeholders. This is because the levels of familiarity with the SQF and EQF, awareness of the SQF’s purpose and the availability of materials and information on the SQF are all statistically significantly higher amongst the stakeholders who are informed about the SQF’s benefits. Furthermore, the stakeholders who are informed about the SQF’s benefits believe that it contributes to better understanding the qualifications compared to the non-informed stakeholders. To some extent, the use of the SQF is also a predictor of achieving the goals of recognisability and understanding of qualifications, as the levels of familiarity with the SQF and EQF and the accessibility of materials and information on the SQF are statistically significantly higher amongst SQF users compared to non-users.

Transparency of qualifications and coordination of the qualifications subsystemsTable 2.

Frequency distributions for items measuring transparency of qualifications and coordination of the qualifications subsystems with the results of the likelihood ratio test regarding the use of the SQF and being informed about the benefits of the SQF

	n	Disagree	Undecided	Agree	Likeiihcxxd ratio test
					IF!	df	p
Contribution of the SQF to a simpler description of qualifications	42	2.4%	40.5%	57.1%
Users ten-users	27 15	0.0% 6.7%	33.3% 53.3%	66.7% 40.0%	4.247	2	0.120
Interned abzut the SQF'S benefits	15	0.0%	6.7%	93.3%	14540		0.001
Not interned abzut the SQF'S benefits	27	3.7%	59.3%	37.0%
Contribution of the SQF to greater transparency of knowledge, skills and competences of each individual qualification	42	7.1%	33.3%	59.5%
Users ten-users	27 15	0.0% 20.0%	259% 46.7%	74.1% 33.3%	10.319	2	0.006
Interned about the SQF'S benefits	15	0.0%	6.7%	93.3%	13246		0.001
Not nfemed about the SQF'S benefits	27	11.1%	48.1%	40.7%

Contribution of the SQF to greater transparency of qualifications in a particular

field of expertise (gjp construction, agriculture, etc.) Users ten-users Interned about the SQF'S benefits Not interned about the SQF'S benefits	40 25 15 14 26	5.0% 0.0% 13.3% 0.0% 7.7%	45.0% 48.0% 40.0% 7.1% 65.4%	50.0%			0.128 0.000
				520% 46.7% 92.9% 26.9%	4.113 18.174	2
Contribution of the SQF to better understanding of the relatione between individual types of qualifications	40	7.5%	42.5%	50.0%
Users	26	3.8%	38.5%	57.7%	2448		0.294
ten-users	14	14.3%	50.0%	35.7%
Interned about the SQF'S benefits	14	0.0%	14.3%	85.7%	12560		0.002
Not interned about the SQF'S benefits	26	11.5%	57.7%	30.8%
Simplifying the understanding of the transition between qualifications acquired through formal and non-formal education and supplementary qualifications of the SQF	38	10.5%	31.6%	57.9%
Users	27	3.7%	259%	70.4%	7.403		0.025
ten-users	11	27.3%	45.5%	27.3%
Interned about the SQF'S benefits	15	0.0%	20.0%	80.0%	7.170		0.028
Not interned about the SQF'S benefits	23	17.4%	39.1%	43.5%
Facilitating comparison of qualifications for the needs of study and work	40	7.5%	17.5%	75.0%
Users	27	0.0%	11.1%	88 9%	10 862		0.004
ten-users	13	23.1%	30.8%	46.2%
Interned about the SQF'S benefits	15	0.0%	0.0%	100.0%	11336		0.003
Not interned about the SQF’S benefits	25	12.0%	28 0%	6.0%

In the context of the SQF’s contribution to the transparency of qualifications and the coordination of the qualifications subsystems (Table 2), stakeholders assessed the facilitation of the comparison of qualifications for the needs of study and work as the highest (75,0%). The results show that there is a significant proportion of stakeholders who remain undecided about the SQF’s contribution to the transparency of qualifications or do not have enough information about it. Despite the small proportion of those who consider the transparency of qualifications to be inadequate based on the following aspects and the high shares of those who acknowledge the SQF’s contribution to the transparency of qualifications in terms of the following: (1) a greater transparency of knowledge, skills and competencies of each qualification (59,5%); (2) a simpler description of qualifications (57,1%); (3) simplified understanding of the transition between qualifications acquired through formal and non-formal education and the SQF’s supplementary qualifications (57,9%), (4) contribution to a better understanding of the relations between individual types of qualifications (50,0%); and (5) contribution to the greater transparency of qualifications in a particular field of expertise (50,0%).

The stakeholders’ opinions regarding the SQF’s contribution to the transparency of qualifications and to the coordination of qualifications subsystems are significantly dependent on whether or not they are informed about the SQF’s benefits. The stakeholders who are informed about the SQF’s benefits recognise its contribution to the transparency of qualifications and to the coordination of the qualifications subsystems in all respects to a statistically significantly greater extent than those who are not informed. The latter are most often undecided regarding the contribution, whilst those who are informed about the SQF’s benefits have a mostly high level of agreement regarding the SQF’s contribution to the transparency of qualifications and to the coordination of the qualifications subsystems.

Furthermore, the usage of the SQF conditions the respondents’ views regarding the SQF’s contribution to the transparency of qualifications and the coordination of qualifications subsystems to some extent. Notably, the SQF’s contributions to the achievement of greater transparency of knowledge, skills and competences of individual qualifications; to the simplification of the understanding of the transition between qualifications acquired through formal and non-formal education and the SQF’s supplementary qualifications; and to a comparison of qualifications for the needs of study and work, are all recognised to a statistically significantly greater extent by SQF users compared to non-users.

Lifelong learning support

Table 3.

Frequency distributions for items measuring lifelong learning support with the results of the likelihood ratio test regarding the use of the SQF and being informed about the benefits of the SQF

	fi	Disagree	Undecided	Agree	Likelihood ratio test
					LR	df	p
Contribution to a better understanding and comparability of Slovenian qualifications abroad	39	12.8%	20.5%	66.7%
Users	27	7.4%	22 2%	70.4%	2.128	2	0.345
Non-users	12	25.0%	16.7%	58.3%
Informed about the SQF's benefits	15	0.0%	0.0%	100.0%	16544		0.000
Not informed about the SQF's benefits	24	20.8%	33.3%	45.8%		z
Contribution of the SQF to a more successful dialogue between the labour market and education and training institutions	36	16.7%	44.4%	38.9%
Users	24	4.2%	458%	50.0%	9.064	2	0.011
Non-users	12	41.7%	41 7%	16.7%
Informed about the SQF's benefits	14	0.0%	42.9%	57.1%	7.822		0.020
Not informed about the SQF's benefits	22	27.3%	45.5%	27.3%		z
Contribution of the SQF to a better match between available market knowledge and skills, and workplace need	35	14.3%	54.3%	31.4%
Users	24	8.3%	54.2%	37.5%	2.714	2	0.257
Non-users	11	27.3%	54.5%	18 2%
Informed about the SQF's benefits	14	0.0%	42.9%	57.1%	10 521		0.005
Not informed about the SQF's benefits	21	23.8%	61.9%	14.3%
Facilitating easier planning of workers'	35	22.9%	42.9%	34.3%
needs and their professional development
Users	23	13.0%	43.5%	43.5%	4.510	2	0.105
Non-users	12	41.7%	41.7%	16.7%
Informed about the SQF's benefits	13	7.7%	15 4%	76.9%	17.558		0.000
Not informed about the SQF's benefits	22	31.8%	59.1%	9.1%		z

The SQF’s LLL support seems to be the weakest area amongst the ones studied (Table 3), as only about one third of the sample acknowledge the SQF’s contributions to a better match between available market knowledge and skills and workplace needs as well as to easier planning of workers’ needs and their professional development.

The SQF’s contribution to a more successful dialogue between the labour market and education and training institutions is acknowledged by more than one third of the stakeholders (38,9 %), with those using the SQF at their work assessing this contribution to a statistically significantly higher extent than nonusers. Furthermore, the stakeholders who are informed about the SQF’s benefits statistically significantly differ in their assessment of the contribution from those who are not informed about such benefits. Two thirds of the stakeholders believe that the SQF contributes to a better understanding and comparability of Slovenian qualifications abroad. Moreover, from all measured aspects, the stakeholders who are informed about the SQF’s benefits recognise its LLL support to a statistically significantly greater extent than those who are not aware of the SQF’s benefits.

Discussion

The EQF policy transfer influenced SQF development through discursive dissemination, i.e. common objectives related to facilitation of LLL, such as the promotion of mobility, transparency and comparability of qualifications; standard setting, i.e. clear timelines and benchmarks related to the use of criteria and procedures for referencing the SQF to the EQF; financial means, i.e. the SQF development and implementation being supported with ESF and Commission grants to the EQF NCP; coordinative functions, i.e. progress made towards common policy aims (the SQF) being monitored through the EQF networks (EQF AG and EQF NCP); and through technical assistance, i.e. three international experts were included in the EQF referencing process. The SQF was built on two waves of reforms undertaken before SQF development: one related to the comprehensive education reform in 1990s and another which made Slovenia part of the Europeanisation process. The SQF’s development started in 2009 and was formally established based on these two waves of educational reforms, which provide the necessary context for better SQF analysis and understanding (cf. Allais, 2017).

As is the case with most EQF-influenced NQFs in Europe (cf. Cedefop, 2018, p. 17 ), the SQF has mostly been designed as a communication framework ( Raffe, 2013 ), which includes some reform elements related to the support for LLL, namely, the mobility and integration of labour market needs with education ( CPI, 2014, p. 29 ).

The results indicate that the SQF is more successful in its communication role compared to the reform role: objectives related to the SQF’s communication role are, from the key stakeholders’ perspectives, realised to a much greater extent compared to those related to the SQF reform role. Hence, objectives related to the reform role, except for mobility, seem to be a much more difficult nut to crack.

The finding that those stakeholders who use the SQF in their work and/or are informed about its benefits assess the framework’s communication role to be much better, compared to those who do not use it or are not informed about its benefits, implies that the communication role is a productive one that may be even improved in time when more people are informed and eventually become SQF users. This is particularly true where recognisability and understanding of the SQF is concerned, but to a lesser extent when transparency and coordination of qualifications subsystems are involved. As for the latter, the high proportion of the undecided users is still rather high, although this improves in those who are informed about the SQF’s benefits.

More specifically, the majority of the stakeholders agree that the SQF contributes to the recognisability and understanding of qualifications; two thirds also agree that it improves the EQF’s recognisability. More than half agree that the SQF improves the transparency of qualifications and coordination of the qualifications subsystems according. This finding is in line with Cedefop’s (2018, p. 16) conclusions on the early impact of the European NQFs. We additionally found that the stakeholders who are informed about the benefits of the SQF, along with those who use the SQF (to slightly lesser extent), recognise the contribution of the SQF to the transparency of qualifications and coordination of the qualifications sub-systems to a statistically significantly greater extent compared to those who do not use it or are not informed about its benefits well enough. This finding implies that the NQFs’ role in the transparency of qualifications and coordination of the qualifications subsystems may improve after more people who use it professionally gain a deeper understanding of the framework’s roles and benefits.

When it comes to the SQF’s support to the LLL, a disparity of results can be noticed. Whilst two thirds of the stakeholders believe that the SQF contributes to the mobility of learners and workers on the one hand, less than 40% of the respondents acknowledge the SQFs contribution to other objectives related to the LLL policy on the other hand. For instance, only 38.9% stakeholders agree that the SQF can contribute to a better integration of labour market needs and education despite the fact that the SQF introduced supplementary qualifications that are tied to the labour market needs into the SQF. Such a finding may be influenced by the sample’s composition sample (the labour market representatives were excluded), yet a recent CPI (2020) report, which included a broader range of respondents, revealed that supplementary qualifications were not successfully implemented, because they were not ‘organically’ integrated into the existing education system and thus functioned as an alien body causing tensions amongst different stakeholders and decision-makers. The SQF’s reform role has not proven to be productive in terms of including new types of qualifications, such as those that did not exist before the framework was introduced, into the SQF. It seems that the disparity can also be explained along the line of the SQF’s communication–reform functions: mobility has been part of the Slovenian education policy ever since Slovenia became a member of the EU in 2004 and is well-embedded in the education practice with full infrastructure in place. The SQF is merely one element in the mobility policy, thereby enhancing but not enabling it. In comparison, the supplementary qualifications are not part of any background policy or pre-existing practices.

Conclusion

The indirect measurement of the impacts of the SQF and the small sample prevented us from drawing very solid conclusions. Therefore, the findings of the study should be seen as an addition to the previous studies investigating the influence of the European education policy on national educational policies, focusing on the development and implementation of NQFs in Europe. The study addresses the tensions and growing gap between the advocates of the EQF and NQFs. Such a gap arises from the IOs and their belief that NQFs and EQFs are a ‘panacea’ that can resolve many educational problems faced by Europe on the one hand, and the critical research community arguing that the EQF and NQFs are unable to fulfil a broader set of objectives and purposes as they claim on the other hand. By examining the case of the Slovenian framework—an example of an EQF-influenced NQF—from the key stakeholders’ perspectives, we tried to find a productive way of approaching the impact measurement problem. A more balanced and larger sample would enable us to reach less tentative conclusion, yet the results persuade us to agree with those who claim that the NQFs indeed cannot function as a panacea. They do, however, seem to have the potential to become ‘the cherry on top of the cake’ of more comprehensive reforms if implemented as a final policy step supporting other relevant policies related to the recognisability, transparency, mobility of qualifications, system coherency and education–labour market cooperation.

Acknowledgements

This work was supported by the Slovenian Research Agency under Grant P5-0174.

Conflict of interests

The authors declare no conflict of interest.