The impact of the COVID-19 pandemic on higher education students’ perceptions of educational applications and platforms

Teachers play many roles in classrooms, including inspiring students, stimulating their interest in the subject, and serving as a source of knowledge and information. Besides, commitment and passion are required, while a combination of teaching paradigms is needed to maintain momentum and transform the classroom into a learning environment full of enthusiasm and interest ( Powell et al., 2012 ).

Teachers increasingly complain that their students do not show interest, do not prepare for lessons, have passive attitudes toward learning activities, and limited awareness of their own learning process ( Buijs and Admiraal, 2013 ). This, and the contemporary competence-oriented higher education, requires a correspondingly strong focus on pedagogy. But there is also a growing recognition that to develop the knowledge and skills needed for the 21^st century, we also need innovations in the field of teaching ( Paniagua and Istance, 2018 ).

As explained by Heacox and Pengal (2009) , the greatest challenge for today’s teacher is to respond to the increasingly diverse learning needs, styles, and social backgrounds of students, which affect their willingness to learn, learning styles, motivation, interests, attitudes to learning, and self-confidence. He, therefore, suggests that differentiation of instruction, which puts students at the center of teaching and learning and also allows learning needs, learning styles, and interests to guide instructional design, can be very helpful, including varying the pace of learning, the level of difficulty and the way instruction is delivered ( Heacox and Pengal, 2009 ).

The traditional lecture is not an effective learning environment for many students, as many do not actively participate during a traditional lecture ( Bajpai, Biberman and Ye, 2019 ). The goal is to keep students active, and approaches to learning are changing. In student-centered learning, the focus is on students’ needs, abilities, interests, and learning styles, and the teacher is only a facilitator of learning. Due

© 2023 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license .

to the development of the World Wide Web and search engines, ‘on-demand’ information has become a reality ( Hirsh, 2018 ). In this context, students need to be active and responsible participants in the learning process, and the teacher plays a key role in the whole process.

E-learning uses various information-communication technology (ICT) tools to make the teaching and learning process more effective ( Bajpai, Biberman and Ye, 2019 ). As a result, classroom lectures, whose main objective is to cover the content of the course, can be expected to become obsolete, as can the willingness of students to attend these classroom lectures, as technology allows them to learn at their own pace and in their own time ( Moreno, 2018 ), expanding the possibilities for individualisation and personalisation of learning ( Urbančič, 2021 ). More and more people are realising the benefits of learning at a pace that suits them best, which is why e-learning is on the rise.

E-learning allows the teacher and the student to be independent in terms of location and time, making it a form of distance learning ( Bregar, Zagmajster and Radovan, 2010 ). Furthermore, the advantage of online education is that many courses are very affordable for the value they offer and are aimed at almost every individual in the world, whereas traditional education involves a limited number of students ( Krastev, 2019 ), which is an additional reason for the rapid rise of educational apps and platforms.

While the concept of e-learning generally refers to the methods of education implemented using ICT, ICT in the context of e-learning represents only part of the technological solutions that allow us to complement or enrich the basic foundations and concepts of traditionally based learning ( Jagodič, 2010 ). Nevertheless, ICT is a potentially powerful tool for expanding educational opportunities, both formal and non-formal ( Adenusi, Adebayo and Oni, 2019 ), and it is the development of ICT that has brought about significant changes in the field of education ( Semerci and Aydın, 2018 ).

However, we recognise that teachers need to have certain digital competencies to be able to use online platforms and applications for communication, teaching, and research. Therefore, national projects to promote the use of ICT by (higher education) teachers are very welcome. Teachers play a central role in integrating ICT into the classroom, so improving their digital competencies and attitudes is crucial for the effective integration of ICT in education ( Semerci and Aydın, 2018 ). In addition, it is important to be aware that today’s students are so-called digital natives, who are expected to have the knowledge and skills that allow them to handle ICT tools ‘naturally’ ( Boh Podgornik et al., 2016 ), so it is not surprising that students often expect this from teachers as well.

The introduction of the use, integration, and dissemination of ICT has ushered in a new era in educational methodologies, radically changing traditional teaching methods and learning patterns in the field and offering modern learning experiences for both teachers and students ( Adenusi, Adebayo and Oni, 2019 ). Therefore, different teaching/learning strategies need to be applied in education. This is because digital technologies are often used to support the extension of university education into the home, social and work environments ( Castañeda and Selwyn, 2018 ).

As a result, ICT has a broader role in teaching and learning activities into which it can be integrated as a tool in the learning process ( Lautenbach, 2014 ). Furthermore, the important role of ICT can also be found in activities such as content creation, student performance monitoring, training, knowledge management and organisation, and formative monitoring ( Margaret et al., 2018 ). Black and Wiliam (2009) , two of the most referenced authors, established a conceptual framework for understanding formative monitoring through a series of publications that can be traced back at least to the mid-1990s. Formative monitoring can be imagined as a “bridge between learning and teaching”. It is based on five key strategies: clarification, participation in setting and understanding learning intentions and success criteria; designing classroom activities that provide evidence of learning; providing feedback; activating students to become teachers to each other; and activating students to self-manage their learning ( Palir Mavrič, 2017 ). The integration of ICT was identified as a good practice, particularly in stimulating and increasing students’ motivation. But the use of technology in learning does not automatically result in engagement in learning activities and improved learning outcomes ( Sinha et al., 2015 ), and for this reason, well-designed computer-supported collaborative learning (CSCL) environments for collaborative learning and problemsolving might intensify learning if used based on a teacher’s instructional support ( Pietarinen, Palonen and Vauras, 2021 ).

Moreover, teachers and students must understand that independent learning using online applications and platforms has a major disadvantage if not carefully addressed and that is a lack of collaboration. Namely, students generally do not recognise the advantages of collaborative learning, and therefore teachers should play an important role in encouraging and influencing student participation and achievement ( Webb et al., 2019 ). The teacher’s role has thus changed to act as a coach and facilitator of students’ thinking while modelling the learning process ( Anderson, 2002 ).

ICT tools are thus playing an increasingly important role in the renewal of educational methods.

This is manifested in the form of digital learning materials instead of printed books, interactive materials, e-communication between teachers and students, etc. ( Berényi and Deutsch, 2018 ). While the pandemic has increased the use of and changed teachers’ attitudes toward digital learning materials, quality teaching in the digital age would require supporting teachers with training to make working with digital materials routine ( Mohar and Kovač, 2021 ). ICT tools used to create and disseminate knowledge including equipment and software solutions have become an indispensable part of modern culture, which is spreading globally through education ( Bajpai, Biberman and Ye, 2019 ). Software that supports the didactic process in higher education includes internet browsers, office suites, communicators, specialised software suitable for classroom use, software that supports group work, and above all e-learning applications and platforms ( Grooms, 2018 ), which are the focus of our research.

As the use of educational applications and online platforms is growing rapidly (e.g., Data Bridge Market Research, 2022 or Dhawal, 2021 ), we decided to explore their use for study purposes. To this end, we analysed the opinions of students from four Informatics and Computer Science degree programmes, two first-degree programmes and two second-degree programmes.

We started the survey in 2019 when we first asked students enrolled in their final year of study on these four study programmes about which apps and online platforms, they use to support their studies. Based on the responses we received, we found that the use of apps and online platforms is much more widespread among students and not limited to those suggested by teachers. In addition, the students presented in their research reports the results of the testing of the apps and online platforms they use to complement their studies.

As today’s teachers (and especially teachers in information technology) and computer science programmes, due to the specificity of these programmes) are increasingly aware of the need to use and even compete with online educational opportunities, it has become clear that they need to change their teaching methods and see their profession as adding value and upgrading the information that students can obtain via the Internet. Our main research objective is therefore to present the findings of a qualitative analysis of students’ views when testing educational applications and online platforms, which we believe could encourage higher education teachers to make greater use of them in their teaching, not as a substitute, but as an additional source of ‘on-demand’ knowledge and motivation for learning. Finally, this paper aims to encourage higher education institutions to reflect on how encouraging the use of educational apps and online platforms can enable students to perform even better in exams, while also spreading awareness that they can deepen their knowledge in this way too.

Materials and Methods

Statistics (e.g., Wise, 2022 ) show that e-learning has grown exponentially in recent years. Studies also show (e.g., Suwal and Singh, 2018 ) that the use of e-learning applications and platforms is well accepted by students as it gives them positive experiences, and consequently, there is a growing need for more integration of e-learning applications and platforms into existing learning activities and practices. In the flood of different types of educational apps and online platforms, it is difficult for a teacher to decide which app or platform is appropriate to use and recommend to students. In addition, more and more new apps and online platforms are emerging, making it difficult to keep up with these advances and the increasing range. In this context, we have identified the need for a systematic review of the educational apps and platforms that students individually choose to use, as well as the way they are perceived. To do so, we have conducted a survey in which students themselves, based on their study interests, chose an app or online platform to test. They presented their views gained during the testing in research reports, which we then used to achieve the aim of our research: to show that teachers need to change their role, as the internet offers instant information on almost any topic and as educational apps and platforms are gaining in importance. Accordingly, teachers also need to change their teaching methods to offer students all they can gain from e-learning.

Based on the content of the students’ research reports, we identified the codes and the most frequently written words to give their perspective on the impact and usefulness of some educational apps and platforms in their fields of study, as the research also aims to encourage teachers to reflect on how the integration of apps and platforms in their courses can bring added value. In addition, the Covid-19 coronavirus pandemic has further stimulated the development of online educational resources. Based on this research objective, the following research questions (RQs) were set:

RQ1: What has proved to be most important in the testing of educational apps and platforms by students?

For this question, we were looking for codes that could be extracted from a close reading of the assignments that students had prepared based on testing educational apps and platforms. We assumed that students would put the user experience at the center, and quality would be the decisive factor (as in Rojko, 2020 ). On the other hand, although some apps and platforms provide certificates and recognition of the knowledge acquired, we assumed that the absence of formal recognition would demotivate students to learn online on their own initiative.

RQ2: Which words stand out in the research reports of students who have tested educational apps and platforms?

We asked students to use different criteria to test different applications and platforms for their research reports, mainly qualitative but sometimes also quantitative. We anticipated that the words quality, user experience, quantity, price and certificate and their synonyms would appear most frequently. We were also interested in whether positive or negative experiences predominated.

RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view educational apps and platforms?

We expected that some new dominant words would appear in the more recent research reports (from the second half of the academic year 2021/22) compared to the reports from the period before the start of the epidemic (from the first half of the academic year 2019/20). Namely, we observed that the Covid-19 pandemic was reflected in an increased motivation of students to search for additional online education resources and in their increased willingness to try them out, as distance education had been the only option for some time.

The survey was launched in 2019 when we asked students enrolled in the final year of four Informatics and Computer Science degree programmes from Slovenia, which educational apps and platforms they use to support their studies. In addition, the majority of these students also produced a research seminar paper based on the results of their testing.

Students chose and tested the following educational apps and/or platforms (in alphabetical order): ArtofProblemSolving, Astra.si, Bussuu, CK-12, Code With Mosh, CodeAcademy, Codewars, Coursera, Cybrary, Drops, Duolingo, edX, freeCodeCamp, FreeVideoLectures, FunEasyLearn, Khan-academy, KodeKloud, Laracast, LinkedIn learning, Mathway, Memrise, Micro:bit, Mimo, MojeZnanje.si, Photomath, Pluralsight, Programming Hub, Qlango, Scratch, Skillshare, SoloLearn, SQLZOO, Stack Overflow, Symbolab, Treehouse, Tutorialspoint, Udemy, Udacity, W3Schools, Wolfram Alpha, Yoast and YouTube. Note that some students chose the same apps and platforms, and Udemy was the most repeated platform, 18 times.

Some of the educational apps and platforms chosen specialise in one area, others cover several different areas, and students were instructed to focus on one specific subject area for any comparisons. As can be seen from the list of names, some of the listed platforms cover different areas of knowledge, although all students focused on computer science, while the listed apps focus on computer science (most frequently tested), mathematics (medium frequency), and foreign language learning (least frequently tested). The majority of the apps and platforms analysed were international, but two Slovenian platforms, Astra.si and MojeZnanje.si, were also included.

Among the 140 students enrolled in the final year of their first (professional study programmes) and second (masters study programmes) cycle studies in the academic years 2019/20, 2020/21 and 2021/22, 113 (81%) submitted their research seminar paper. Among these, we further excluded those reports in which students tested other applications that they had used during their studies but were not relevant to our research - e.g., for storing data and literature, such as Dropbox, Evernote, Google Drive, OneNote and Zotero. Thus, the final 90 research reports (50 reports by first-cycle students and 40 reports by second-cycle students) were considered, out of these 29 from the academic year 2019/20 (first half of the year (in continuation referred to as the pre-pandemic period) 16 reports), 36 from 2020/21, and 25 from 2021/22 (second half of the year (in continuation referred to as post-pandemic period) 10 reports). We considered the content of research reports as transcripts, which were then subjected to qualitative analysis to answer our research questions. We resorted to manual coding and used the Atlas.ti qualitative analysis tool.

Manual coding (using the open axial coding methodology) required close reading and careful consideration of the students’ opinions and observations, taking special care to avoid concepts we had learned from studying the literature and our views and expectations. The attribution of concepts was done both by direct conceptualisation and synonymy. The selection was made based on the conceptual framework and within it based on recurring terms in the students’ assignments. Individual terms were further evaluated and linked to semantically equivalent terms, which were grouped into key categories (codes) based on their meaning.

Atlas.ti was used to search for the most frequently repeated words in the assignments. Although this tool also allows the preparation of word clouds based on Slovene texts, it only formats them appropriately in some of the world’s leading languages. As the tool is not able to combine Slovenian words in different conjugations, does not correctly identify word types, etc., we first translated the 90 analysed seminar reports, with an average size of one author field (The author’s pole consists of 30 000 characters (letters, numbers, punctuation, spaces) or approximately 16 typed pages.), into English. We then extracted nouns, verbs, adjectives, and adverbs, followed by manual removal of the remaining inappropriate words. Finally, for the most common words, we first considered separately the terms for the full spectrum of experience from positive to negative, and then all the remaining most commonly used words.

In addition, we also analysed the research reports from the first half of the 2019/20 academic year (considered as pre-pandemic period reports) and the research reports from the second half of the 2021/22 academic year (considered as post-pandemic period reports) separately and used the results to compare the two periods, which led us to detect the changes in the testing experience as well as in students’ attitudes towards educational apps and platforms. To do so, we applied chi-squared test at significance level of 0.05 to find potential association between pre- and post-pandemic periods regarding the testing experience and students’ attitudes. In the case of a statistically significant chi-squared test, the association was confirmed and was then tested for strength using the Cramer’s V contingency coefficient at 1 degree of freedom. We used the value of the latter to determine whether the association was weak (Cramer’s V value between 0.1 and 0.3), moderate (Cramer’s V value between 0.3 and 0.5) or strong (Cramer’s V value between 0.5 and 1.0). To visually represent the observed associations, we used mosaic plots in which two-dimensional frequency table is displayed using rectangles. The rectangular regions in the mosaic plot are proportional to the cell frequencies they represent, where shadings indicate the residuals between observed and expected frequencies. Positive value of a residual is represented by the blue color, and negative value of a residual is represented by the red color, whereas the shade of color indicates the site of residual: the larger the absolute value of residual, the darker the color, and vice versa, the smaller the absolute value of residual, the lighter the color. This is also presented in the legend added to the mosaic plot ( Kabacoff, 2015 ).

Results

We first present the results of the manual coding (RQ1) and then the results of the analysis using the Atlas.ti tool, which was used to produce linear word clouds for all analysed research reports together (RQ2) and separately based on research reports from before the Covid-19 epidemic and two and a half years later (RQ3).

Coding results

Based on a close reading of a selection of research reports (N=90), we identified 10 key content categories (codes) in the texts: content, personal preference, reason, user experience, price, quantity, basic, development, quality, advanced and endorsement.

Figure 1. Comparison of paradigm models with Rojko (2020)

Figure 1 shows that our analysis produced slightly different results than Rojko (2020) as interaction, motivation, recommendation, and traditional were replaced by reason, development, and content, in Figure 1 right, according to our findings. Similarly, Rojko (2020) found that quality dominates over user experience and quantity, whereas our findings showed a different picture – as presented in the Figure 2.

Figure 2. Representation of each dominant word by code

Figure 2 shows the representation of each of the dominant words by codes (We did not add the code area to the selection (dominant words: coding, programming, foreign, language(s), technology) because it is not relevant, as these are the fields of study of the students whose research seminar papers are analysed.), which were written at least 200 times in all research reports combined. It turns out that content stands out the most (words: content, tutorial, example, material, new, video, game, help, information, data), followed by personal preferences (words: option, year, years, time, different, decision, way, possibility, relevant, important, choose, criterion, need, work). For content, we would highlight the words video, game, and help as they best convey the students’ observations, while for personal preferences we would highlight time, different, and way as they best convey the students’ thinking.

Dominant words

The dominant words were extracted using Atlas.ti tool, but despite the translation of the analysed research reports into English, a lot of manual refinement was necessary (see Appendix 1 for a full list of removed words). The words that appear in Figure 3 appear at least 200 times in the analysed reports. The most frequently occurring word is language (N=1476; also, languages in 16th place (N=657)), which in turn refers to the domains of the applications and platforms tested (computer languages, foreign languages). This is also the case for the third-ranked word programming (N=1342), the twelfth-ranked word coding, etc. The second-ranked word is knowledge (N=1419), which we have classified under the code reason. The results based on the most frequent individual words thus show that knowledge is the most important for students, research is needed, followed by content, time is a constraint, and free, variety, and quality are also among the most important.

It should also be noted that it is necessary to group certain words that appear separately in the figure below, e.g. research (N=1318) and search (N=235), information (N=595) and data (N=508), find in the present and past tense (N=559 and N=371), video (N=459) and videos (N=270), year (N=238) and years (N=204), play (N=358) and games (N=205), etc.

language knowledge programming research content time free different Detter new good code education foreign experience anguages way information find help mobile data video many case level criteria available need work wel much skills problem important results found quality game access example basic suitable choose process step possible development decision start training certificate easy comparison model interface useful videos paid result exercises problems rating price like technology features lot years search material advantages similar individual option description best specific average simple beginners field solution design games evaluation year difficult understand chosen designed possibility

Figure 3. Dominant words appearing at least 200 times (all assignments).

The sentiment analysis provided by Atlas.ti, without manual cleaning of the results, shows that negative sentiment prevails (2305 in paragraphs and 4131 in sentences) over positive sentiment (1756 in paragraphs and 2593 in sentences). But a quick look at the results shows that a lot of manual cleaning would be needed, as the sentiments are very often improperly attributed. Therefore, to identify the experience from positive to negative, we also focused on the most frequently written words. This revealed that the positive experience of students’ testing educational apps and platforms is the predominant one, and Figure 4 shows the words that were used most frequently to describe the experience. These words are also included in Figure 3.

Figure 4. Dominant experience words from positive to negative, occurring at least 200 times (all assignments)

As shown in the figure above, 81% (N=2,781) have a positive sentiment. We would add that the word mediocre also appeared 215 times among the most frequent words, but we did not include it in Figure 4. If we had evaluated it as a neutral sentiment, the positive sentiment would have dropped to 76%. The limitation is that the included words are subjectively classified as sentiment and that only words that appear at least 200 times in all research reports are used, but the result is clear. Again, in personal conversations with each student, we perceived an overwhelmingly positive attitude, although occasionally they encountered some problems when taking the test.

Comparison

Following the findings presented above we also carried out a comparative analysis between the periods before and after Covid-19 pandemic. In this context, we have referred to the research reports from the first half of the 2019/20 academic year as pre-pandemic period research reports, and the research reports from the second half of the academic year 2021/22 as post-pandemic period research reports.

The analysis of the pre-pandemic period research reports considered 16 reports, and the analysis of the post-pandemic period research reports considered 10 reports. They both involved a lot of manual cleaning and Figure 5 shows the result – words that appear at least 50 times. Considering the prepandemic period reports, if we remove the words related to the fields of study (programming, coding, and language), we see that the most important are content, reason, and personal preferences, and the top words within the codes are content (N=352), followed by knowledge (N=275) and research (N=166). Also standing out are way, different, video, time, new, experience, find, and information. Considering the postpandemic period research, we notice that the words relating to the fields of study do not appear first in the top positions (in fact, only language appears in the list, in the 15th position).

5 contentprogramming

(a)

|_e knowledge research code way different video

-| & :ime new language education experience find good information

§.    help skills number case level need better game available free quality

2? development process work languages problem training want example important results "      access data basic result found mathematical suitable many solution videos mobile maths solving description model problems several solve well possible much community search

(b)

contentgood experience knowledge better rating research criteria average education information data different decision language free find easy case model price time search interface important many want well helpful mobile quality suitable .vay video results step useful beginners much covered great level training

Figure 5. Dominant words appearing at least 50 times: (a) research reports from the first half of the academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half of the academic year 2021/22 (post-pandemic period reports)

However, in the post-pandemic period reports content is still the most important for students, while personal preferences and reason changed their position, as personal preferences came as second most important. Namely, the top words within the codes are content (N=311), followed by experience (N=177) and knowledge (N=172). Rating, criteria, education, information, data, different, and decision also stand out. This suggests that students became more focused on their own preferences when choosing educational applications and platforms, likely because they became more acknowledged with the (enlarged) offering compared to pre-pandemic.

The comparison was applied to the sentiment analysis as well. Figure 6 shows the words that were used most frequently to describe the experience in pre- and post-pandemic period reports. We acknowledge that od the sentiment in pre-pandemic period reports, good comes first, followed by better, then problem/issues. Positive sentiment dominates with 64%. In post-pandemic period reports, however, sentiment has come to the front, with the words good and better appearing in the top five.

pre-pandemic period reports                     post-pandemic period reports

Figure 6. Dominant experience words from positive to negative: (a) research reports from the first half of the academic year 2019/20 (pre-pandemic period reports), and (b) research reports from the second half of the academic year 2021/22 (post-pandemic period reports)

Thus, in post-pandemic period reports positive sentiment is 100%, as the words problem or problems do not appear, which is one of the most important differences compared to the results of the students’ test results two and a half years earlier. This is per our belief due to the changed attitude of students towards this type of learning, but also to the rapid development of educational applications and platforms, which has been accelerated by the pandemic. The word mediocre appears, which, if taken into account, could reduce satisfaction with testing by 33%.

In order to support the findings presented above, we also performed chi-square tests at significance level 0.05 where the strength of the association was determined by the value of the Cramer’s V contingency coefficients. Figure 7 represents the mosaic plots for a) dominant words, and b) dominant experience, in both cases comparing the pre- and post-periods periods.

Figure 7. Comparison considering (a) dominant words, and (b) dominant experience in the pre- and postpandemic period reports

To compare the pre- and post-pandemic period reports regarding the dominant words, we have considered only those words that appear among the dominant ones in both periods. In this case we observe significant (p < 0.05) but weak (Cramer’s V = 0.184) association between pre- and post-pandemic periods. The discrepancies between the two compared periods occur in the case of the words different, experience, and knowledge, where in the post-pandemic period experience has significantly gained in importance, while the importance of different and knowledge has declined significantly as compared to the pre-pandemic period.

Considering sentiment analysis, we observe strong (Cramer’s V = 0.482) and significant (p < 0.05) association between dominant experience in pre- and post-pandemic period reports, which is consistent with the fact, that positive sentiment among post-pandemic research reports is 100%, as the words problem or problems do not appear. This finding can most likely be attributed to the rapid development of educational applications and platforms, and the accelerated spread of their use as a consequence of the pandemic.

Discussions

The methodology of extracting the main findings from the students’ research reports, where we paid attention to the students’ opinions and observations without combining them with our views and expectations, proved to be a difficult task but helped us to provide neutral answers to our research questions. The content analysis of the students’ reports was carried out using manual coding and the qualitative analysis tool Atlas.ti provided a detailed insight into the students’ opinions. Below we summarise the main findings that answered our research questions:

RQ1: What has proved to be the most important in the testing of educational apps and platforms by students?

The most important category was content, where we would point out the following most frequently written words: video, game (including tutorial, example), and help. This is followed by the category of personal preference, as students supplement their education with educational apps and platforms according to their possibilities, needs, and time, and variety and method are also important to them. This is because nowadays, the diversity of learning needs, styles, and social backgrounds of students are increasingly coming to the fore in education ( Heacox and Pengal, 2019 ), which the second-ranked personal preference code confirms. Reason, user experience, price, and other categories follow.

However, we expected that, although some apps and platforms provide certificates of knowledge, the absence of official recognition might demotivate students to learn online on their own initiative. But we found that formal education is not the only criterion for some companies that also seek recruits via e-learning platforms and applications (in the cases studied, these are so far only foreign companies), and the qualitative analysis also resulted in the category of certificate appearing as a certificate in the word cloud (N=303) being ranked last.

RQ2: Which words stand out in the research reports of students who have tested educational apps and platforms?

We expected that the words quality, user experience, quantity, price, and certificate, and their synonyms (as in Rojko, 2020 ) would appear most often. But it turned out quite differently. If we exclude the most frequent words related to fields of study, the word knowledge ranks highest, which we have placed under the category of reason, followed by research, content, time, free, variety, and quality. The word certificate is ranked only 52^nd.

It also showed that students’ experience of testing educational apps and platforms is predominantly positive (81%), with the words better, good, useful, like, advantage and best being the most frequent, while negative sentiment is characterised by the word problem or problems.

RQ3: Has the impact of the Covid-19 coronavirus pandemic changed the way students view educational apps and platforms?

While the analysis of the research reports from the first half of the 2019/20 academic year shows that the most important is content, followed by knowledge, research, method and different, the reports from the second half of the 2021/22 academic year the word content also ranks best, followed by good, experience, knowledge and better. Thus, in the post-Covid-19 reports, sentiment has come to the front, with 100% of these research reports expressing positive sentiment, as compared to 64% positive sentiment in the reports from two and a half years ago. We also observe (considering the ranking of words by code) that students now prioritise user experience, reason, and quality over content and personal preferences compared to the pre-pandemic period but are still aware of the need for exploration. Namely, the development of the World Wide Web and search engines has made information ‘on demand’ a reality ( Hirsh, 2018 ), and technology allows them to learn at their own pace and time ( Moreno, 2018 ).

Conclusions

Our results show that the development of ICT and the related rise of educational applications and platforms is having a significant impact on teaching and learning. This requires teachers to take these applications and platforms into account and to use new teaching methods, the first prerequisite being a certain level of digital competence. However, we have found that this is in fact a huge burden for the majority of teachers, as it requires additional and continuous research and learning. It is also becoming increasingly apparent that traditional education must compete with e-learning opportunities, or the profession may gradually become meaningless for many teachers. It has also become clear that teachers, therefore, need to change their mindset and see their profession as adding value and building on what students can learn online, while at the same time becoming facilitators of such learning.

Although most courses do not yet foresee the use of educational apps and platforms, when teachers use some (at least occasionally) during their teaching, it can further motivate students to participate and pay attention. In addition, teachers can recommend certain apps and online platforms for students to use for self-learning to deepen and consolidate their knowledge. Our research showed that students already know and use many of the apps and platforms, but it turned out that they also appeared to be very interested in exploring and learning about others that exist in their field of interest, as they have certain advantages, disadvantages, and specificities, and they were always very interested to follow the presentations of their classmates’ testing results.

Most students do not actively participate during a traditional lecture ( Bajpai, Biberman and Ye, 2019 ), but educational apps require exactly that. Nevertheless, we would like to stress that we do not believe that the integration of apps and online platforms in faculty courses is always necessary or that it can always improve teaching and learning. However, in the spirit of the times, where students are used to being constantly connected to the World Wide Web and using connected devices at the same time for learning, searching for information, communicating, etc., we have decided to present the results of our survey to give an insight into students’ opinions and testing results. In addition, this encouraged students to think about the possibilities and options for lifelong learning that will be required of them, especially intensively in the future career field of the students whose research reports were the source of the data of our empirical research. Therefore, this assignment was proposed to final-year students in the field of computer science and informatics.

The limitations of our research are clear; we only present tests of a few online platforms and applications, but the list can never be definitive as it is subject to constant updates. Moreover, depending on the fields of study, we only present the findings of a limited number of students, all from the four Informatics and Computer Science degree programmes from one faculty from Slovenia. Finally, although we do not assess the performance, usability, user experience, etc. of applications and web platforms, we present the main findings of students when testing them to gain insight into their thinking, our assignment of codes is subjective, as is the manual cleaning of the analysis results by Atlas.ti.

Despite these limitations, the scientific contribution of our paper stems from the fact that other authors have not used the same methodology to shed light on the topic of the paper. This topic is also extremely interesting in the so-called post-coronal period, as it shows how quick the change was, due to the coronal period and thus has important practical implications. Moreover, the empirical results of the qualitative analysis of students’ opinions are particularly valuable, as their perspective is often neglected. At the same time, we also conducted a personal interview with each student, which allowed us to code and analyse the word cloud more appropriately.

Online education is certainly an area that will continue to grow rapidly in the future. Indeed, with new technologies such as machine learning and 5G, we can expect further rapid growth. But here is the problem – teachers and students do not know what is out there already, which limits them. Therefore, lifelong learning also applies to teachers, who should also spend their working time exploring these possibilities so that they know how to use them and recommend them to students as an additional resource, and their role should focus even more on a mentoring style of teaching. However, this will only be effective if students are committed to this way of learning, which will require them to be even more self-controlled and self-organised.

During the Covid-19 coronavirus pandemic teachers and students experienced a situation where distance teaching and learning became mandatory, requiring them to change their methods significantly, which further increased the use of educational applications and platforms, not to mention the various online communication tools. As Kodelja (2020) states, it is difficult to say how successful distance education has been as a result of the new coronavirus, but it seems that not all the possibilities offered by ICT have been exploited, due to both objective and subjective circumstances. But due to this pandemic, we cannot go back to the learning and teaching methods we used before. We need to take a further step based on the experiences we have gained and exploit all the possibilities offered by ICT, thoughtfully and adapted for different groups of students.

Acknowledgements

We thank to all students who made their research reports after testing the educational applications and platforms.

Conflict of interests

The authors declare no conflict of interest.

Author Contributions

Conceptualization, K.R. and N.E.; methodology, K.R., and N.E.; formal analysis, K.R. and N.E.; investigation, K.R.; validation, K.R. and N.E.; visualization, K.R. and N.E; writing—original draft preparation, K.R. and N.E.; writing—review and editing, K.R. and N.E. All authors have read and agreed to the published version of the manuscript.