Problem Solving and Decision-Making Skills for ESD: A Bibliometric Analysis
Автор: Mujib Ubaidillah, Putut Marwoto, Wiyanto Wiyanto, Bambang Subali
Журнал: International Journal of Cognitive Research in Science, Engineering and Education @ijcrsee
Рубрика: Original research
Статья в выпуске: 3 vol.11, 2023 года.
Бесплатный доступ
Problem-solving and decision-making skills are essential for individuals across various fields. These skills emphasize the importance of preparing a generation capable of solving problems and making informed decisions. Therefore, this study aimed to learn the publication trends related to problem-solving and decision-making skills for ESD (Education for Sustainable Development) from 2013 to 2022 through Bibliometric analysis. In line with the analysis, a VOSviewer software was used to graphically analyze the obtained bibliographic data. A total of 1519 documents were also analytically acquired from the Scopus database. The results showed a fluctuating trend in the number of publications, with the Journal of Chemical Education and Social Sciences being the highest contributor and the most prevalent field of study at 147 and 689 documents, respectively. The United States was also ranked first in the documents emphasizing problem-solving and decision-making skills, at 512 documents. Moreover, the University of Toronto was the most prolific affiliation, contributing the most publications at 17 documents. The representatives from Indonesia were also grouped into two institutions in the global top twenty affiliates, namely (1) the Indonesian University of Education and (2) the State University of Malang. In line with the results, 159 study experts from Indonesia contributed to the analyzed theme, as the top author originated from the United States having 7 documents. The top document excerpts were also published 240 times in the journal Expert Systems with Applications. The trend of the study visualization subsequently produced 9 clusters, problem-solving and decision-making skills, human, psychology, clinical competencies, education, curriculum, support systems, creativity, and content analysis. These results were helpful to relevant experts, regarding the analytical trend in problem-solving and decision-making skills, recommending directions for future analyses.
Bibliometric analysis, decision-making skills, ESD, problem-solving, VOSviewer
Короткий адрес: https://sciup.org/170200018
IDR: 170200018 | DOI: 10.23947/2334-8496-2023-11-3-401-415
Текст научной статьи Problem Solving and Decision-Making Skills for ESD: A Bibliometric Analysis
Education for Sustainable Development (ESD) is a new paradigm in the educational field, that provides opportunities to the younger generation regarding contribution to sustainable development ( Tripon, 2014 ). This paradigm was believed to transform the learning mindsets of students, enabling a plan for a better life in the future ( Jauhariyah et al., 2021 ). The ESD curriculum implementation also fosters student’s competencies, skills, empathy, responsibility, problem-solving, democratic decision-making, hope, and respect for humanity as well as nature ( Laurie et al., 2016 ). Furthermore, ESD stimulates students to question, analyze, think critically, and make informed decisions ( UNESCO, 2012 ). This paradigm employs an active learning model, which emphasizes academic participation and collaboration.
Based on the aforementioned descriptions, problem-solving and decision-making skills are the methodological competencies required for future success ( Erol et al., 2016 ). This indicated that collaborative decision-making skills were needed in the industrial era 4.0 ( Torres, Pimentel and Matias, 2023 ). The methodological competencies were also selected as essential and relevant characteristics for physics careers ( Nasri, Nasri and Talib, 2020 ), with decision-making skills closely related to the abilities of critical thinking ( Molokhina, Pishchik and Fomin, 2021 ) and communication ( Tsai et al., 2020 ). Therefore, schools need to promote decision-making skills through classroom learning. Collaboration

© 2023 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license .
among students was also expected to become increasingly important in an environment of complex and uncertain decision-making. This is because the interaction among various stakeholders and knowledge exchange can enhance the processes of making appropriate decisions ( Cai et al., 2023 ). According to a study in various fields, the outcomes obtained were significantly related to decision-making skills in learning mathematics ( Pertl, Zamarian and Delazer, 2017 ), physics ( Silvianty, Suhandi and Setiawan, 2019 ), chemistry ( Gao, 2015 ; Lieber et al., 2022 ; Wilson and Wilson, 2017 ), medicine ( Delavari et al., 2022 ), computers ( Carlsson, 2019 ; Chen and Utama, 2022 ; Méndez et al., 2020 ), engineering ( Fruchter, Katz and Grey, 2018 ), business ( De Villiers, Hankin and Woodside, 2016 ), and nursing ( Chen et al., 2021 ; Macduff, Stephen and Taylo, 2016 ; Saab et al., 2021 ). Kwok and Hodgson (2017) also stated that decisionmaking model consisted of six steps, namely identifying problems, generating, evaluating, and selecting alternatives, as well as implementing and assessing decisions.
Several study trends in decision-making and problem-solving skills have subsequently been conducted through bibliometrics. This is because bibliometric studies are significantly related to decisionmaking processes in various domains, such as engineering ( Cai et al., 2023 ), universities ( Vílchez-Román, Sanguinetti and Mauricio-Salas, 2021 ), health ( Holden, Rosenberg and Barker, 2005 ), linguistic probability ( Liao, Mi and Xu, 2020 ), business ( Costa et al., 2017 ; Pacheco-Velázquez et al., 2023 ), laboratory safety ( Abdullah and Abd Aziz, 2021 ), and artificial intelligence ( Di Vaio, Hassan and Alavoine, 2022 ). These trends are also relevantly associated with problem-solving in the fields of teaching physics ( Masitoh et al., 2021 ) and mathematics education ( Chen et al., 2021 ; Suseelan, Chew and Chin, 2022 ). A bibliometric study approach is also commonly implemented to emphasize problem-solving and decision-making skills as variables yet to be concurrently investigated by previous experts. This explains that both skills are necessary in sustainable development education across various fields of knowledge. Therefore, this study aimed to identify the trends in publication growth, countries, subject areas, journals, productive authors, university affiliations, highly cited articles, and future experimental opportunities, regarding the concepts of problem-solving and decision-making skills.
The main contribution of the analysis is to provide comprehensive information concerning the trends in publication growth, subject areas, top countries, university affiliations, journals, highly cited articles, productive authors, contributions of Indonesian experts, and future experimental opportunities in the field of problem-solving and decision-making skills. In this case, article authors and journal editors were essentially expected to possess awareness of the disciplinary landscape and a thorough understanding of the analyzed fields, enabling the identification of future experimental opportunities. Policymakers are also capable of recognizing leading countries in the analysis of problem-solving and decision-making skills, which were implemented in the instructional design of several dimensional educators. Therefore, this study aimed to acquire knowledge about problem-solving and decision-making for ESD, through the review and evaluation of the research questions:
-
1. How has the trend in annual publications, document sources, subject areas, and countries for problem-solving and decision-making skills evolved from 2013 to 2022?
-
2. How has the distribution of problem-solving and decision-making skills publications evolved based on university affiliation?
-
3. Who is the most productive writer in the analysis of global problem-solving and decision-making skills?
-
4. How has the contribution of Indonesian experts evolved in analyzing problem-solving and decision-making skills?
-
5. Which paper is most cited and related to problem-solving and decision-making skills?
-
6. What are the future study opportunities associated with problem-solving and decision-making skills?
Materials and Methods
A bibliometric analysis approach was employed to draw conclusions based on a collection of bibliographic data, exhibiting the structure, social, and author networks, as well as present analytical interests in a specific experimental area (Ha et al., 2020; Karakus, Ersozlu and Clark, 2019; Suprapto et al., 2021; Zupic and Čater, 2015). The approach also included the statistical analysis of published articles and their citations, to measure relevant impacts (Maditati et al., 2018; Saregar et al., 2022) and analyze gaps or emerging topics (Gao et al., 2021). In addition, bibliographical analysis implemented the relevant information obtained from online databases, enabling scientific study and a global perspective on related interest areas (Secinaro et al., 2020). Figure 1 shows the implementation of the five steps emphasizing bibliometric method.
Study Design
* |
Data Collection |
* |
Data Analysis |
* |
Data Visualization |
* |
Interpretation |
-
Figure 1. Bibliometric methodology
Study design
The study design was executed by formulating research questions, as well as selecting keywords and databases. This indicated the existence of the six questions investigating the following features: i) annual publication trends, document sources, subject areas, and countries,; ii) affiliations,; iii) most productive authors,; iv) contributions of Indonesian experts,; v) highly cited papers,; and vi) future experimental opportunities related to problem-solving and decision-making skills. The design also proved that the implemented search keywords were “Problem-solving decision-making skills”, with the Scopus database selected as the bibliometric source.
Data collection
A title search strategy (problem AND solving AND decision AND making AND skills) AND PUBYEAR > 2012 AND PUBYEAR < 2023 were used by the data collection process from the Scopus database. This indicated that 1519 documents for 2013-2022 were acquired from the database, including journals, conference proceedings, review articles, paper chapters, and books. The documents were also downloaded in Comma-Separated Values (CSV) and Research Information System (RIS) formats, to obtain article titles, authors, references, and keywords.
Data analysis
Data analysis was initiated by exporting CSV and RIS information from the Scopus database to Microsoft Excel and VOSviewer. This analysis was conducted to analyze various trends, including publication output characteristics, document sources, country and institutional affiliation distribution, subject category dissemination, top authors and citations, as well as publicized tendencies spanning 2013 to 2022.
Data visualization
The visualization of data was carried out using VOSviewer software ( Van Eck and Waltman, 2010 , 2012 ) and Microsoft Excel. By implementing VOSviewer, processed RIS metadata was mapped based on network, overlay, and density visualizations. Meanwhile, the visualization, emphasizing Microsoft Excel, was used to describe the study trends presented as tables and diagrams.
Interpretation
Data visualized by VOSviewer were analyzed and interpreted regarding the number of clusters formed, old and recent studies, and density visualization, which specifically exhibited saturation and future experimental opportunities.
Results
Annual publication trends between 2013-2022
The trend of publication productivity over ten years (2013-2022) was based on the Scopus database, emphasizing relevant fluctuations. In this case, the analytical trends of problem-solving and decisionmaking skills in various disciplines had a 10-years fluctuation. This was evidenced by the document elevations in 2019, 2020, 2021, and 2022 at 196, 172, 197, and 201 publications, respectively. Figure 2 presents the annual publication trends.

Figure 2. Annual publication trends for the period 2013-2022
Main Source Document
The article documents were sourced from journals, proceedings, paper chapters, reviews, and books. Most of these sources focused on problem-solving and decision-making skills from various fields, including education, computer science, engineering, medicine, nursing, public health, psychology, etc. Table 1 shows the twenty most prolific sources for publishing articles related to problem-solving and decision-making skills.
Table 1
Distribution of documents by related sources from 2013-2022
„ SJR index Number of ource(Scimagojr2022) document
Journal of Chemical Education 0.56 (Q2) 147
ASEE Annual Conference and Exposition Conference Proceedings |
O(-) |
40 |
Journal of Physics Conference Series |
0.18 (-) |
18 |
Advances In Intelligent Systems and Computing |
o(-) |
16 |
Nurse Education Today |
0.95 (Q1) |
10 |
BMC Medical Education |
0.91 (Q1) |
10 |
Pios One |
0.89(Q1) |
9 |
Proceedings of the European Conference on Games-Based Learning |
O(-) |
8 |
International Journal of Environmental Research and Public Health |
0.83 (Q2) |
8 |
Sustainability Switzerland |
0.37 (Q2) |
7 |
AIP Conference Proceedings |
0.16(-) |
7 |
International Journal of Engineering Education |
0.26 (Q2) |
6 |
Frontiers In Psychology |
0.89 (Q2) |
6 |
ACM International Conference Proceeding Series |
0.21 (-) |
6 |
Journal of Vocational Rehabilitation |
0.38 (Q2) |
5 |
Journal of Veterinary Medical Education |
0.45 (Q3) |
5 |
Journal of General Internal Medicine |
1.81 (Q1) |
5 |
Journal of Engineering Education Transformations |
0.21 (Q3) |
5 |
Journal of Dental Education |
0.56 (Q2) |
5 |
Thinking Skills and Creativity 1.15(Q1) 4
Documents based on Subject Area
A maximum of 689 problem-solving and decision-making skills publications were obtained for the 2013-2022 documents emphasizing the subject areas within the field of social sciences. This indicated that “Medicine”, “Engineering”, ‘Computer Science”, “Psychology”, “Chemistry”, “Business, Management, and Accounting”, “Health Professions”, “Nursing”, and “Arts and Humanities” had 364, 246, 237, 162, 153, 94, 79, 78, and 74 publications, respectively. Figure 3 illustrates the document classification emphasizing subject area.

Figure 3. Subject area
Top Publications by Country
Based on country classification, a total of 1519 documents were distributed, with 512, 131, 93, 76, 60, 55, and 42 articles originating from the United States, United Kingdom, Australia, Canada, India, Spain, and Indonesia, respectively. This proved that Indonesia, as a country on the Asian continent, participated in the study trends of America, Europe, and Africa regarding the analysis of problem-solving and decision-making skills. Figure 4 shows the distribution of the top 20 of 102 countries in the publication of problem-solving and decision-making skills.

Figure 4. Top countries publication on problem-solving and decision making skills
University Affiliation
Figure 5 presents the distribution of the top 20 university affiliations emphasizing problem-solving and decision-making skills.

Figure 5. Top affiliation
The distribution of publications was observed from various university affiliates, namely; America, Malaysia, Indonesia, Iran, Australia, England, Africa, etc. In this case, the University of Toronto was the most productive institution with 17 published documents on the Scopus database, accompanied by the Universities of Queensland, Melbourne, and Curtin having 13, 12, and 12 articles, respectively. The Tehran and Indonesian Universities of Medical Sciences and Education also had 11 documents, accompanied by the higher institutions of Pittsburgh, Michigan, and Malang (state) having 10 publications. According to the results, Indonesia contributed two institutions to the top 20 affiliates, namely (1) The Indonesian University of Education, and (2) The State University of Malang.
Based on the Scopus database from 2013 to 2022, the top 10 of 39 Indonesian university affiliates emphasizing the publication of problem-solving and decision-making skills in learning processes were observed. This indicated that the Indonesian University of Education was ranked first, accompanied by the Universities of Malang (state), Bina Nusantara, Mataram, Sebeleas Maret, and Semarang (state). Table 2 presents the top ten affiliates from Indonesia.
Table 2
Top ten affiliations of the University of Indonesia
No |
Affiliation |
Number of documents |
1 |
Indonesia University of Education |
11 |
2 |
State University of Malang |
10 |
3 |
Bina Nusantara University |
4 |
4 |
University of Mataram |
2 |
6 |
Sebelas Maret University |
2 |
7 |
State University of Semarang |
2 |
8 |
Kanjuruhan University Malang |
2 |
9 |
Siliwangi University |
1 |
10 |
University of Technology |
1 |
Top 10 Authors
According to the evaluation of the top 10 authors regarding the most publications, Shogren, K.A. was ranked first with 7 documents. This was accompanied by Yousefi, M., Suhandi, A., Issa, T., Hooshyar, D., and Ahmad, R.B., which had five (5) documents each. York, S., Shultz, G.V., Raley, S.K., and Horng, S.J. also possessed four publications each. In line with the results, an Indonesian writer, namely Suhandi, A. from the University of Education, analyzed decision-making and problem-solving skills. However, many universities in the country did not publish relevant documents based on the Scopus database. Table 3 illustrates the details for the top 10 authors with the highest number of publications.
Table 3
Ranking of authors with a minimum of four published documents
No |
Author |
Affiliation |
Number of documents |
Number of citations |
1 |
Shogren, K.A. |
University of Kansas, Lawrence, United States |
7 |
107 |
2 |
Yousefi, M. |
Universiti Tenaga Nasional, Kajang, Malaysia |
5 |
186 |
3 |
Suhandi, A. |
Indonesia University of Education, Bandung, Indonesia |
5 |
7 |
4 |
Issa, T |
Curtin University, Australia |
5 |
2 |
5 |
Hooshyar, D. |
University of Malaya, Kuala Lumpur, Malaysia |
5 |
186 |
6 |
Ahmad, R.B. |
University of Malaya, Kuala Lumpur, Malaysia |
5 |
186 |
7 |
York, S. |
University of Nevada, Las Vegas, United States |
4 |
231 |
8 |
Shultz, G.V. |
University of Michigan, Ann Arbor, United States |
4 |
67 |
9 |
Raley, S.K. |
University of Kansas, Lawrence, United States |
4 |
88 |
10 |
Horng, S.J. |
National Taiwan University of Science and Technology, Taiwan |
4 |
177 |
Contribution of Indonesian Experts Evolved in Problem-solving and Decision-making skills
Based on the Scopus database spanning from 2013 to 2022, the top ten of the 159 authors that published the articles exploring problem-solving and decision-making skills in learning were identified. In this case, Suhandi from the Indonesian University of Education mostly published problem-solving and decision-making themes. However, the entire learning processes emphasizing the themes remained inadequate and limited. Table 4 presents the top 10 authors from Indonesian university affiliates.
Table 4
Top ten author contributions from university affiliates in Indonesia
No |
Author |
Affiliation |
Publication |
Number of citations |
1 |
Suhandi, A. |
Indonesia University of Education |
5 |
7 |
2 |
Samsudin, A. |
Indonesia University of Education |
3 |
3 |
3 |
Syaodih, E. |
Indonesia University of Education |
2 |
3 |
4 |
Suwono, H. |
State University of Malang |
2 |
2 |
5 |
Ningsih, A. R. |
Indonesia University of Education |
2 |
3 |
6 |
Kaniawati, I. |
Indonesia University of Education |
2 |
3 |
7 |
Hakim, A. |
University of Mataram |
2 |
35 |
8 |
Zarlis, M. |
North Sumatra University |
1 |
0 |
9 |
Zarkasi, A. C |
University- Nusantara PGRI Kediri |
1 |
0 |
10 |
Yuliardi, R. |
STKIP Muhammadiyah Kuningan |
1 |
1 |
Document Citation
According to the Scopus database, the top 10 documents with consecutive highest citations were as follows, Liu, Chan and Ran, 2016 (NC=240), Leider and Griffiths, 2019 (NC=220), Ellis et al. 2017
(NC=212), Dineen-Griffin et al. 2019 (NC=144), Orgill et al. (2019; NC=107), Hooshyar et al. 2016 (NC=92), Kazancoglu and Ozkan-Ozen, 2018 (NC=91), Kegler, Stone and Holland, 2017 (NC=89), Opstoel et al. 2020 (NC=83), and Devan et al. 2018 (NC=81). Table 5 illustrates the information on the documents with the most citations.
Table 5
A breakdown of the top 10 documents on Scopus by number of citations
No |
Authors |
Year |
Title |
Source |
Number of Citation |
1 |
Liu et al. |
2016 |
Decision-making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes |
Expert Systems with Applications |
240 |
2 |
Lieder & Griffiths |
2019 |
Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources |
Behavioral and Brain Sciences |
220 |
3 |
Ellis et al. |
2017 |
Beyond Risk and Protective Factors: An Adaptation-Based Approach to Resilience |
Perspectives on Psychological Science |
212 |
4 |
Dineen-Griffin et al. |
2019 |
Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice |
PLoS ONE |
144 |
5 |
Orgill et al. |
2019 |
Introduction to Systems Thinking for the Chemistry Education Community |
Journal of Chemical Education |
107 |
6 |
Hooshyar et al. |
2016 |
Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills |
Computers and Education |
92 |
7 |
Kazancoglu & Ozkan-Ozen |
2018 |
Analyzing Workforce 4.0 in the Fourth Industrial Revolution and proposing a road map from an operations management perspective with fuzzy DEMATEL |
Journal of Enterprise Information Management |
91 |
8 |
Keg Ie ret al. |
2017 |
Trends in suicide by level of Urbanization -United States, 1999-2015 |
Morbidity and Mortality Weekly Report |
89 |
9 |
Opstoel et al. |
2020 |
Personal and social development in physical education and sports: A review study |
European Physical Education Review |
83 |
10 |
Devan et al. |
2018 |
What works and does not work in a selfmanagement intervention for people with chronic pain? Qualitative systematic reviewand meta-synthesis |
Physical Therapy |
81 |
Mapping Problem Solving and Decision Making with VOSviewer
Co-occurrence Analysis of Keywords
The classification of study subjects related to problem-solving and decision-making skills was interestingly emphasized, as a minimum number of 58,017 keywords occurred, with 743 words meeting the threshold. Network visualization was also categorized into 9 clusters, 42399 links, and 101961 total bond strengths. In this case, the red, green, blue, yellow-green, purple, deep sky blue, orange, Mocca, and light purple clusters prioritized problem-solving and decision-making skills, humans, psychology, clinical competencies, education, curriculum, support systems, creativity, and content analysis through 170, 146, 127, 126, 85, 47, 36, 5, and 1 items, respectively. Regarding the results, the bigger circle led to the high frequency of keywords in the document ( Zhang et al., 2022 ). Figure 6 presents the network visualization display using VOSviewer, emphasizing the most repeated keywords and networks.

Figure 6. Network visualization of co-occurrences with index keywords.
Overlay Visualization
The overlay visualization was symbolized by the colors showing yearly experimental trends, where purple and yellow prioritized years of study and recent analysis, respectively. These recent trends focused on the following terms, namely collaborative learning, gender, e-learning, employee, academic system, social media, formative assessment, project-based education, human experiment, creativity, quality of life, child, educational technology, grounded theory, career, theoretical study, software, games, primary care, anxiety, and machine knowledge. The overlay visualization of co-occurrence is shown in Figure 7.

Figure 7. Overlay of visualization of co-occurrences with index keywords.
Density Visualization
Density visualization was responsible for the saturation of a topic, where the blue, yellow, and red colors were emphasized yet to be analyzed, rarely studied, and highly evaluated, respectively. This indicated that brighter image colors caused the frequent use of the analyzed term in a relevant experiment. The distribution of keyword density also illustrated the frequency level of study topics ( Huang et al.,
2020). For example, problem-solving, human, and decision-making were often analytically implemented, while the rarely analyzed relevant terms focused on problem-based learning, collaboration, sustainability, metacognitive skills, qualitative and quantitative analyses, creativity, life competencies, computational thinking, game-based education, human-computer interaction, childhood, virtual patients, expertise, as well as analytic and hierarchy processes. Figure 8 presents the density visualization of decision-making and problem-solving skills based on keyword index.

inual patients


students, medical
consensus computer-assisted instruction
Clinical competence education, pharmacy operating rooms professional standard medical education education, nursing
health personnel attitude doctor patient relation procedures
computer simulation
simulation experiential learning
physician-patient relations
editorial validity
emergency health service nursing student
curriculum
psychometry
self care
cision making
pandemic questionnaire
priority journal
psychology human
leadership group process education
skill
learning
synthesis
analytical chemistry
quantitative analysis
problem solving/decision making
peer group training
teaching
writin8 engineering critical thinking . , saudi arabia e-learning
qualitative analysis
student-centered learning
technology students
scaffolds project based learning sustainability
problem-solving skills learning systems
systems thinking
behavioral research system thinkings
quality of life evidence based practice problem solving controlled study male decision making risk assessment risk management young adult daily life activity cognitive defect
, depression [dementia
Complication sc behavior game decison making
, managers employee knowledge management

product design
Figure 8. Density visualization of topic decision-making and problem-solving based on keywords index.
Discussions
According to Figure 2, a fluctuating growth for the keywords “problem-solving” and “decisionmaking skills”, was observed regarding the publication trend from 2013 to 2022, with the entire productivity remaining relatively positive. This indicated significant document elevations from 2017-2019 and 20202022, underscoring the continuous development of the study topic and providing potential for subsequent exploration of novel aspects. Furthermore, Table 1 showed that the Journal of Chemical Education was the leading source, publishing 147 articles on “problem-solving” and “decision-making skills”. These articles extensively explored the interplay between both skills in handson and mindson laboratory activities. For instance, laboratory experiment activities in organic chemistry were commonly problem-based, regarding the following performances: (1) developing students information literacy skills ( Shultz and Li, 2016 ); (2) guiding inquiry lab practices in the quantitative determination of aluminum within deodorant brands ( Sedwick et al., 2018 ); and (3) integrating environmentally friendly chemistry into the curriculum ( Aubrecht et al., 2019 ). Problem-solving was also essential for training students in systemic thinking, decisionmaking, and sustainability.
Based on Figure 3, the publications integrating problem-solving and decision-making skills within the “Social Sciences” discipline totaled 689 documents. This indicated that “Social Sciences” focused on the widest range of disciplines. It also exhibited the significance of the subject matter prioritizing the study, problem-solving, and decision-making skills, which explored the relationship between humans and their social interactions. Furthermore, “medicine” generated 364 publications and was closely related to problem-solving and decision-making skills, which were integral to effective medical practice. This proved that both skills complemented each other and helped doctors in providing high-quality patient care. Medical analysis is also presently integrated with artificial intelligence for problem-solving and decision-making ( Secinaro et al., 2021 ). From the results, “Engineering” subsequently produced 246 publications, which were closely related to both problem-solving and decision-making skills. In this case, an engineer was commonly confronted with the complex tasks requiring technical problem solutions, and rational decisionmaking processes ( Kipper et al., 2021 ). The ‘Computer Science’ discipline was also observed, with 237
publications emphasizing problem-solving and decision-making skills. These skills were considered an important foundation for developing innovative and effective technical solutions.
In line with the aforementioned descriptions, “psychology” was another discipline that yielded 162 publications, with decision-making and problem-solving skills being relevant integral components ( Bar-On, 2010 ). These skills assisted psychology professionals in handling complex tasks and aiding individuals regarding the achievement of mental and emotional well-being. Moreover, the “Chemistry” field produced 153 documents, with the subject area highly significant as the decision-making and problem-solving skills helping scientists and relevant professionals overcome intricate knowledge and management challenges ( Chen et al., 2021 ). “Business, Management and Accounting” also generated 94 publications, which were pertinent to problem-solving and decision-making skills. These skills emphasized individuals navigating complex business challenges, devising effective strategies, and efficiently managing resources to achieve organizational goals ( Proctor, 2010 ). Another field, “Health Professions”, also produced 79 documents, with the significant subject matter focusing on professional-patients relationships, diagnosing diseases, and designing treatment plans, and managing emergency situations. From the results, “Nursing” subsequently produced 78 publications, with the subject matter emphasizing the significance of problem-solving and decision-making skills. This was because the skills prioritized nurse-patient interaction, problem-solving, completing complex tasks, and nursing decisionmaking skills ( Ahmady and Shahbazi, 2020 ). In addition, “Arts and Humanities” provided 74 publications, whose significant subject matter played an essential role in understanding and interpreting relevant works and analyzing cultural contexts.
According to Figure 4, the United States, among 102 countries, was ranked first in document productivity with 512 published articles. This proved that geographic-based information retrieval helped to determine the highest publication distribution in a country ( Emmer, 2018 ; Zupic and Čater, 2015 ). Figure 5 also showed that the University of Toronto, as a top university affiliate, was ranked first with 17 documents. This was accompanied by the contributions of the Universities of Education and Malang, which represented the Indonesian institutions possessing 11, and 10 documents respectively. Moreover, the contribution of Indonesian experts in the integration of problem-solving and decision-making skills remained low according to the Scopus database. The results also indicated the top 10 contributions from 39 college affiliations within the country, regarding the publication of problem-solving and decisionmaking skills analyses. From 2013-2022, these publications remained low when perceived as document productivity. Therefore, learning should focus on fostering integrative problem-solving and decision-making skills for implementing education for sustainable development. This was because the skills were crucial in the Industry 4.0 era due to being extensively adopted by workers and managers ( Torres, Pimentel and Matias, 2023 ). Learning including real-world problems also stimulated active student engagement in applying their individual and cognitive knowledge in collaborative problem-solving ( Erol et al., 2016 ).
Based on Table 3, Shogren, K.A. from the University of Kansas, Lawrence, United States, was the most productive author with a total of 7 documents. The author also had the highest number of document citations, totaling 107. Moreover, Yousefi, M., Hooshyar, D., Ahmad, R. B., and Horng, S. J. explored intelligent guidance by using flowcharts to enhance problem-solving and decision-making ( Hooshyar et al., 2015 ). The authors also designed formative assessment with online games in an intelligent tutoring system, to enhance problem-solving and decision-making skills ( Hooshyar et al., 2016 ). In line with the results, York, S. from the University of Nevada, Las Vegas, United States, published 4 documents emphasizing Chemistry Education, with the highest citation of 231. These publications emphasized the systematic thinking approaches in Chemistry Education, to strengthen the conceptual understanding, problem-solving, and decision-making skills related to global sustainability-based issues ( Flynn et al., 2019 ; York and Orgill, 2020 ). The documents also prioritized the systematic thinking approaches in STEM Education ( York et al., 2019 ) and learning processes applicable to future chemistry educators, experts, and global citizens ( Orgill, York and Mackellar, 2019 ). According to Table 4, the top ten authors from Indonesia were Suhandi, A., Samsudin, A., Syaodih, E., Suwono, H., Kaniawati, I., Hakim, A., Zarlis, M., Zarkasi, A. C., and Yuliardi, R. This confirmed that Suhandi, A. from the Indonesia University of Education was ranked first with 5 published documents, emphasizing physics learning through the implementation of systematic thinking, to enhance problem-solving and decision-making skills. Hakim, A. from the University of Mataram also published 2 documents in the Journal of Chemical Education, with a total of 35 citations. This explained that the chemistry education prioritizing students project laboratories enhanced understanding, problem-solving, and decision-making skills ( Hakim, Kadarohman and Syah, 2016, 2020 ).
In line with the aforementioned descriptions, the citations of a document demonstrated its visibility and importance as a reference in the academic world. This was because the documents with more citations were considered more significant within a specific topic (Zupic and Čater, 2015). The number of references obtained by an article also represented recognition, influence, and popularity within the scholarly community. Based on Table 5, the authors focused on problem-solving and decision-making skills were exhibited. This indicated that Liu, Chan and Ran (2016) had the highest number of citations at 240, exhibiting the following title:”Decision-making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes”. The second-ranking publication, authored by Lieder and Griffiths in 2019, also had 220 citations with the following title, “Resourcerational analysis: Understanding human cognition as the optimal use of limited computational resources”.
According to Figure 6, a keyword network with a minimum occurrence of five terminologies linked to problem-solving and decision-making skills was illustrated. This indicated that the overlay and density visualization analyses were used to identify experimental themes, with future reports emphasizing the VOSviewer outcomes. Several terminologies such as problem-solving, decision-making, and human, were also frequently analyzed. Therefore, the abundance of publications on problem-solving and decisionmaking skills across diverse countries and disciplines prioritized the topic’s significance. This emphasized the importance of curricula designation in fostering both skills across various fields. In this case, future studies should employ collaborative approaches, gamification, computational thinking, problem-based learning, e-learning, creativity, and qualitative designs.
Conclusions
In conclusion, this study examined the bibliometric analysis of problem-solving and decision-making skills for ESD (Education for Sustainable Development) to provide insights for future analyses. This showed that the implementation of both skills was necessary across various educational levels and professions. The academic interest in problem-solving and decision-making skills also exhibited fluctuation over the previous decade. Based on the results, the Journal of Chemical Education contributed the highest number of documents, with social sciences being the most prominent subject area. The United States was also the leading contributor of articles, with the University of Toronto being the most productive institution. Moreover, the Universities of Education and Malang (state) were the top-contributing affiliations from Indonesia. Shogren, K.A., from the University of Kansas, Lawrence, United States, was also considered the most productive author with seven documents. In line with the results, Liu, Chan and Ran (2016) was the frequently referenced document published in the journal titled “Expert Systems with Applications”. According to the recommendations, numerous potential experimental themes were outlined for future consideration, specifically within the field of education. These themes underscored curriculum design, teaching models, methodologies, assessments, and learning resources, focusing on all levels from primary to higher education. The implications of this research can help related researchers find out trends in problem-solving and decision-making skills for ESD in the educational sector in general. Education for sustainable development should pay special attention to problem-solving and decision-making skills to prepare individuals to face future challenges.
Acknowledgements
Conflict of interests
The authors declare no conflict of interest.
Author Contributions
Список литературы Problem Solving and Decision-Making Skills for ESD: A Bibliometric Analysis
- Abdullah, K. H., & Abd Aziz, F. S. (2021). Mapping of laboratory safety research: a bibliometric review. Malaysian Journal of Public Health Medicine, 21(1), 303–310. https://doi.org/10.37268/mjphm/vol.21/no.1/art.864 DOI: https://doi.org/10.37268/mjphm/vol.21/no.1/art.864
- Ahmady, S., & Shahbazi, S. (2020). Impact of social problem-solving training on critical thinking and decision making of nurs-ing students. BMC Nursing, 19(1). https://doi.org/10.1186/s12912-020-00487-x DOI: https://doi.org/10.1186/s12912-020-00487-x
- Aubrecht, K. B., Bourgeois, M., Brush, E. J., Mackellar, J., & Wissinger, J. E. (2019). Integrating Green Chemistry in the Cur-riculum: Building Student Skills in Systems Thinking, Safety, and Sustainability. Journal of Chemical Education, 96(12), 2872–2880. https://doi.org/10.1021/acs.jchemed.9b00354 DOI: https://doi.org/10.1021/acs.jchemed.9b00354
- Bar-On, R. (2010). Emotional intelligence: An integral part of positive psychology. South African Journal of Psychology, 40(1), 54–62. https://doi.org/10.1177/008124631004000106 DOI: https://doi.org/10.1177/008124631004000106
- Cai, Y., Jin, F., Liu, J., Zhou, L., & Tao, Z. (2023). A survey of collaborative decision-making: Bibliometrics, preliminaries, methodologies, applications and future directions. Engineering Applications of Artificial Intelligence, 122, 106064. DOI: https://doi.org/10.1016/j.engappai.2023.106064
- Carlsson, C. (2019). Combining ANFIS and Digital Coaching for Good Decisions in Industrial Processes. In K. R.B., B. I., R. M., C. M., & K. V. (Eds.), Advances in Intelligent Systems and Computing (Vol. 1000, pp. 190–200). Springer Verlag. https://doi.org/10.1007/978-3-030-21920-8_18 DOI: https://doi.org/10.1007/978-3-030-21920-8_18
- Chen, L. C., & Utama, D. N. (2022). Decision Support Model for Determining the Best Employee using Fuzzy Logic and Sim-ple Additive Weighting. Journal of Computer Science, 18(6), 530–539. https://doi.org/10.3844/jcssp.2022.530.539 DOI: https://doi.org/10.3844/jcssp.2022.530.539
- Chen, L., Kong, D., Zhang, S., & Yang, L. (2021). A quasi-experimental study of specialized training on the clinical deci-sion-making skills and social problem-solving abilities of nursing students. Contemporary Nurse, 57(1–2), 4–12. https://doi.org/10.1080/10376178.2021.1912616 DOI: https://doi.org/10.1080/10376178.2021.1912616
- Chen, P.-Y., Hwang, G.-J., Yeh, S.-Y., Chen, Y.-T., Chen, T.-W., & Chien, C.-H. (2021). Three decades of game-based learning in science and mathematics education: an integrated bibliometric analysis and systematic review. Journal of Computers in Education, 1–22. https://doi.org/10.1007/s40692-021-00210-y DOI: https://doi.org/10.1007/s40692-021-00210-y
- Chen, S.-Y., Tsai, J.-C., Liu, S.-Y., & Chang, C.-Y. (2021). The effect of a scientific board game on improving creative problem solving skills. Thinking Skills and Creativity, 41. https://doi.org/10.1016/j.tsc.2021.100921 DOI: https://doi.org/10.1016/j.tsc.2021.100921
- Costa, A. L., & Kallick, B. (2009). Habits of mind across the curriculum: Practical and creative strategies for teachers. ASCD.
- Costa, D. F., de Melo Carvalho, F., de Melo Moreira, B. C., & do Prado, J. W. (2017). Bibliometric analysis on the association between behavioral finance and decision making with cognitive biases such as overconfidence, anchoring effect and confirmation bias. Scientometrics, 111, 1775–1799. https://doi.org/10.1007/s11192-017-2371-5 DOI: https://doi.org/10.1007/s11192-017-2371-5
- Devan, H., Hale, L., Hempel, D., Saipe, B., & Perry, M. A. (2018). What works and does not work in a self-management intervention for people with chronic pain? Qualitative systematic review and meta-synthesis. Physical Therapy, 98(5), 381–397. https://doi.org/10.1093/ptj/pzy029 DOI: https://doi.org/10.1093/ptj/pzy029
- De Villiers, R., Hankin, R., & Woodside, A. G. (2016). Making decisions well and badly: How stakeholders’ discussions influ-ence individual executives’ decision confidence and competence. Advances in Business Marketing and Purchasing, 24, 87–116. https://doi.org/10.1108/S1069-096420160000024006 DOI: https://doi.org/10.1108/S1069-096420160000024006
- Delavari, S., Arabshahi, K. S., Amini, M., Aalaa, M., Pourbairamian, G., Bahoosh, N., Asadi, N., Dalal, B., Kojuri, J., Hamidi, H., & Delavari, S. (2022). The Relationship between Experiences Level and Clinical Decision-Making Skill in Midwifery Students: A Cross-Sectional Study. Medical Journal of the Islamic Republic of Iran, 36(1). https://doi.org/10.47176/mjiri.36.80 DOI: https://doi.org/10.47176/mjiri.36.80
- Di Vaio, A., Hassan, R., & Alavoine, C. (2022). Data intelligence and analytics: A bibliometric analysis of human–Artificial intel-ligence in public sector decision-making effectiveness. Technological Forecasting and Social Change, 174, 121201. https://doi.org/10.1016/j.techfore.2021.121201 DOI: https://doi.org/10.1016/j.techfore.2021.121201
- Dineen-Griffin, S., Garcia-Cardenas, V., Williams, K., & Benrimoj, S. I. (2019). Helping patients help themselves: A systematic review of self-management support strategies in primary health care practice. PLoS ONE, 14(8). https://doi.org/10.1371/journal.pone.0220116 DOI: https://doi.org/10.1371/journal.pone.0220116
- Ellis, B. J., Bianchi, J., Griskevicius, V., & Frankenhuis, W. E. (2017). Beyond Risk and Protective Factors: An Adaptation-Based Approach to Resilience. Perspectives on Psychological Science, 12(4), 561–587. https://doi.org/10.1177/1745691617693054 DOI: https://doi.org/10.1177/1745691617693054
- Emmer, A. (2018). Geographies and scientometrics of research on natural hazards. Geosciences, 8(10), 382. https://doi.org/10.3390/geosciences8100382 DOI: https://doi.org/10.3390/geosciences8100382
- Erol, S., Jäger, A., Hold, P., Ott, K., & Sihn, W. (2016). Tangible Industry 4.0: A Scenario-Based Approach to Learning for the Future of Production. Procedia CIRP, 54, 13–18. https://doi.org/10.1016/j.procir.2016.03.162 DOI: https://doi.org/10.1016/j.procir.2016.03.162
- Flynn, A. B., Orgill, M., Ho, F. M., York, S., Matlin, S. A., Constable, D. J. C., & Mahaffy, P. G. (2019). Future Directions for Systems Thinking in Chemistry Education: Putting the Pieces Together. Journal of Chemical Education, 96(12), 3000–3005. https://doi.org/10.1021/acs.jchemed.9b00637 DOI: https://doi.org/10.1021/acs.jchemed.9b00637
- Fruchter, R., Katz, G., & Grey, F. (2018). From technical to tactical, and strategic BIM coordination in support of model-based decision making. In Transforming Engineering Education: Innovative Computer-Mediated Learning Technologies (pp. 173–196). American Society of Civil Engineers (ASCE). https://doi.org/10.1061/9780784414866.ch06 DOI: https://doi.org/10.1061/9780784414866.ch06
- Gao, P., Meng, F., Mata, M. N., Martins, J. M., Iqbal, S., Correia, A. B., Dantas, R. M., Waheed, A., Xavier Rita, J., & Farrukh, M. (2021). Trends and future research in electronic marketing: A bibliometric analysis of twenty years. Journal of Theo-retical and Applied Electronic Commerce Research, 16(5), 1667–1679. https://doi.org/10.3390/jtaer16050094 DOI: https://doi.org/10.3390/jtaer16050094
- Gao, R. (2015). Incorporating students’ self-designed, research-based analytical chemistry projects into the instrumentation curriculum. Journal of Chemical Education, 92(3), 444–449. https://doi.org/10.1021/ed500502w DOI: https://doi.org/10.1021/ed500502w
- Ha, C. T., Thao, T. T. P., Trung, N. T., Huong, L. T. T., Dinh, N. Van, & Trung, T. (2020). A Bibliometric Review of Research on STEM Education in ASEAN: Science Mapping the Literature in Scopus Database, 2000 to 2019. Eurasia Journal of Mathematics, Science and Technology Education, 16(10). https://doi.org/10.29333/ejmste/8500 DOI: https://doi.org/10.29333/ejmste/8500
- Hakim, A., Jufri, A. W., & Mutmainnah, P. A. (2020). Understanding the Uniqueness of Artocarpus Flavonoids: Isolation and Structure Elucidation of Cycloartocarpin from the Roots of Artocarpus altilis. Journal of Chemical Education, 97(11), 4133–4136. https://doi.org/10.1021/acs.jchemed.0c00199 DOI: https://doi.org/10.1021/acs.jchemed.0c00199
- Hakim, A., Kadarohman, A., & Syah, Y. M. (2016). Making a Natural Product Chemistry Course Meaningful with a Mini Project Laboratory. Journal of Chemical Education, 93(1), 193–196. https://doi.org/10.1021/ed500930s DOI: https://doi.org/10.1021/ed500930s
- Holden, G., Rosenberg, G., & Barker, K. (2005). Bibliometrics: A potential decision making aid in hiring, reappointment, tenure and promotion decisions. Social Work in Health Care, 41(3–4), 67–92. https://doi.org/10.1300/J010v41n03_03 DOI: https://doi.org/10.1300/J010v41n03_03
- Hooshyar, D., Ahmad, R. B., Yousefi, M., Fathi, M., Horng, S.-J., & Lim, H. (2016). Applying an online game-based formative assessment in a flowchart-based intelligent tutoring system for improving problem-solving skills. Computers and Edu-cation, 94, 18–36. https://doi.org/10.1016/j.compedu.2015.10.013 DOI: https://doi.org/10.1016/j.compedu.2015.10.013
- Hooshyar, D., Ahmad, R. B., Yousefi, M., Yusop, F. D., & Horng, S.-J. (2015). A flowchart-based intelligent tutoring system for improving problem-solving skills of novice programmers. Journal of Computer Assisted Learning, 31(4), 345–361. https://doi.org/10.1111/jcal.12099 DOI: https://doi.org/10.1111/jcal.12099
- Huang, C., Yang, C., Wang, S., Wu, W., Su, J., & Liang, C. (2020). Evolution of topics in education research: a systematic review using bibliometric analysis. Educational Review, 72(3), 281–297. https://doi.org/10.1080/00131911.2019.1566212 DOI: https://doi.org/10.1080/00131911.2019.1566212
- Jauhariyah, M. N. R., Prahani, B. K., Syahidi, K., Deta, U. A., Lestari, N. A., & Hariyono, E. (2021). ESD for physics: How to infuse education for sustainable development (ESD) to the physics curricula? Journal of Physics: Conference Series, 1747(1). https://doi.org/10.1088/1742-6596/1747/1/012032 DOI: https://doi.org/10.1088/1742-6596/1747/1/012032
- Karakus, M., Ersozlu, A., & Clark, A. C. (2019). Augmented reality research in education: A bibliometric study. Eurasia Journal of Mathematics, Science and Technology Education, 15(10). https://doi.org/10.29333/ejmste/103904 DOI: https://doi.org/10.29333/ejmste/103904
- Kazancoglu, Y., & Ozkan-Ozen, Y. D. (2018). Analyzing Workforce 4.0 in the Fourth Industrial Revolution and proposing a road map from operations management perspective with fuzzy DEMATEL. Journal of Enterprise Information Management, 31(6), 891–907. https://doi.org/10.1108/JEIM-01-2017-0015 DOI: https://doi.org/10.1108/JEIM-01-2017-0015
- Kegler, S. R., Stone, D. M., & Holland, K. M. (2017). Trends in suicide by level of Urbanization – United states, 1999-2015. Morbidity and Mortality Weekly Report, 66(10), 270–273. https://doi.org/10.15585/mmwr.mm6610a2 DOI: https://doi.org/10.15585/mmwr.mm6610a2
- Kipper, L. M., Iepsen, S., Dal Forno, A. J., Frozza, R., Furstenau, L., Agnes, J., & Cossul, D. (2021). Scientific mapping to identify competencies required by industry 4.0. Technology in Society, 64, 101454. https://doi.org/10.1016/j.techsoc.2020.101454 DOI: https://doi.org/10.1016/j.techsoc.2020.101454
- Kwok, P. M. H., & Hodgson, P. (2017). Practicing decision making: Using flowcharts with twenty-first-century marketers. In-ternational Journal of Pedagogy and Curriculum, 24(2), 1–14. https://doi.org/10.18848/2327-7963/CGP/v24i02/1-14 DOI: https://doi.org/10.18848/2327-7963/CGP/v24i02/1-14
- Laurie, R., Nonoyama-Tarumi, Y., Mckeown, R., & Hopkins, C. (2016). Contributions of Education for Sustainable Develop-ment (ESD) to Quality Education: A Synthesis of Research. Journal of Education for Sustainable Development, 10(2), 226–242. https://doi.org/10.1177/0973408216661442 DOI: https://doi.org/10.1177/0973408216661442
- Liao, H., Mi, X., & Xu, Z. (2020). A survey of decision-making methods with probabilistic linguistic information: bibliometrics, preliminaries, methodologies, applications and future directions. Fuzzy Optimization and Decision Making, 19, 81–134. https://doi.org/10.1007/s10700-019-09309-5 DOI: https://doi.org/10.1007/s10700-019-09309-5
- Lieber, L. S., Ibraj, K., Caspari-Gnann, I., & Graulich, N. (2022). Students’ Individual Needs Matter: A Training to Adaptively Address Students’ Argumentation Skills in Organic Chemistry. Journal of Chemical Education, 99(7), 2754–2761. https://doi.org/10.1021/acs.jchemed.2c00213 DOI: https://doi.org/10.1021/acs.jchemed.2c00213
- Lieder, F., & Griffiths, T. L. (2019). Resource-rational analysis: Understanding human cognition as the optimal use of limited computational resources. Behavioral and brain sciences, 43, e1. https://doi.org/10.1017/S0140525X1900061X DOI: https://doi.org/10.1017/S0140525X1900061X
- Liu, S., Chan, F. T. S., & Ran, W. (2016). Decision making for the selection of cloud vendor: An improved approach under group decision-making with integrated weights and objective/subjective attributes. Expert Systems with Applications, 55, 37–47. https://doi.org/10.1016/j.eswa.2016.01.059 DOI: https://doi.org/10.1016/j.eswa.2016.01.059
- Macduff, C., Stephen, A., & Taylor, R. (2016). Decision precision or holistic heuristic?: Insights on on-site selection of student nurses and midwives. Nurse Education in Practice, 16(1), 40–46. https://doi.org/10.1016/j.nepr.2015.06.008 DOI: https://doi.org/10.1016/j.nepr.2015.06.008
- Maditati, D. R., Munim, Z. H., Schramm, H. J., & Kummer, S. (2018). A review of green supply chain management: From bibliometric analysis to a conceptual framework and future research directions. Resources, Conservation and Recycling, 139, 150-162. https://doi.org/10.1016/j.resconrec.2018.08.004 DOI: https://doi.org/10.1016/j.resconrec.2018.08.004
- Masitoh, P. N. A., Latifah, S., Saregar, A., Aziz, A., & Jamaluddin, W. (2021). Bibliometric analysis of physics problem solving. Journal of Physics: Conference Series, 1796(1), 12009. https://doi.org/10.1088/1742-6596/1796/1/012009 DOI: https://doi.org/10.1088/1742-6596/1796/1/012009
- Méndez, B. E. P., Calleros, J. M. G., Garciá, C. A. R., & Garciá, J. G. (2020). Fuzzy Models for Implementation of the Deci-sion-Making Module in Networked Didactic Prototypes. Computacion y Sistemas, 24(2), 539–551. https://doi.org/10.13053/CyS-24-2-3378 DOI: https://doi.org/10.13053/cys-24-2-3378
- Molokhina, G. A., Pishchik, V. I., & Fomin, A. (2021). Features of critical thinking of individual entrepreneurs with different levels of self-regulation. International Journal of Cognitive Research in Science, Engineering and Education, 9(2), 253–264. https://doi.org/10.23947/2334-8496-2021-9-2-253-264 DOI: https://doi.org/10.23947/2334-8496-2021-9-2-253-264
- Nasri, N. M., Nasri, N., & Talib, M. A. A. (2020). Physics teachers’ perceptions on sustainable physics education. Journal of Baltic Science Education, 19(4), 569–582. https://doi.org/10.33225/jbse/20.19.569 DOI: https://doi.org/10.33225/jbse/20.19.569
- Opstoel, K., Chapelle, L., Prins, F. J., De Meester, A., Haerens, L., van Tartwijk, J., & De Martelaer, K. (2020). Personal and social development in physical education and sports: A review study. European Physical Education Review, 26(4), 797–813. https://doi.org/10.1177/1356336X19882054 DOI: https://doi.org/10.1177/1356336X19882054
- Orgill, M., York, S., & Mackellar, J. (2019). Introduction to Systems Thinking for the Chemistry Education Community. Journal of Chemical Education, 96(12), 2720–2729. https://doi.org/10.1021/acs.jchemed.9b00169 DOI: https://doi.org/10.1021/acs.jchemed.9b00169
- Pacheco-Velázquez, E. A., Vázquez-Parra, J. C., Cruz-Sandoval, M., Salinas-Navarro, D. E., & Carlos-Arroyo, M. (2023). Business Decision-Making and Complex Thinking: A Bibliometric Study. Administrative Sciences, 13(3), 80. https://doi.org/10.3390/admsci13030080 DOI: https://doi.org/10.3390/admsci13030080
- Pertl, M.-T., Zamarian, L., & Delazer, M. (2017). Reasoning and mathematical skills contribute to normatively superior decision making under risk: evidence from the game of dice task. Cognitive Processing, 18(3), 249–260. https://doi.org/10.1007/s10339-017-0813-x DOI: https://doi.org/10.1007/s10339-017-0813-x
- Proctor, T. (2010). Creative problem solving for managers: developing skills for decision making and innovation. Routledge.
- Saab, M. M., Hegarty, J., Murphy, D., & Landers, M. (2021). Incorporating virtual reality in nurse education: a qualitative study of nursing students’ perspectives. Nurse Education Today, 105, 105045. https://doi.org/10.1016/j.nedt.2021.105045 DOI: https://doi.org/10.1016/j.nedt.2021.105045
- Saregar, A., Sunyono, Haenilah, E. Y., Hariri, H., Putra, F. G., Diani, R., Misbah, & Umam, R. (2022). Natural Disaster Educa-tion in School: A Bibliometric Analysis with a Detailed Future Insight Overview. International Journal of Educational Methodology, 8(4), 743–757. https://doi.org/10.12973/ijem.8.4.743 DOI: https://doi.org/10.12973/ijem.8.4.743
- Secinaro, S., Brescia, V., Calandra, D., & Biancone, P. (2020). Employing bibliometric analysis to identify suitable business models for electric cars. Journal of Cleaner Production, 264, 121503. https://doi.org/10.1016/j.jclepro.2020.121503 DOI: https://doi.org/10.1016/j.jclepro.2020.121503
- Secinaro, S., Calandra, D., Secinaro, A., Muthurangu, V., & Biancone, P. (2021). The role of artificial intelligence in healthcare: a structured literature review. BMC medical informatics and decision making, 21, 1-23. https://doi.org/10.1186/s12911-021-01488-9 DOI: https://doi.org/10.1186/s12911-021-01488-9
- Sedwick, V., Leal, A., Turner, D., & Bakarr Kanu, A. (2018). Quantitative Determination of Aluminum in Deodorant Brands: A Guided Inquiry Learning Experience in Quantitative Analysis Laboratory. Journal of Chemical Education, 95(3), 451–455. https://doi.org/10.1021/acs.jchemed.7b00336 DOI: https://doi.org/10.1021/acs.jchemed.7b00336
- Shultz, G. V, & Li, Y. (2016). Student Development of Information Literacy Skills during Problem-Based Organic Chemistry Laboratory Experiments. Journal of Chemical Education, 93(3), 413–422. https://doi.org/10.1021/acs.jchemed.5b00523 DOI: https://doi.org/10.1021/acs.jchemed.5b00523
- Silvianty, A., Suhandi, A., & Setiawan, W. (2019). Video supported critical thinking test in the kinetic theory of gases: Validity and reliability. In P. I., A. R.R., S. null, S. null, N. A.B.D., & A. A.G. (Eds.), Journal of Physics: Conference Series (Vol. 1157, Issue 3). Institute of Physics Publishing. https://doi.org/10.1088/1742-6596/1157/3/032052 DOI: https://doi.org/10.1088/1742-6596/1157/3/032052
- Suprapto, N., Sukarmin, S., Puspitawati, R. P., Erman, E., Savitri, D., Ku, C. H., & Mubarok, H. (2021). Research trend on TPACK through bibliometric analysis (2015-2019). International Journal of Evaluation and Research in Education, 10(4), 1375–1385. https://doi.org/10.11591/IJERE.V10I4.22062 DOI: https://doi.org/10.11591/ijere.v10i4.22062
- Suseelan, M., Chew, C. M., & Chin, H. (2022). Research on mathematics problem solving in elementary education conducted from 1969 to 2021: A bibliometric review. International Journal of Education in Mathematics, Science, and Technology (IJEMST), 10(4), 1003-1029. https://doi.org/10.46328/ijemst.2198 DOI: https://doi.org/10.46328/ijemst.2198
- Torres, D., Pimentel, C., & Matias, J. C. (2023). Characterization of Tasks and Skills of Workers, Middle and Top Managers in the Industry 4.0 Context. Sustainability, 15(8), 6981. https://doi.org/10.3390/su15086981 DOI: https://doi.org/10.3390/su15086981
- Tripon, A. (2014). Innovative Technology for Sustainable Development of Human Resource Using Non-formal and Informal Education. Procedia Technology, 12, 598–603. https://doi.org/10.1016/j.protcy.2013.12.535 DOI: https://doi.org/10.1016/j.protcy.2013.12.535
- Tsai, M. N., Liao, Y. F., Chang, Y. L., & Chen, H. C. (2020). A brainstorming flipped classroom approach for improving students’ learning performance, motivation, teacher-student interaction and creativity in a civics education class. Thinking Skills and Creativity, 38(July), 100747. https://doi.org/10.1016/j.tsc.2020.100747 DOI: https://doi.org/10.1016/j.tsc.2020.100747
- UNESCO. (2012). Education for Sustainable Development in Action. In United Nations Educational, Scientific and Cultural Organization.
- Van Eck, N. J., & Waltman, L. (2010). Software survey: VOSviewer, a computer program for bibliometric mapping. Scien-tometrics, 84(2), 523–538. https://doi.org/10.1007/s11192-009-0146-3 DOI: https://doi.org/10.1007/s11192-009-0146-3
- Van Eck, N. J., & Waltman, L. (2012). Manual for VOSviewer version 1.5.2. Leiden Univeristy, September, 1–28. http://www.vosviewer.com/documentation/Manual_VOSviewer_1.5.4.pdf
- Vílchez-Román, C., Sanguinetti, S., & Mauricio-Salas, M. (2021). Applied bibliometrics and information visualization for deci-sion-making processes in higher education institutions. Library Hi Tech, 39(1), 263–283. https://doi.org/10.1108/LHT-10-2019-0209 DOI: https://doi.org/10.1108/LHT-10-2019-0209
- Wilson, M. V, & Wilson, E. (2017). Authentic performance in the instrumental analysis laboratory: Building a visible spectro-photometer prototype. Journal of Chemical Education, 94(1), 44–51. https://doi.org/10.1021/acs.jchemed.6b00515 DOI: https://doi.org/10.1021/acs.jchemed.6b00515
- York, S., Lavi, R., Dori, Y. J., & Orgill, M. (2019). Applications of Systems Thinking in STEM Education. Journal of Chemical Education, 96(12), 2742–2751. https://doi.org/10.1021/acs.jchemed.9b00261 DOI: https://doi.org/10.1021/acs.jchemed.9b00261
- York, S., & Orgill, M. (2020). ChEMIST Table: A Tool for Designing or Modifying Instruction for a Systems Thinking Approach in Chemistry Education. Journal of Chemical Education, 97(8), 2114–2129. https://doi.org/10.1021/acs.jchemed.0c00382 DOI: https://doi.org/10.1021/acs.jchemed.0c00382
- Zhang, L., Carter, R. A., Qian, X., Yang, S., Rujimora, J., & Wen, S. (2022). Academia’s responses to crisis: A bibliometric analysis of literature on online learning in higher education during COVID-19. British Journal of Educational Technology, 53(3), 620–646. https://doi.org/10.1111/bjet.13191 DOI: https://doi.org/10.1111/bjet.13191
- Zupic, I., & Čater, T. (2015). Bibliometric Methods in Management and Organization. Organizational Research Methods, 18(3), 429–472. https://doi.org/10.1177/1094428114562629 DOI: https://doi.org/10.1177/1094428114562629