Henrietta Molnár MUZSALYINÉ – András HORVÁTH – Sándor LÉNÁRD – István NAHALKA– Katalin RADNÓTI[1]

Current trends and developments in the application of digital tools and implementation of environmental awareness and sustainability in PBL-based STEM education: a dual systematic literature review

Introduction

Problem-Based Learning (PBL) is a part of meaningful, experiential learning, where students learn by solving problems and reflecting on their experiences (Barrows and Tamblyn, 1980). PBL can be used to help students become active learners by shifting learning to real-world problems and making students responsible for their learning. It places a dual emphasis on helping students develop strategies and construct knowledge (Collins et al., 1989; Hmelo and Ferrari, 1997; Kolodner et al., 1996). Cooperative learning methods are often used in problem-based learning (Hmelo-Silver, 2004).

STEM (Science Technology Engineering and Mathematics) in education is both a curriculum and a pedagogy. Moore et al. (2014) designated a framework that includes six important tenets for quality STEM education:

·     the inclusion of math and science content,

·     student-centered pedagogy,

·     lessons are situated in an engaging and motivating context,

·     inclusion of engineering design or redesign challenge,

·     students learn from making mistakes,

·     teamwork is emphasized.

However, Honey et al. (2014) argue that STEM education is not a well-defined concept.

The problem-based approach has been widely used in STEM education in recent years. PBL directly affected interest in a future STEM career (LaForce et al., 2017). PBL-STEM significantly increased students' problem-solving skills compared to the traditional class (Parno et al., 2019). Moreover, it was also found that STEM PBLs in schools helped lower-achieving students more and reduced achievement gaps (Han et al., 2015).

The combination of advances in technology, the digital literacy of Generation Z students, and online education due to the Covid-19 pandemic has meant that PbL STEM has very often been delivered in a digital environment in the recent past. For this reason, we considered it essential to examine how digitally-enhanced PBL in STEM education has been implemented in the last two years (from 2020 to 2022).

The most recent trends in the development of STEM curricula are implementing new, relevant topics, including environmental awareness (Nantsou and Tombras, 2022) and sustainability (Smith and Watson, 2019). Since these topics involve many fields of knowledge (Clark and Button, 2011), complex learning outcomes can be set and pursued (Rogers et al., 2015) – all the necessary vital competencies must be improved accordingly to critical thinking (Erikson and Erikson, 2019).

Methodology

Research questions

Many recent publications which feature research on PBL-based STEM teaching and curricular development contain curricular elements and components of environmental education and sustainability. Therefore, we intended to analyze, classification, and assess the practical methods and tools along with the following criteria:

·     RQ1: What methods and tools are used in problem-based STEM education that can raise environmental awareness and form the students' attitude concerning sustainability?

·     RQ2: What key factors can we identify in successfully implementing such curricular content?

·     RQ3: What types of digital devices are used in problem-based STEM education? What exactly is the role of digital tools in problem-based STEM learning?

·     RQ4: What problems do students have to solve in problem-based STEM learning using a digital device?

Literature search process

We chose the Web of Science (WoS) database as it is the gateway for all the Science Citation Indexed (SCI) and Social Science Citation Indexed (SSCI) journals. We queried the WoS database for the first time on the 1st of April 2022. The search string (problem-based learning AND STEM) was keyed into the advanced search option of the Web of Science database. We then specified the range of years from 2020 to 2022. Problem-based learning and STEM research areas have been very active over the years. However, we investigate only the most recent literature because technology evolves and rapidly changes. We, therefore, focused our research on considering studies from 2020 to 2022.

Eligibility criteria

Inclusion and exclusion criteria were set to refine further the 377 results obtained. First, we read the 377 abstracts. We then looked at how these articles were grouped around broad themes. We found two particular groups:

·     PBL in STEM education by using some digital device,

·     PBL is related to environmental education or sustainability in STEM education. 

Afterward, we narrowed our list to only articles on topics 1 or 2 above (126 articles). Of the studies with topics mentioned above, only the following were considered:

PBL in STEM education by using some kind of digital device

Of the studies with topics mentioned above, only the following were considered:

·     Inclusion criteria: (a) empirical studies, (b) PBL in STEM education must be the central topic of the article, and (c) the study must focus on students and their PBL activities in STEM education.

·     Exclusion criteria: (a) inadequate genre (theoretical study, literature, or systematic literature review), (b) the teachers are the main actors of the study (instead of students), (c) the study focuses on the digital tool that can be used to implement PBL in STEM education, (d) non-English articles, (e) articles in which the full text was not available.

After this procedure, we got 14 articles about problem-based learning in STEM education using a digital device.

PBL related to environmental education or sustainability in STEM education

Out of the studies mentioned above, only the following ones were considered: 

·     Inclusion criteria: (a) empirical studies, (b) PBL in STEM education must be the main instrument to propagate environmental awareness towards the pupils, and (c) the study focuses at least partially on students and their PBL activities in STEM education.

·     Exclusion criteria: (a) inadequate genre (theoretical study, literature, or systematic literature review), (b) non-English articles, and (c) articles in which the full text was not available.

After this procedure, we got 13 articles about problem-based learning in STEM education-related to either environmental education or sustainability. Out of the remaining articles, one was related to both of our chosen topics, so altogether, this article covers the systematic review of those 26 articles (Fig. 1).

Figure 1: An overview of the search protocol

How PBL in STEM education contributes to environmental awareness and sustainability – and how it can be implemented successfully

General characteristics of RQ1

A minority (n=3) of the collected articles were based mainly on the development of a specific framework (Nawi et al., 2019) or model for (not necessarily only) environmental education, which put the focus of the entire learning process on the students, e.g., IGNITE (Dotson et al., 2020) or PRInK (Yahaya et al., 2021). However, since Nawi et al. tested their framework on a specific topic, we also classified their article as a case study. Therefore, this article and all the remaining publications (n=11) were specific case studies in which the application of certain problem-based and other learning tools and methods were used during the learning process. The subjects of the studies varied from 3rd-grade pupils to university students and pre-service teachers (Rico et al., 2021; Maass et al., 2022).

Educational methods for the frameworks

Since it is common practice to the couple (C)PBL and TEL (Technology-Enhanced Learning) (Nawi et al., 2019) or even its more advanced form of TEAL (Technology-Enabled Active Learning) (Shay et al., 2020), the most common component of these models was the technology as mentioned above assisted learning in many forms, especially Computer-Assisted Learning (Yahaya et al., 2021; Molan et al., 2022).

Other primary aspects were contextualizing the problem and the student's motivation. For this, the teachers and educators used the 5E (Engage, Explore, Explain, Elaborate, Evaluate) model (Bybee et al., 2006) for instruction in many cases (n=7).

Educational tools for the methods

As Table 1. shows, the educational tools used by the studies ranged from minor (Nawi et al., 2019) to no (Songer and Ibarrola Recalde, 2021; Talley et al., 2021) use of apps as a learning platform to partly (Rossano and Calvano, 2020) or immersive virtual environments (Molan et al., 2022).

The case studies can be divided into three categories based on the level of digitalization their methods and tools used based on the SAMR model (Hamilton et al., 2016):

·     In some cases, there is no need to use technology further than to substitute or augment the already existing research and education tools to make the learning tools and the process more compatible with the younger generations (Maass et al., 2022; Rico et al., 2021; Songer and Ibarrola Recalde, 2021; Snell-Rood et al., 2021). It also makes research and education activities more easily conductive for technologically less developed institutions and communities (Gallay et al., 2021; 2021; Talley et al., 2021).

·     In the remaining cases, digital technology can modify or even wholly redefine certain parts of the learning process by providing a new feature (Araya and Collanqui, 2021; Nawi et al., 2019; Shay et al., 2021) or a completely new task (Molan et al., 2021; Rossano and Calvano, 2020).

Table 1: Classification of the case studies by scientific topic, research subjects and main educational tool.

Education system

The first group of such features is those that can be regulated and monitored by levels and members of the education system or systems, which serve as a stage for innovation. One of the most important contributing factors is the clarity of all involved education policies – whether the policymakers (and the teachers) know and accept the goals of the modernization.

Those goals are usually clearly set in the learning outcomes, which can be categorized in many ways - e.g., by the types (content, practice) of learning items (Songer and Ibarrola Recalde, 2021) or by the classical three-way partitioning of the outcomes (stores of learning, capabilities, and attitudes). The latter was done implicitly in some articles (e.g., Rico et al., 2021) and more explicitly in others. For example, Molan et al. (2021) classify even more precise outcomes (learning objectives) for each step of the virtual PBL-based learning tool they designed - the most poignant examples of that are the three articles that involve different, precisely designed frameworks (Nawi et al., 2019) or models of learning (Dotson et al., 2020; Yahaya et al., 2021).

Another important systematic component of implementational success can be the instruments for development - however, the impact of these tools depends strongly on the complexity of the learning tools utilized by the learners. This impact can vary within vast limits, from having very little weight in the development (Maass et al., 2022) to being the sole instrument of the learning process (Rossano and Calvano, 2020).

A final addition to the potential contributing factors is a capable assessment system which may be created by developments of specific components of the internal assessment system of the school to support the means and goals of the modernization (Yahaya et al., 2021) and by creating and operating an independent, external assessment system to provide a risk-based evaluation (Nawi et al., 2019), or both (Dotson et al., 2020).

Role of the community (learners, teachers)

Other than formal education, some (n=3) studies introduced community science (Talley et al., 2021) or civic science (Gallay et al., 2021; Snell-Rood et al., 2021), especially so to compensate for the underrepresentation of participants in specific ethnic (Hispanic minorities in the State of California (Talley et al., 2021; Shay et al., 2020)) or socio-economic status (urban (Gallay et al., 2021) and foster students (Songer and Ibarrola Recalde, 2021)) status. The most probable general idea behind applying this approach was to elevate the socio-scientific status of these communities.

Horizontal cooperation (another operational system through which the teachers can access pedagogical consultation opportunities, professional support (e.g., literature), or horizontal network cooperation) also has to be considered a contributing factor. It may happen between schools in the exact count(r)y (Gallay et al., 2021; Songer and Ibarrola Recalde, 2021; Talley et al., 2021) or at the same level of education but in different countries (Araya and Collanqui, 2021) and also between different levels of research and education (Dotson et al., 2020). The latter can largely contribute to a broader range of applications as education methods and tools used in multiple systems have to be up to many different standards - a good practice that has been developed that way is the already referenced 5E model, which was incorporated in many of the studies researched by us (n=7).

Perhaps the most crucial contributing factor belongs to the teachers as individuals: their lesson planning practice has to enable said development processes; otherwise, the necessary learning tools cannot be adequately utilized. It is incredibly decisive when it comes to complex ICT tools (Molan et al., 2021; Shay et al., 2020), new or uncommon methods and practices (Molan et al., 2021; Rossano and Calvano, 2020) - especially new frameworks (Nawi et al., 2019) or learning models (Dotson et al., 2020; Yahaya et al., 2021).

Discussion of RQ1 and RQ2 via SWOT of the implementation of PBL

After identifying the key factors necessary for implementational success, we can also analyze other features of PBL that may motivate and navigate the schools and teachers during the implementation process. To do that, instead of subjecting PBL to SWOT, we subject the implementation process itself to the analysis. As Table 2. suggests, these possible strengths, weaknesses, opportunities, and threats concerning the implementation of PBL can be divided into features that can be managed or altered by either the institutes or the educators themselves (and in some rare cases, both).

Table 2: SWOT analysis for the implementation of PBL

Unsurprisingly, the SWOT of the implementation is very similar to the SWOT of PBL itself (Azaceta et al., 2018; Patrick et al., 2020). however, the focus on specific areas is shifted towards the systematic and human resources rather than the characteristics of the theoretical framework of (Collaborative) Problem Based Learning.

Problem-based learning in STEM education by using some kind of digital device

General characteristics of RQ3 (the types and role of digital devices)

Two large groups of digital devices appear in the studies examined. First, robotics was most popular - it was used in half of the cases (n=7). Different kinds of programming were used in problem-based STEM education in three cases. In the remaining four studies, virtual reality, computer games, computer agents, and interactive online boards occurred only once. The summary of these results is shown in Table 3.

Table 3: The types of digital devices used in problem-based STEM education

It should be made clear that, although programming is also used in robotics, in our study, programming means that the task is performed in a programming environment only, without using any physical panels, robots, sensors, or detectors.

Robotics in problem-based STEM education

Based on the studies reviewed, using robotics seems to provide an appropriate problem-based STEM learning environment for all ages. Robotic devices, like Arduino, Micro: bit, or Raspberry PI are based on more straightforward block-based programming; therefore, they are suitable for elementary or secondary school students (Weng et al., 2022; Cui et al., 2021) and beginners (Major et al., 2021; Wang et al., 2021). For high school students with programming backgrounds or university students, robotic tools based on a text-based programming language (like Python or C++) are more common. Robots supported by text programming exercises can do much more - for example, move their arms and grip something (Wang et al., 2021), play sumo (Sisman et al., 2022) or act as drones for reconnaissance (Bhuyan et al., 2020). Among the studies examined was only one in which a motion-capable device had to be coded using block-based programming (Kim et al., 2021).

Programming in problem-based STEM education

In our review of the articles, we found three cases where students had to solve a STEM problem in a programming environment (it is important to note that we are discussing non-robotics cases in this chapter). In two of three cases, it was used Scratch, a block-based programming tool that solves problems like saving money and fractal drawing among middle school students (Weng et al., 2022) or social and ecological problems among elementary school students (Century et al., 2020). Python, a text-based programming tool, was used among university students to solve scientific problems.

Other types of digital devices used in problem-based STEM education

Four other types of digital devices appeared in the articles reviewed. A computer game about operations with Roman numbers was used among elementary school students to detect students gifted in Mathematics (Juric et al., 2021).

A computer agent helped junior high school students with a web-based collaborative problem-solving system. A computer agent is a computer-simulated participant that can develop goals, perform tasks, communicate messages, respond to other participants' messages, react to the environment, adapt to changes in the environment, and learn concurrently (Lin, KY et al., 2020).

Active learning (via cooperative problem-based learning and peer assessment) was implemented using an interactive online whiteboard in a first-year calculus class (Ng et al., 2020).

An innovative virtual reality (VR) based approach was tried among engineering students to understand and simulate manufacturing concepts (Dayarathna, 2020).

Discussion on RQ3

Automated tools are the most popular digital devices for problem-based STEM learning. It could be because robotics makes the results of programming immediately visible. For example, the "robot arm" moves where students code it (Wang et al., 2021), and the aerial drone flies and takes pictures where they program it (Bhuyan et al., 2020). On the other hand, the fact that the physical device is not working well (or at all) is immediate feedback of lousy coding. Students' motivation in the world of mobile robots can be enhanced through gamification, as seen in the sumo robot championship (Sisman et al., 2022) or the robot ball competition (Kim et al., 2021).

Robotic devices based on block-based programming, which is much simpler than text-based programming, are also standard. These allow younger students (primary and secondary school) or beginners in programming to succeed in robotics. We could see an example of this in Hooke's Law experiment (Major et al., 2021), in collecting and processing sensor data for robotics electronics integration (Wang et al., 2021), in building a room capacity detector and a thermometer (Cui et al., 2021), or in such a real-world problem, like helping hospitals prioritize patients (Shahin et al., 2021).

However, there are some cases where we cannot use robotics. On the one hand, robotics tools are pretty expensive; on the other hand, if a course is only delivered online (Weng et al., 2022), it is more appropriate to use programming alone. For beginners and young students, block-based programming is the best - Scratch is the most popular of these, based on the articles studied (Century et al., 2020; Weng et al., 2022). Nevertheless, for example, complex scientific phenomena can also be modeled by programming - usually using text-based programming languages, like Python (Lin, YT et al., 2020).

We saw some further examples of the use of digital tools in problem-based STEM learning. For example, a VR-based approach can be used to understand and learn engineering concepts regardless of gender (Dayarathna, 2020). However, the authors argue that a new type of cheaper virtual reality technology is needed because of its high price.

Digital tools such as an interactive online whiteboard (Ng et al., 2020) and a computer agent (Lin, KY et al., 2020) can significantly enhance student collaboration and help them understand subject content better. While in the former case, the device helps students work transparently and simultaneously between groups (Ng et al., 2020), in the latter case, it also supports learning (Lin, KY et al., 2020). Moreover, a digital tool can also be used to identify mathematical talent among primary school pupils based on a computer game (Juric et al., 2021).

In general, the role of digital tools in problem-based STEM learning is

a) it is a problem-solving tool (using the tool to solve the problem).

It is the case for all robotics projects, where the robot has to be "made" to do something (Major et al., 2021; Cui et al., 2021; Wang et al., 2021; Shahin et al., 2021; Sisman et al., 2022; Bhuyan et al., 2020; Kim et al., 2021). However, it also includes programming tasks (Weng et al., 2022; Century et al., 2020; Lin, YT et al., 2020), computer games (Juric et al., 2021), and virtual reality (Darayathna et al., 2020).

b) helps with collaboration

It is either achieved by having to solve the task together, for example, in robotics cases (Major et al., 2021; Cui et al., 2021; Wang et al., 2021; Shahin et al., 2021; Sisman et al., 2022; Bhuyan et al., 2020; Kim et al., 2021) and the science simulation programming task (Lin, YT et al., 2020). Or it can be implemented by making the simultaneous work transparent and assessable, as in the case of the interactive online whiteboard (Ng et al., 2020)

c) supports learning (provides an environment that helps understand the subject content)

We saw examples of this in the use of computer games (Juric et al., 2021), computer agents (Lin, KY et al., 2020), and virtual reality (Dayarathna et al., 2020).

General characteristics of RQ4 (the type of problems)

In this chapter of the study, we want to answer the question of what problems the students had to solve during the STEM education supported by PbL-based digital tools. We grouped the cases into three broad categories based on the problems found in the articles reviewed. These large groups were: real-world problems (15 problems in 7 articles), subject-specific problems (8 problems in 7 articles), and robot operations (3 problems in 3 articles). Within the categories, we also separated subcategories for clarity. In this way, we separated the socially relevant problems, building or designing problems, and financial management problems within real-world problems. Most of the subject-specific problems were related to mathematics. However, we also found one example that was related to biology and two that were related to physics. Two of the three robot operation problems were based on gamification, and one was embedded in project-based learning. A summary of the general characteristics of RQ4 is shown in Table 4.

Table 4: The types of problems appeared in PbL STEM using by some kind of digital devices

Real-world problems

Real-world problems are prevalent in digital PBL-based STEM education. These include problems arising from everyday difficulties. For example, in a study by Bhuyan et al. (2020), students are looking for solutions to socially fundamental problems such as finding landfills or finding land suitable for growing crops. Moreover, this project involved students from minority, disadvantaged socio-economic backgrounds. The students in Shahin et al.'s (2021) study had to approach it from a different perspective, also looking for solutions to socially significant problems. They took on an entrepreneurial role in trying to offer solutions to hospitals in prioritizing patients. Lin, KY et al. (2020) and Cui et al. (2021) write in their paper about several problems related to design and creation, like making a thermometer or designing a room capacity detector.

Responsible financial management is a real-world issue. Within this, we encountered problems such as saving money (Weng et al., 2022), bank deposits (Cui et al., 2021), responsible shopping (Lin, KY et al., 2020), or business working (Century et al., 2020).

Subject-specific problems

Here we included problems related to the learning of subject content. Most of them have a mathematical focus, like Roman numbers among elementary school students (Juric et al., 2021), fractal drawing (Weng et al., 2022), and prime and composite number detectors (Cui et al., 2021) among secondary school students. Furthermore, the students' digital STEM PbL exercises were based on topics such as solving various calculus problems (Ng et al., 2020) and queuing theory (Dayarathna et al., 2020).

Technical high school students tested Hooke's law in Major et al. (2021) study. Lin et al. (2021) report on one-dimensional kinetic problem solving by science students. Also, the last paper reports on one-dimensional kinematic problem-solving.

Robot operation problems

First, we would like to clarify that studies, where robotics was merely a tool to achieve a goal were classified in one of the upper categories. In this section, we have only included studies where the ultimate goal of the problem-solving was a particular movement of the robot. Two of these three studies were based on gamification, such as the sumo robot tournament (Sisman et al., 2022) and robot ball path planning (Kim et al., 2021). Moreover, Wang et al. (2021) write about a robot design and construction project among undergraduate students.

Discussion on RQ4

Schools are often criticized for having little to do with everyday life. Nevertheless, exposing pupils to real-world problems is essential as early as possible. Reflecting on socially relevant problems can lead to a change in attitudes towards social engagement, as seen in Bhuyan et al. (2020) study. In addition, it is also imperative for students to understand how, for example, a socially relevant problem can be used to build an enterprise (Shahin et al., 2021). However, not only future entrepreneurs but also engineers and designers are now in school, so various creative tasks (Cui et al., 2021; Lin, KY et al., 2020) and design tasks (Lin, KY et al., 2020) are also of great importance.

Good money management is part of responsible living. Unfortunately, in Hungary, students in public education have little (if any) exposure to money management. However, the articles reviewed provide good examples of money management (Weng et al., 2022; Cui et al., 2021), prudent purchasing (Lin, KY et al., 2020), and the functioning of business (Century et al., 2020).

Based on the reviewed articles, mathematics subject-specific content seems the most difficult to embed in an integrated STEM problem. While all digital PbL STEM studies on social, environmental, or even robotics have mathematics as a tool in the background, the learning of mathematical content is most often done through subject-specific mathematical problem solving (Ng et al., 2020; Cui et al., 2021; Weng et al., 2022). However, digital tools can help to provide a more pleasant user experience for this learning (Dayarathna et al., 2020).

Problem-solving on how to operate robots fosters collaboration between students. These situations also create a natural working environment by jointly operating the robot while solving various problems and resolving conflicts (Wang et al., 2021). At the same time, robotics can be used to develop a deeper understanding of different subject content, as was the case with the topic of angle pairs, which was aided by designing the path of the robot ball (Kim et al., 2021). Furthermore, the competitive and gamification environment can motivate students to operate the robots more (Kim et al., 2021; Sisman et al., 2022).

A significant finding of Century et al.'s (2020) study on problem-based STEM education supported by digital tools. In a comprehensive, longitudinal study, the authors found that this type of learning performs as well as traditional learning in the subject content. In addition, they develop a range of skills, an integrated approach to science, learn about social problems and develop a different work ethic.

Conclusion

This study presented a systematic literature review revealing the current trends and developments in applying digital tools and implementing environmental awareness and sustainability in PBL-based STEM education. We examined 377 studies published in PBL STEM education from 2020 to 2022. We focused on the types and roles of digital tools, the types of problems solved with digital tools, the types of tools and methods used for environmental awareness and sustainability, and the critical factors for successful implementations.

Our study showed that besides entirely empirical studies, in some particular cases, the general testing, development, and calibration of specific frameworks, methods, or tools can also largely contribute to the effectiveness of the implementation of new environmental awareness- or sustainability-related topics and problems. We found that the most critical factors to success were partly systematic (clearly set goals and policies, state-of-the-art tools and instruments for development, effective assessment system) and partly (inter)personal (role of communities and horizontal networks, lesson planning of the teachers). The results of the SWOT analysis of the implementation of PBL (based on the articles of this study) were consistent with previous ones concerning PBL itself. It also showed that PBL has immense opportunities to help the schools and teachers develop the learners' soft skills, metacognitive skills, and met practical functions.

In our research, we also concluded that problem-based STEM learning in the last two years is most often conducted in a digital environment or with the help of a digital tool. Robotics and programming are the most popular of these. We also found that PBL supported by digital tools is more effective in collaboration than the non-digital version. The digital environment is not only helpful in improving attitudes towards computers or in promoting computational thinking. Nevertheless, it is also a suitable environment for solving problems relevant to everyday life - be it a socially relevant problem, a design-creation task, or even money management. One of the key findings of our research is that digital problem-based STEM learning does not continuously improve students' subject-specific knowledge, but it does not make it worse. However, it does develop skills and abilities such as collaboration, work ethic, creativity, critical thinking, and communication.

References

·     Araya, R.; Collanqui, P. (2021). Are Cross-Border Classes Feasible for Students to Collaborate in the Analysis of Energy Efficiency Strategies for Socioeconomic Development While Keeping CO2 Concentration Controlled? Sustainability 2021, 13, 1584. https://www.mdpi.com/2071-1050/13/3/1584

·     Azaceta, N. R., Malagon, D. E., de Gordoa, G. J. G. R., & Jimeno, N. D. (2018). SWOT analysis of a Collaborative Project-Based Learning (CPBL) methodology in higher education, and a proposal of activities for continuous improvement. In Edulearn 18. 10th International Conference on Education and New Learning Technology:(Palma, 2nd-4th of July, 2018). Conference proceedings (pp. 777-786). IATED Academy.

·     Barrows, H. S., and Tamblyn, R. (1980). Problem-Based Learning: An Approach to Medical Education, Springer, New York.

·     Bhuyan, J., Wu, F., Thomas, C. Koong, K., Hur, JW. & Wang, CH. (2020). Aerial Drone: an Effective Tool to Teach Information Technology and Cybersecurity through Project Based Learning to Minority High School Students in the U.S. Techtrends 64(6), 899-910. DOI 10.1007/s11528-020-00502-7

·     Bybee, R. W., Taylor, J. A., Gardner, A., Van Scotter, P., Powell, J. C., Westbrook, A., & Landes, N. (2006). The BSCS 5E instructional model: Origins and effectiveness. Colorado Springs, Co: BSCS, 5, 88-98.

·     Century, J., Ferris, KA & Zuo, HF (2020). Finding time for computer science in the elementary school day: a quasi-experimental study of a transdisciplinary problem-based learning approach. International Journal of STEM Education 7(1), a.n.20. DOI10.1186/s40594-020-00218-3

·     Clark, B., & Button, C. (2011). Sustainability transdisciplinary education model: Interface of arts, science, and community (STEM). International Journal of Sustainability in Higher Education.

·     Collins, A., Brown, J. S., and Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In Resnick, L. B. (ed.), Knowing, Learning, and Instruction: Essays in Honor of Robert Glaser, Erlbaum, Hillsdale, NJ, pp. 453–494.

·     Cui, ZH. & Ng, OL. (2021). The Interplay Between Mathematical and Computational Thinking in Primary School Students' Mathematical Problem-Solving Within a Programming Environment. Journal Of Educational Computing Research 59(5), 988-1012. DOI10.1177/0735633120979930

·     Dayarathna, VL., Karam, S., Jaradat, R., Hamilton, MA., Nagahi, M., Joshi, S., Ma, JF., Ashour, O. & Driouche, B. (2020). Assessment of the Efficacy and Effectiveness of Virtual Reality Teaching Module: A Gender-Based Comparison. International Journal of Engineering Education 36(6), 1938-1955.

·     Dotson ME, Alvarez V, Tackett M, Asturias G, Leon I and Ramanujam N (2020). Design Thinking-Based STEM Learning: Preliminary Results on Achieving Scale and Sustainability Through the IGNITE Model. Front. Educ. 5:14. doi:10.3389/feduc.2020.00014

·     Erikson, M. G., & Erikson, M. (2019). Learning outcomes and critical thinking–good intentions in conflict. Studies in Higher Education, 44(12), 2293-2303.

·     Gallay E, Flanagan C and Parker B (2021). Place-Based Environmental Civic Science: Urban Students Using STEM for Public Good. Front. Educ. 6:693455. doi:10.3389/feduc.2021.693455

·     Hamilton, E. R., Rosenberg, J. M., & Akcaoglu, M. (2016). The substitution augmentation modification redefinition (SAMR) model: A critical review and suggestions for its use. TechTrends, 60(5), 433-441.

·     Han, S., Capraro, R. & Capraro, M.M.(2015). How Science, Technology, Engineering and Mathematics (STEM) Project-based Learning (PBL) Affects High, Middle and Low Achievers Differently: The Impact of Student Factors on Achievement. International Journal of Science and Mathematic Education 13, 1089–1113. https://doi.org/10.1007/s10763-014-9526-0

·     Hmelo, C. E., and Ferrari, M. (1997). The problem-based learning tutorial: Cultivating higherorder thinking skills. J. Educ. Gifted 20, 401–422

·     Hmelo-Silver, CE. (2004). Problem-Based Learning: What and How Do Students Learn? Educational Psychology Review, 16(3), 235-266.

·     Honey, M., Pearson, G., & Schweingruber, H. (2014). STEM Integration in K-12 Education: Status, Prospects, and an Agenda for Research (pdf version). https://doi.org/10.17226/18612.

·     Juric, P., Bakaric, MB. & Matetic M. (2021). Cognitive Predispositions of Students for STEM Success and Differences in Solving Problems in the Computer Game for Learning Mathematics. International Journal of Engineering Pedagogy 11(4), 81-95. DOI10.3991/ijep.v11i4.20587

·     Kim, YR., Park, MS. & Tjoe, H. (2021). Discovering Concepts of Geometry through Robotics Coding Activities. International Journal of Education in Mathematics Science and Technology 9(3), 406-425. DOI10.46328/ijemst.1205

·     Kolodner, J. L., Hmelo, C. E., and Narayanan, N. H. (1996). Problem-based learning meets case-based reasoning. In Edelson, D. C., and Domeshek, E. A. (eds.), Proceedings of ICLS 96, AACE, Charlottesville, VA, 188–195.

·     LaForce, M., Noble, M. & Blackwell, C. (2017). Problem-Based Learning (PBL) and Student Interest in STEM Careers: The Roles of Motivation and Ability Beliefs. Educ. Sci. 7(4), 92 https://doi.org/10.3390/educsci7040092

·     Lin, KY., Hsiao, HS., Chang, YS. & Chien, YH. (2020). Effects of web-based versus classroom-based STEM learning environments on the development of collaborative problem-solving skills in junior high school students. International Journal of Technology and Design Education 30(1), 21-34. DOI10.1007/s10798-018-9488-6; https://link.springer.com/article/10.1007/s10798-018-9488-6

·     Lin, YT., Yeh, MKC. & Hsieh, HL. (2020). Teaching computer programming to science majors by modeling. Computer Application in Engineering Education 29(1), 130-144. DOI10.1002/cae.22247

·     Maass, K., Sorge, S., Romero-Ariza, M., Hesse, A., & Straser, O. (2022). Promoting Active Citizenship in Mathematics and Science Teaching. International journal of science and mathematics education, 20(4), 727-746.

·     Major, S., Hubalovska, M., & Waclawek, M. (2021). Using the Raspberry PI Microcomputers in STEM Education in Technically Oriented High Schools. Chemistry-Didactics-Ecology-Metrology 26(1-2), 73-88. DOI10.2478/cdem-2021-0006

·     Moore, T. J., Stohlmann, M. S., Wang, H. H., Tank, K. M., Glancy, A., & Roehrig, G. H. (2014). Implementation and integration of engineering in K-12 STEM education. In J. Strobel, S. Purzer, & M. Cardella (Eds.), Engineering in precollege settings: research into practice. Rotterdam: Sense Publishers.

·     Nantsou, T. P., & Tombras, G. S. (2022, March). STEM Lab on Climate Change with Simple Hands-on Experiments. In 2022 IEEE Global Engineering Education Conference (EDUCON) (pp. 249-256). IEEE.

·     Nawi, N. D., Phang, F. A., Mohd-Yusof, K., Rahman, N. F. A., Zakaria, Z. Y., bin Syed Hassan, S. A. H., & Musa, A. N. (2019). Instilling Low Carbon Awareness through Technology-Enhanced Cooperative Problem Based Learning. International Journal of Emerging Technologies in Learning, 14(24).

·     Ng, OL, Ting, F., & Lam, WH., Liu, M. (2020). Active Learning in Undergraduate Mathematics Tutorials Via Cooperative Problem-Based Learning and Peer Assessment with Interactive Online Whiteboards. Asia-Pacific Education Researcher 29(3), 285-294. DOI10.1007/s40299-019-00481-1

·     Parno, L, Yuliati, L. & Ni'mah, BQA (2019). The influence of PBL-STEM on students' problem-solving skills in the topic of optical instruments. Journal of Physics: Conference Series 1171 a.n.:012013.

·     Patrick, P. G., Bryan, W., & Matteson, S. M. (2020). Applying the Strengths, Weaknesses, Opportunities, and Threats (SWOT) Framework During a Community College Chemistry Project-Based Learning Activity. Journal of College Science Teaching, 50(2), 6-13.

·     Rico, A.; Agirre-Basurko, E.; Ruiz-González, A.; Palacios-Agundez, I. & Zuazagoitia, D. (2021). Integrating Mathematics and Science Teaching in the Context of Education for Sustainable Development: Design and Pilot Implementation of a Teaching-Learning Sequence about Air Quality with Pre-Service Primary Teachers. Sustainability 2021, 13, 4500. https://www.mdpi.com/2071-1050/13/8/4500

·     Rogers, M., Pfaff, T., Hamilton, J., & Erkan, A. (2015). Using sustainability themes and multidisciplinary approaches to enhance STEM education. International Journal of Sustainability in Higher Education.

·     Rossano, V., & Calvano, G. (2020). Promoting Sustainable Behavior Using Serious Games: SeAdventure for Ocean Literacy. IEEE Access, 8, 196931-196939.

·     Shahin, M., Ilic, O., Gonsalvez, C., & Whittle, J. (2021). The impact of a STEM-based entrepreneurship program on the entrepreneurial intention of secondary school female students. International Entrepreneurship and Management Journal, 17(4), 1867-1898. DOI10.1007/s11365-020-00713-7

·     Shay, J. E., Solis, R. & García-Ojeda, M. E. (2020). Tackling Real-World Environmental Paper Pollution: A Problem-Based Microbiology Lesson About Carbon Assimilation. Front. Microbiol. 11:588918. DOI10.3389/fmicb.2020.588918

·     Sisman, B., Kucuk, S. & Ozcan, N. (2022). Collaborative behavioral patterns of elementary school students working on a robotics project. Journal of Computer Assisted Learning. DOI10.1111/jcal.12659

·     Smith, C., & Watson, J. (2019). Does the rise of STEM education mean the demise of sustainability education? Australian Journal of Environmental Education, 35(1), 1-11.

·     Snell‐Rood, E. C., Smirnoff, D., Cantrell, H., Chapman, K., Kirscht, E., & Stretch, E. (2021). Bioinspiration as a method of problem‐based STEM education: A case study with a class structured around the COVID‐19 crisis. Ecology and evolution, 11(23), 16374-16386.

·     Songer, N. B. and Ibarrola Recalde, G. D. (2021) Eco-Solutioning: The Design and Evaluation of a Curricular Unit to Foster Students’ Creation of Solutions to Address Local SocioScientific Issues. Front. Educ. 6:642320. DOI10.3389/feduc.2021.642320

·     Talley, T. S., Ruzic, R., McKay, L. G., Venuti, N., & Mothokakobo, R. (2021). Using Community Science to Address Pollution in an Urban Watershed: Lessons about Trash, Diverse Engagement, and the Need for Science Mindsets. Journal of Contemporary Water Research & Education, 174(1), 21-44.

·     Yahaya, J., Fadzli, S., Deraman, A., Yahaya, N. Z., Halim, L., Rais, I. A. I., & Ibrahim, S. R. A. (2021). PRInK: Environmental virtual interactive based education and learning model for STEM motivation. Education and Information Technologies, 1-21.

·     Wang, WT., Coutras, C. & Zhu, M. (2021). Empowering computing students with proficiency in robotics via situated learning. Smart Learning Environments 8(1), a.n.:24. DOI10.1186/s40561-021-00167-6

·     Weng, XJ, Cui, ZH, Ng, OL, Jong, MSY & Chiu, TKF (2022). Characterizing Students' 4C Skills Development During Problem-based Digital Making. Journal of Science Education and Technology 31, 372-385. DOI10.1007/s10956-022-09961-4.

[1] Henrietta Molnár Muzsalyiné (1ab), András Horváth (1ab), Sándor Lénárd (1ab), István Nahalka (1b) Katalin Radnóti (2)

(1a) School Organization, Teacher Education and Professional Learning Research Group (STeP.RG),

(1b) Institute of Education, Faculty of Education and Psychology, Eötvös Loránd University

(  2) Department of Materials Physics, Faculty of Science, Eötvös Loránd University