MANOJLOVICH Helena

Escape Rooms and Collaborative Problem-Solving: Examining the Competence of Teacher Candidates

Introduction

The impact of technological advancements on the way people works and interact with each other is significant, especially as knowledge becomes increasingly specialized. To prepare individuals for 21st-century employment, it is imperative that school curricula encompass the necessary competencies. These competencies include mindsets, common ways of working, methods to utilize new tools, and guidance for productive living (Care et al., 2012). They encompass a range of skills such as creativity, critical thinking, problem-solving, decision-making, communication, collaboration, information, and communication technology (ICT) and information literacy, life, and career, as well as personal and social responsibility.

Collaborative problem-solving has emerged as a core competence of the knowledge-based economy and has been a focus of recent theoretical and technological developments in educational research (Awwal et al., 2017). This field of research is relatively new, and its concepts, methods, and research ideas combine collaborative learning, problem-solving, and psychometry (Scardamalia et al., 2010).

The essence of collaborative problem-solving is the integration of critical thinking, problem-solving, communication, and collaboration (Griffin & Care, 2015). It involves a shared activity where group members work together to solve different tasks and achieve a common goal. The individuals within a group may require varying knowledge, expertise, and a range of soft skills, both in terms of interpersonal dynamics and cognitive processes. The key aspect is that the task's goal can only be achieved through collaboration. When competencies require such complexity, tasks designed to measure the construct can be challenging.

Collaborative problem-solving skills

Collaborative problem solving consists of two main components, a collaborative or social element and a problem-solving or cognitive element.

The collaborative or social component refers to the group work activity that takes place to achieve common goals (Hesse et al., 2015). This competence takes part in the implementation of social interactions (Nagy, 1998) and helps the individual to understand the perspective of others, adapt to new conditions, learn from experiences, and apply knowledge in other situations (Semrud-Clikeman, 2007).

The problem-solving or cognitive component is a cognitive process that tries to transform a given situation into a target situation, so that the problem-solving person does not have an obvious method to reach the solution (Funke & Frensch, 2007).

The measurement of collaborative competencies consists of observing, recording, and summarizing the complex behaviour of individuals and groups, from which conclusions can be drawn about learning and behavioural processes (Awwal et al., 2015). Collaborative problem-solving combines problem-solving thinking, communication, and cooperation (Griffin & Care, 2015).

To date, only a few structured frameworks are available for evaluating collaborative problem-solving competence (Mughal & Shaikh, 2018). One of these is the framework of the ATC21S project, which, with the participation of 250 researchers from all over the world and based on the literature of several research fields (cognitive science, education, social psychology, and psycholinguistics), developed a framework consisting of a hierarchy of sub-skills. These sub-skills play a key role in collaborative problem-solving.

The ATC21S framework links critical thinking, problem-solving, decision-making, and collaboration through three social (participation, perspective-taking, and social regulation) and two cognitive (task regulation and knowledge construction) skills (Hesse et al., 2015). Effective problem-solving does not rely on uniform skills, but on distinguishable sub-skills, some of which are applied according to situational needs (Care et al., 2015).

Fig. 1. Describing skills and subskills from the ATC21S CPS framework

Pöysä-Tarhonen et al., 2018) modified from (Hesse et al., 2015)

These two assumed components of collaborative problem-solving are not mutually exclusive. The social component relies on literature from social and organizational psychology, while the cognitive component relies heavily on classical approaches to individual problem-solving (Hesse et al., 2015).

The existence of these sub-competencies at a high level of development is the prerequisite for successful cooperative problem-solving. Assessment relies on and captures these activities or processes so that the measurement reflects the construct.

The objective of the framework is to evaluate the performance indicators of the collaborative problem-solving competence of the individual and the group, in collaborative situations, in our case in the educational escape room environment. The measurement of competence consists of observing, recording, and summarizing the complex behaviour of the individual and the group, from which reasonable conclusions can be drawn about the problem-solving and cooperation processes (Awwal et al., 2015).

Educational escape rooms

In a physical escape room game, participants are most often locked in one (or more) room and tasked with solving puzzles within a predetermined time limit. The goal of the game can be to get out of the room, find an object, free a hostage, etc. (Nicholson, 2015). Many studies have shown that escape rooms provide people with opportunities to practice a wide range of collaborative skills (Borrego et al., 2017; Clapson et al., 2020; Escribano, 2018; Pan et al., 2017). Clues and hints for escape room puzzles are often hidden in boxes sealed with a combination or key lock that must be opened in a predetermined order. Cognitive puzzles are prevalent in the game (Nicholson, 2015). These can be coded messages, combination locks, ciphers, logic puzzles and other thought-provoking puzzles (Eukel & Morrell, 2021). The rooms are decorated with various objects and decorations, which are often part of the puzzles. It is important that they fit the theme and narrative of the game (Clarke et al., 2017).

The game is suitable for developing many skills, such as collaboration, communication, and logic, as well as critical thinking, search, observation, reasoning, pattern recognition, problem-solving, creativity, application of knowledge and coping with time pressure (Pan et al., 2017; Wiemker et al., 2015). The escape rooms are also interesting in that they offer researchers and designers the opportunity to explore a range of social and technical research questions. They are also suitable for exploring research questions related to communication and collaboration skills (Pan et al., 2017).

Game-based learning has become increasingly popular in education (von Kotzebue et al., 2022). It is true that the spare rooms were primarily developed for commercial and recreational purposes, but teachers quickly realized their advantages that can be used in education as well. In the last few years, a lot of research has been published in the field of educational escape rooms, although most of them are exploratory case studies.

In our research, the students participated in a self-developed escape room game for free-time educational purposes. The goal of the game is to open the money box, which must be completed in 60 minutes. The game encompasses a wide spectrum of escape room puzzles. It does not contain puzzles aimed at teaching or practising educational material, the focus is solely on the observation and measurement of collaborative problem-solving competence and the related sub-competencies.

Fig. 2. The escape room set up in one of the halls of the Faculty of Economic and Social Sciences (BME), Department of Technical Pedagogy, in Budapest, Hungary

Our framework for creating educational escape rooms – SmarTeacheRoom

To date, several frameworks have been published for designing educational escape rooms that include step-by-step procedures for game construction (Clarke et al., 2017; Guigon et al., 2018; Snyder, 2018). Frameworks provide guidance for the design of physical rooms. However, some teachers prefer the development of more cost-effective and more accessible digital escape rooms, since there are many digital tools available that can be used to create educational games in a hybrid or online environment (Kroski, 2020).

Relying on the frameworks that describe the structure of existing escape rooms, we developed our own integrated model that combines and complements different approaches.

We had to create a model that shows that activities aimed at solving the problem cannot always be completed without the use of social skills. There are also activities that promote the functioning of the group in a collaborative problem-solving process, which do not require the cognitive abilities necessary for specific problem-solving. In the model, the focus is on directly observable activities, since our observations always refer to behavioral elements, we can only assume the abilities and processes behind the activity.

The framework supports the way of building and evaluating the escape room for educational purposes, as well as the direct observation of problem-solving groups, based on which we want to get an answer to how individuals solve problems and how they cooperated with their group mates.

Incorporating elements such as objectives, participants, context, and evaluation allows for a game-like performance of tasks in the educational environment.

Fig. 3. Theoretical framework (SmarTeacheRoom - STR)

Methodology

Goal

The aim of study was to explore the methodological possibilities of using educational escape rooms to measure collaborative problem-solving competence among students in higher education. The students were observed in the self-developed educational escape room environment, and student behaviour was evaluated based on the observation system developed within the framework of the ATC21S project. The observation criteria system fits perfectly with the competence construct we want to measure in our research: the evaluation of collaborative problem solving, which means working together with others, including the sharing and exchange of ideas, knowledge, or resources to achieve a common goal. The code data obtained from the video analysis were organized into a combined scoring Guttman diagram to allow a visual representation of the scoring.

Specifically, study focused on students from the Faculty of Economic and Social Sciences (BME), Department of Technical Pedagogy, in Budapest, Hungary. The escape room game consisted of a variety of puzzles, and we used a camera to monitor the progress of the groups, which consisted of 4 to 5 participants. After the game, we conducted focus group interviews with the participants and administered four questionnaires and a test to measure various aspects of their collaborative problem-solving competence. These included the Scrambled Adaptive Matrices (SAM) (Klein et al., 2018), Big Five (Caprara et al., 1993), Teamwork Skills Questionnaire (O’Neil et al., 1999), Tóth-féle Kreativitás Becslő Skála (TKBS) (Tóth, 2006), and a background questionnaire.

The main question of the research is whether the escape room method is suitable for measuring collaborative problem-solving competence, and what characterizes the collaborative problem-solving competencies of students.

In this paper, we focused on the qualitative results of the study.

Approach

In our research, content analysis is carried out through video and audio analysis. During the escape room game, student groups were directly observed, and video recordings were made of their activities in the room. One way to review video data is through coding. Coding is an inherently theoretical process and is related to observational practices. We conducted systematic coding using a pre-developed theoretical approach to coding. The observational criteria perfectly match the competence construct we aim to measure: collaborative problem-solving, which involves working with others, including sharing, and exchanging ideas, knowledge, or resources, to achieve a common goal. The use of technical tools (cameras and computers) provides advantages in terms of objectivity, reliability, and reproducibility. The results of the observed behaviour are compared with the results of the interviews, questionnaires, and tests, ensuring data triangulation. The code data obtained from video analysis were arranged in several merged scoring Guttmann diagrams to allow for visual representation of scoring. The Guttmann diagram arranges the performance of students according to their competence level, and the assessment items are arranged according to their level of difficulty (Guttman, 1974).

The analysis of video data was facilitated by the Dedoose software. The research is based on the mixed-methods approach, and this software is specifically designed for analyzing data from mixed-methods research. The application, which is used for qualitative data analysis, facilitates coding and integration with demographic and other quantitative data.

Evaluation

In research the video data were analysed using the Dedoose software. The research is based on a mixed methodology, for the analysis of which the software is especially suitable. Dedoose is a SaaS application, which means that you only need access to log in through the program's website, although it is recommended to use the desktop application, which does not require an internet connection to work.

The software supports the import of data (text, image, video, sound, Excel tables) and provides the possibility to create, delete and modify variables and code data. It stores the media contents (Media) as text, image, video, or sound files, while the descriptive variables (Descriptors) are stored in tabular format. Variables and media content can be linked (Linked Media) and details can be selected within the media, which are treated as excerpts (Excerpts). Extracts (video, image, audio, or text) are created by assigning codes and code weights. All these objects can be searched, filtered, sorted, and analysed in Dedoose.

The limitation of the software was that it is not possible to assign multiple participants to a video segment. This means that only one participant (or group) can be linked to a video extract. Therefore, e.g., the video material related to the first group was combined with the BME1 group, and the coding results were evaluated at the group level. The collaborative problem-solving competence evaluation framework allows for manual coding, so after the group analysis, we separately evaluated each student's sub-competence level in an Excel table. With the help of the framework, the level of individual collaborative problem-solving competence can be determined in the educational escape rooms, where we evaluate the individuals based on the data recorded through video recordings and observation.

Fig. 4. Manually coding the level of individual sub-competencies – the part of the table

Results

As mentioned earlier, the code data from the video analysis were organized into several combined scoring Guttmann charts to allow a visual representation of the scoring. We have created several statements: group and individual level.

The columns of the Guttmann diagram represent the scores for the individual sub-competencies, while the rows show the scores of the groups and students. The order of sub-competencies from left to right places the sub-competency with the highest score first, indicating that this sub-competency is the most developed among the participants and their groups. On the right side, we can see that, for example, group-level self-evaluation (group evaluation) was not present during the game, and the groups did not even evaluate themselves during the interview. The rows show the total scores of groups and individuals from the highest to the lowest, from top to bottom. So, the best-performing group and individual is at the top of the table.

The diagram is easy to analyse because it is easy to read who has which sub-competence, and which sub-competences should be focused on developing in the case of individual groups and students.

Fig. 5. Results of qualitative data analysis - at the level of the individual – the part of the table

Fig. 6. Results of qualitative data analysis - at the group level

Results indicate that the educational escape room can be used to evaluate collaborative problem-solving competence, as the observed student behaviour during the game can be used to assess the sub-competencies of the observation framework. Based on Gutteman diagrams, more successful groups demonstrated outstanding performance both in social and cognitive competencies. Most of the cognitive elements were only observed at a high level in the three best-performing groups during the game. Hypothesis formation, responsibility initiative, cause and effects, systematicity, relationships, problem analysis, flexibility and actions were the key competencies that distinguished the best performers from all other groups.

The results will be compared with the results of interviews, questionnaires and tests, which will ensure data triangulation.

Summary

This research is pioneering because there are no similar studies to date. Many studies have been published on educational escape rooms, but very few are empirical in nature. Most studies are case studies. Other studies aim to demonstrate the positive impact of escape rooms on learning outcomes through pre- and post-tests and describe the opportunities for using the game in a pedagogical context. Similarly, we have not found any detailed video analysis, and the Dedoose software has not been used in previous studies on educational escape rooms.

Existing experience and research results also support that the escape room for educational purposes can be a tool for any teaching area, as it is easy to apply, student-centered, promotes research thinking, logical and critical thinking (Aubeux et al., 2020; Eukel et al., 2020). Study provides insights into the methodological possibilities of using educational escape rooms to measure collaborative problem-solving competence among students in higher education. We found that our educational escape room game was effective in observing and measuring collaborative problem-solving competences, and we identified several factors that characterized the collaborative problem-solving competence of students from the Faculty of Economic and Social Sciences (BME), Department of Technical Pedagogy, in Budapest, Hungary. Our findings have implications for the design of future educational escape room games and for the development of collaborative problem-solving skills among students in higher education. In conclusion, the use of educational escape rooms allows teachers to measure the current level of their students' collaboration-based problem-solving skills. A well-designed game flow provides an immersive experience that requires full dedication and focus on the task at hand, which presents a challenge for the student. The flow experience makes individuals feel productive and capable of performing at their peak. When the activity involves appropriate challenges and is attractive, it generally contributes to the development of student competencies. As a result, the use of escape rooms in an educational context has been shown to facilitate the strengthening and development of numerous key competencies. This method can significantly contribute to the development of pedagogical work and contribute to its positive improvement.

References

Aubeux, D., Blanchflower, N., Bray, E., Clouet, R., Remaud, M., Badran, Z., Prud’homme, T., & Gaudin, A. (2020). Educational gaming for dental students: Design and assessment of a pilot endodontic-themed escape game. European Journal of Dental Education, 24(3), 449–457. https://doi.org/10.1111/eje.12521

Awwal, N., Alom, M., & Care, E. (2015). Game design for learning to solve problems in a collaborative environment. European Conference on Games Based Learning, 25–34.

Awwal, N., Griffin, P., Zhang, Z., Scoular, C., Jimenez, D., & Wilson, M. (2017). Validation of Collaborative Problem Solving Process Framework from Evidence of Student Observations for Developing Generic Measures. Workshop Proceedings of the 25th International Conference on Computers in Education, 12.

Borrego, C., Fernández, C., Blanes, I., & Robles, S. (2017). Room escape at class: Escape games activities to facilitate the motivation and learning in computer science. Journal of Technology and Science Education, 7(2), 162. https://doi.org/10.3926/jotse.247

Caprara, G. V., Barbaranelli, C., Borgogni, L., & Perugini, M. (1993). The “big five questionnaire”: A new questionnaire to assess the five factor model. Personality and Individual Differences, 15(3), 281–288. https://doi.org/10.1016/0191-8869(93)90218-R

Care, E., Griffin, P., Scoular, C., Awwal, N., & Zoanetti, N. (2015). Collaborative Problem Solving Tasks. In P. Griffin & E. Care (Eds.), Assessment and Teaching of 21st Century Skills: Methods and Approach (pp. 85–104). Springer. https://doi.org/10.1007/978-94-017-9395-7_4

Care, E., Griffin, P., Woods, K., & Mountain, R. (2012). Defining and assessing 21st century skills. ATC21S. http://www.atc21s.org/uploads/3/7/0/0/37007163/pd_module_2_for_web.pdf

Clapson, M. L., Gilbert, B., Mozol, V. J., Schechtel, S., Tran, J., & White, S. (2020). ChemEscape: Educational Battle Box Puzzle Activities for Engaging Outreach and Active Learning in General Chemistry. Journal of Chemical Education, 97(1), 125–131. https://doi.org/10.1021/acs.jchemed.9b00612

Clarke, S. J., Peel, D. J., Arnab, S., Morini, L., Keegan, H., & Wood, O. (2017). EscapED: A Framework for Creating Educational Escape Rooms and Interactive Games to For Higher/Further Education. International Journal of Serious Games, 4(3), 73–86. https://doi.org/10.17083/ijsg.v4i3.180

Escribano, G. D. (2018). Escape rooms from a collaborative and interdisciplinery perpective. Encuentro: Revista de Investigación e Innovación En La Clase de Idiomas, 27, 22–37.

Eukel, H., Frenzel, J., Frazier, K., & Miller, M. (2020). Unlocking Student Engagement: Creation, Adaptation, and Application of an Educational Escape Room Across Three Pharmacy Campuses. Simulation & Gaming, 51, 104687811989850. https://doi.org/10.1177/1046878119898509

Eukel, H., & Morrell, B. (2021). Ensuring Educational Escape-Room Success: The Process of Designing, Piloting, Evaluating, Redesigning, and Re-Evaluating Educational Escape Rooms. Simulation & Gaming, 52(1), 18–23. https://doi.org/10.1177/1046878120953453

Funke, J., & Frensch, P. (2007). Complex Problem Solving: The European Perspective—10 Years After. In Learning to solve complex scientific problems (pp. 25–48). Routledge. https://www.taylorfrancis.com/chapters/edit/10.4324/9781315091938-2/complex-problem-solving-european-perspective%E2%80%9410-years-joachim-funke-peter-frensch

Griffin, P., & Care, E. (Eds.). (2015). Assessment and Teaching of 21st Century Skills. Springer. https://doi.org/10.1007/978-94-017-9395-7

Guigon, G., Humeau, J., & Vermeulen, M. (2018). A Model to Design Learning Escape Games: SEGAM. Proceedings of the 10th International Conference on Computer Supported Education, 191–197. https://doi.org/10.5220/0006665501910197

Guttman, L. (1974). The basis for scalogram analysis. In G. Maranell (Ed.), Scaling: A Sourcebook for Behavioral Scientists (1st Edition, pp. 60–90). Princeton University Press. https://cir.nii.ac.jp/crid/1570291226422491648

Hesse, F., Care, E., Buder, J., Sassenberg, K., & Griffin, P. (2015). A Framework for Teachable Collaborative Problem Solving Skills. In Assessment and teaching of 21st century skills educational assessment in an information age (pp. 37–56). https://doi.org/10.1007/978-94-017-9395-7_2

Klein, B., Raven, J., & Fodor, S. (2018). Scrambled Adaptive Matrices (SAM) – a new test of eductive ability. Psychological Test and Assessment Modeling, 60, 441–482.

Kroski, E. (2020). Chapter 1. What Is a Digital Breakout Game? Library Technology Reports, 56(3), Article 3.

Mughal, A. M., & Shaikh, S. H. (2018). Assessment of collaborative problem solving skills in Undergraduate Medical Students at Ziauddin College of Medicine, Karachi. Pakistan Journal of Medical Sciences, 34(1), 185–189. https://doi.org/10.12669/pjms.341.13485

Nagy J. (1998). Az érdekérvényesítő szociális képességek rendszere és fejlesztése. Iskolakultúra, 8(1), 34–47.

Nicholson, S. (2015). Peeking Behind the Locked Door: A Survey of Escape Room Facilities - PDF Free Download. https://docplayer.net/1868485-Peeking-behind-the-locked-door-a-survey-of-escape-room-facilities.html

O’Neil, H. F., Lee, C., Wang, S., & Mulkey, G. (1999). Final report for analysis of teamwork skills questionnaire (Deliverable to Statistics Canada). In Aircrew Training and Assessment (pp. 257–288). Advanced Design Information. https://doi.org/10.1201/b12464-19

Pan, R., Lo, H., & Neustaedter, C. (2017). Collaboration, Awareness, and Communication in Real-Life Escape Rooms. Proceedings of the 2017 Conference on Designing Interactive Systems (DIS ’17), 1353–1364. https://doi.org/10.1145/3064663.3064767

Pöysä-Tarhonen, J., Care, E., Awwal, N., & Häkkinen, P. (2018). Pair interactions in online assessments of collaborative problem solving: Case-based portraits. Research and Practice in Technology Enhanced Learning, 13(1), 12. https://doi.org/10.1186/s41039-018-0079-7

Scardamalia, M., Bransford, J., Kozma, B., & Quellmalz, E. (2010). Assessment and Teaching of 21st Century Skills. Assessment and Teaching of 21st Century Skills. https://doi.org/10.1007/978-94-007-2324-5_5

Semrud-Clikeman, M. (2007). Social Competence in Children. MA. https://link.springer.com/chapter/10.1007/978-0-387-71366-3_1?noAccess=true

Snyder, J. (2018). A Framework and Exploration of a Cybersecurity Education Escape Room. Theses and Dissertations. https://scholarsarchive.byu.edu/etd/6958

Tóth L. (2006). Új módszer a kreativitás megállapítására: A Tóth-féle Kreativitás Becslő Skála (TKBS). Magyar Pedagógia, 106(4), Article 4.

von Kotzebue, L., Zumbach, J., & Brandlmayr, A. (2022). Digital Escape Rooms as Game-Based Learning Environments: A Study in Sex Education. Multimodal Technologies and Interaction, 6(2), Article 2. https://doi.org/10.3390/mti6020008

Wiemker, M., Elumir, E., & Clare, A. (2015). Escape Room Games: ‘Can you transform an unpleasant situation into a pleasant one?’ Game Based Learning. https://www.academia.edu/38973807/Escape_Room_Games_Can_you_transform_an_unpleasant_situation_into_a_pleasant_one.