Abstract
Pair programming (PP) can help improve students’ computational thinking (CT), but the trajectory of CT skills and the differences between high-scoring and low-scoring students in PP are unknown and need further exploration. In this study, a total of 32 fifth graders worked on Scratch tasks in 16 pairs. The group discourse of three learning topics (comprising 9 projects) was collected. After the audio files were transcribed, 1,303 conversations were obtained. They were analyzed via Epistemic Network Analysis (ENA) Webkit, which can reveal the trajectory of students’ CT development via analyzing codes of discourse related to CT in PP. Three Scratch learning topics were assessed based on the Dr. Scratch platform to acquire the level of students’ CT and to determine the low- and high-scoring groups. Results indicated that CT concepts and CT practices were always closely related in PP and CT practices, and CT perspectives could be gradually and closely related after a long period of CT training. A significant difference between the two groups’ CT structures was found. The high-scoring group had more fragments of CT practice and connecting of CT perspectives, while the low-scoring group showed more fragments of CT concepts and expressing of CT perspectives. This research provides insights into cultivating primary school students’ CT using Scratch in the context of PP. The findings can provide suggestions for instructors to design instructional interventions to facilitate students’ CT skills via PP learning. Instructors can improve CT skills by guiding students to constantly ask questions, and specifying the role swap between driver and navigator in PP. Besides, instructors could give more consideration to the development of CT perspectives, and especially the ability to question.
Introduction
Computational thinking (CT) is regarded as a concept of using algorithmic approaches to solve real problems or to obtain a transferable solution (Shute et al., 2017). CT has become an important aspect of K-12 students’ essential abilities as well as a key feature of developing students’ digital literacy in the modernization of education (Madariaga et al., 2023). K-12 learners’ CT skills should be cultivated through programming courses (Ozmutlu et al., 2021). Students’ CT can also be improved by programming learning, including solo programming and collaborative programming (Bodaker & Rosenberg-Kima, 2023; Sun et al., 2021; Wei et al., 2020). Pair programming (PP) as a form of collaborative programming activities is often used in programming courses (Chang & Tsai, 2019; Wei et al., 2020). PP can promote students’ learning outcomes better than solo programming, which refers to students completing programming activities by themselves (Iskrenovic-Momcilovic, 2019), due to students being able to resolve the problems rapidly with the help of peer support (Demir & Seferoglu, 2020). Younger students can also attain significant learning achievements, new programming languages, partnership, and understanding of the programming concepts faster in the context of PP learning (Iskrenovic-Momcilovic, 2019; Zhong, Wang, & Chen, 2016). Thus, PP is an effective approach which can be used to improve learners’ CT skills in K-12 programming courses (Ouyang et al., 2022; Wei et al., 2020).
Students’ learning behavior patterns in PP were observed to differ significantly (Hopcan et al., 2022). Recording and analyzing students’ discussion of the programming process can fully reflect students’ input thought processes and the level of skills (Lye & Koh, 2014; Ouyang et al., 2022). Analyzing students’ discourse in collaborative learning to understand learners’ cognitive activities can provide teachers with more clues to formulate corresponding teaching strategies (Vandenberg et al., 2021). CT formative assessments could help instructors understand the learning trajectories of students developing CT and effectively intervene in students’ CT (Hadad et al., 2019). However, how to visualize the formation process of students’ CT and evaluate it effectively is an issue that needs further discussion (Stewart et al., 2021). Therefore, this study aimed to explore K-12 students’ development trajectory of CT in PP learning.
Epistemic Network Analysis (ENA) is a discourse analysis method. Due to its ability to reveal the interaction, support, and temporal process in the collaboration (Zhang et al., 2022), ENA has been used in many studies to understand the development trajectory of learners’ ideas in scientific creativity tasks (Sun et al., 2022), higher-order thinking (Ba et al., 2022), metacognitive patterns (Wu et al., 2020), social-cognitive engagement (Ouyang et al., 2022), and even to compare metacognitive differences between low- and high-scoring groups (Wu et al., 2020; Zhang et al., 2019). ENA can model the connections between features of student dialogue by quantifying their co-occurrences in the dialogue, and thus produce a weighted network of co-occurrences and visualizations linked with each data analytical component. ENA can be used to examine these networks to create a cluster of networks that can be statistically and graphically compared (Shaffer, 2018). Hence, through the visual network, it is possible to find the development trajectory in students’ CT skills. Sun et al. (2022) discussed students’ development trajectory of divergent thinking, and Ba et al. (2022) explored students’ development process of higher-order thinking in online inquiry-based discussion via ENA. Thus, this study aimed to apply ENA to analyze the process trajectory of CT among primary school learners in PP, as well as to compare the difference between the low- and high-scoring students’ CT.
The CT skills of high-level and low-level students result in differences in their programming learning (Hsu et al., 2023; Wei et al., 2020). Scratch can help students explore and master CT skills at different levels, depending on their level of cognition (Zhang & Nouri, 2019). However, Scratch could not visualize the formation process of students’ CT (Stewart et al., 2021). ENA, as a discourse analysis tool, can present the formation process of thinking (Ba et al., 2022; Sun et al., 2022), which provides a perfect opportunity for visualizing the formation process of CT. Thus, this study aimed to apply ENA to compare the difference between low- and high-scoring students’ CT in PP learning.
Literature Review
Computational Thinking
Computational thinking (CT) as a thinking process, could draw on concepts from computer science to solve problems, design systems, and understand human behavior (Wing, 2006). The Computer Science Teachers Association (CSTA) and the International Society for Technology in Education (ISTE) consider CT as comprising complex skills, such as the ability to problem-solving, creativity, as well as algorithmic thinking (CSTA & ISTE, 2011). Brennan and Resnick (2012) regarded CT as a comprehensive skill and suggested that it could be composed of computational concepts, practices, and perspectives. Chao (2016) indicated that CT as a process to solve problems can be divided into CT design, CT practice, and CT performance. Therefore, researchers hold different opinions on the connotations of CT, and have suggested that the definition of CT can be classified into two types of views (Tang et al., 2020). One of the views of CT held by researchers is that of programming with computing concepts (e.g., Brennan & Resnick, 2012). Programming skills have attracted more attention for developing CT (Madariaga et al., 2023). The second view is of the competencies which help individuals develop both general problem-solving skills and domain-specific knowledge, such as in the definitions of CSTA and ISTE (2011). The second category definition considers CT as thought processes which do not necessarily translate into code (Israel-Fishelson & Hershkovitz, 2022). This study is an exploration of CT based on PP with Scratch learning for elementary school students. Thus, the first category of CT definitions was considered and was used to assess students’ CT in this study.
Programming activities have been an effective way to foster early age children’s CT in school education (Tang et al., 2019). Visual programming is popular for younger programmers (Teng et al., 2018). Visual programming tools have a significant positive effect on cultivating CT (Ouyang et al., 2023). Scratch is a visual block-based programming tool which has become one of the most popular tools to develop programming and CT skills (Moreno-León & Robles, 2015a; Wei et al., 2020; Zhang & Nouri, 2019). Scratch provides opportunities and environments in which students can develop and examine their CT by visualization programs (Fagerlund et al., 2022), and allows novice students to construct scripts through dragging the blocks, while providing visual feedback (Kong et al., 2023). Students’ CT skills can be evaluated by teaching visual programming via Scratch, which is an effective tool for developing elementary school students’ CT skills in PP (Fagerlund et al., 2022; Wei et al., 2020). CT was used to conceptualize the learning and development of learners in Scratch programming (Brennan & Resnick, 2012). To meet design principles, instructional strategies, and assessments related to school programming learning, the CT framework was developed by Brennan and Resnick (2012), two Scratch research personnel, by studying Scratch activities in workshops and online communities (Bråting & Kilhamn, 2022). They divided CT into three aspects: CT concepts, CT practices, and CT perspectives. Every dimension is defined in detail and is easy for learners and teachers to interpret. Brennan and Resnick (2012) observed Scratch workshops as well as online learning communities for many years, and identified that CT concepts are frequent and practical in computer languages, including sequences, loops, parallelism, events, conditionals, operators, and data. CT practices are identified from the various strategies and practices that children commonly use in Scratch learning, including being incremental and iterative, testing and debugging, reusing and remixing, and abstracting and modularizing. As a dimension that is difficult to capture by the CT concepts and CT practices, CT perspectives come from Scratchers’ understanding of individuals, groups, and the technological world around them. CT perspectives include expressing, connecting, and questioning. This framework can cover CT widely, especially Scratch programming learning (Zhang & Nouri, 2019). Thus, it has come to be used frequently in many K-12 learners’ programming courses and has been used as a form of assessment (Fagerlund et al., 2022; Litts et al., 2020). For example, computational concepts and practices were applied to perform a descriptive analysis of primary school students’ programming activities in textbooks (Bråting & Kilhamn, 2022). Gillott et al. (2020) used this framework to evaluate the development of learners’ CT through interviews after Scratch learning.
Previous studies have shown that Scratch can promote learners’ CT (e.g., Kong et al., 2023). However, researchers have pointed out that using Scratch, combined with programming, is not as easy to visualize as robotics is to see how elementary students form their CT (Fagerlund et al., 2022; Stewart et al., 2021). More studies are required to fully comprehend the process of abstract formation of CT and interaction with external support (Fagerlund et al., 2022). Brennan and Resnick (2012) specifically developed a framework for implementing CT with Scratch in K-12 education. In particular, this framework guided the design of Scratch (Brennan & Resnick, 2013). Hence, Brennan and Resnick’s (2012) framework was regarded as applicable to conceptualizing CT skills, and gave Scratch a theoretical foothold for designing programming learning which extensively covers CT skills for primary school students (Sáez-López et al., 2016; Vourletsis & Politis, 2022; Zhang & Nouri, 2019). Elementary school students’ CT skills can also be improved with Scratch (Vourletsis & Politis, 2022). It is beneficial for illustrating the knowledge and skills that students use to create programming output (Falloon, 2016).
The framework of Brennan and Resnick (2012) has been proven to effectively detect and promote the improvement of CT concepts, CT practice and CT perspectives (Mouza et al., 2020). Ternik et al. (2017) designed programming learning projects based on this framework that could assist children in strengthening their CT skills, and evaluated the effectiveness of CT training with Dr. Scratch, the scoring criteria of which are closely related to the CT subcomponents defined by Brennan and Resnick (2012). Ternik et al. (2017) used Dr. Scratch to conduct a quantitative evaluation of Scratch projects, and found that the CT dimensions of the framework, especially the CT concepts, can be matched in the scoring criteria of Dr. Scratch. They proposed that the sub-dimensions of CT concepts as well as practices could be related to the scoring criteria of Dr. Scratch. Dr. Scratch scores CT on seven dimensions: abstraction corresponding to CT practices, logical thinking corresponding to operator, synchronization to event, parallelism to parallelism, flow control to sequence and loop, user interactivity to conditional, and data representation to data (Ternik et al., 2017). Besides, CT concepts have been widely evaluated through Dr. Scratch (Park & Shin, 2019; Troiano et al., 2019). For instance, Park and Shin (2019) revealed the effectiveness of Scratch for fostering CT concepts. Troiano et al. (2019) used Dr. Scratch to observe and analyze the development process of each CT dimension of students in STEM courses. Although Dr. Scratch can objectively evaluate CT concepts, the evaluation of Scratch projects does not accurately capture the acquisition of CT perspectives due to its high complexity and abstraction (Zhang & Nouri, 2019). Thus, after obtaining scores of Scratch projects via Dr. Scratch, this study also analyzed group discourse based on the above three dimensions to broaden the horizon of the processing of CT in PP by ENA.
Pair Programming
Peer collaboration, and in particular pair programming (PP), is the main pedagogy that has been adopted to cultivate students’ CT (Fagerlund et al., 2022; Tang et al., 2019; Wei et al., 2020). PP involves a process of completing a programming task which requires two programmers to work collaboratively and simultaneously. In PP, the two programmers’ roles, which could be periodically exchanged, consist of a driver and a navigator. The driver as an operator of the programming actions is usually the one who uses the keyboard, screen, and mouse. The navigator of the pair focuses more on the direction of the work through observing the work of the driver, and is the one who engages in strategic and longer-term thinking (Fagerlund et al., 2022; Wei et al., 2020).
The PP approach has obvious benefits compared with solo programming, both in terms of facilitating and supporting learning and understanding the concepts of basic programming, as well as improving attitudes toward programming (Iskrenovic-Momcilovic, 2019). For example, Zhong, Wang, and Chen (2016) indicated that students who learn programming through collaboration tend to execute better performance than individuals in terms of completing programming tasks and improving their CT. Furthermore, it is considered effective to develop elementary school students’ CT through PP in a Scratch curriculum (Fagerlund et al., 2022). However, it is difficult for a teacher to monitor the role switching of drivers and navigators in a large-sized class (Wei et al., 2020). Several scholars have emphasized that PP, which requires pairs of students to collaborate, could be used to effectively complete the programming project (Fagerlund et al., 2022; Wei et al., 2020). Higher education has been the main subject of major empirical studies related to the PP method (Tang et al., 2019). Empirical studies of PP focused on primary schools have rarely attracted attention. Thus, this study applied PP pedagogy in a primary school, and focused on the students’ CT level and development trajectory rather than on their role switching.
It is beneficial to visualize learners’ thinking performance process by comparing high- and low-scoring groups (Sun et al., 2022). Distinguishing students with high and low CT scores can help instructors conduct individualized teaching for students to effectively improve their CT skills (Xu et al., 2022). Previous studies (Hsu et al., 2023; Wei et al., 2020) have indicated the differences in the CT skills of high- and low-scoring groups of students in programming learning. Hsu et al. (2023) found that students with high scores for learning performance could scan the learning content forward and acquire CT skills better than students with low scores. Wei et al. (2020) found that high-scoring and high self-efficacy students in PP were able to clearly describe their programming work, and were more likely to self-challenge in more complex programming projects. Conversely, low-scoring and low self-efficacy students in PP could not clearly show that collaboration was effective for CT and programming learning (Wei et al., 2020). However, these studies focused on the total CT skills but ignored the development of CT skills in different subdimensions and reasons for different CT skills in high- and low-scoring groups. This may be because Scratch has certain difficulties in the formation of visual CT skills (Stewart et al., 2021), which makes it impossible to visually display specific differences between high and low CT skill groups in one aspect. Furthermore, teachers need to clarify the differences in the understanding levels of students with different cognitive levels to master the three different dimensions of CT skills through Scratch (Zhang & Nouri, 2019). No significant gender differences have been found in primary school students’ CT (Tsarava et al., 2022), especially in children’s PP learning (Choi, 2014; Zhong, Wang, & Chen, 2016). Hence, we paid more attention to the differences in the CT skills of high- and low-scoring groups without considering gender differences. This study explored the differences in the dimensions of CT with different groups, and put forward targeted suggestions for cultivating CT by comparing the differences between students with high and low scores. It is significant to explore the differences in the CT skills of students with different cognitive levels to promote the development of CT.
Epistemic Network Analysis
Previous studies on programming education have focused on programming skills, but there has been a lack of comprehensive evaluation of the formation process of CT skills. Additionally, CT assessment usually focuses on summative assessments. For example, a test experiment was conducted by Zhong, Wang, Chen, et al. (2016) at the end of the semester to evaluate CT. A scale developed by Korkmaz et al. (2017) was used to determine the degree of students’ CT skills. However, there are few formative evaluations of programming promoting the development of CT, and traditional evaluations focus on single programming results. To study the formation process of learners’ CT via collaborative programming, an effective method of formative evaluation is to analyze the content of discussions between students. The traditional qualitative analysis of discussions is time-consuming and laborious, which makes it difficult to carry out large-scale analysis and evaluation. To solve the difficulty of analyzing massive amounts of conversational data, Shaffer (2017) proposed an analysis method called “quantitative ethnography” for which the key is Epistemic Network Analysis (ENA).
ENA as a discourse analysis technique is used to construct computer-based models that illustrate the co-occurrences of codes in relevant discourse between collaborating individuals (Swiecki & Shaffer, 2020). ENA has been shown to be effective in terms of learning process analysis (Guo et al., 2023; Tan et al., 2022). Guo et al. (2023) used ENA to reveal the evolution process of learners’ cognitive engagement with different levels of interaction in online learning. Tan et al. (2022) analyzed the development trajectory of learners’ co-cognitive agency in online collaborative learning environments via ENA. However, it has rarely been applied to evaluate students’ CT skills. ENA has great potential to understand the development of students’ CT in collaborative programming. ENA can effectively detect differences in connections between codes with similar frequencies compared with automated content analysis (Ba et al., 2022) and quantitative content analysis (Csanadi et al., 2018). It also goes beyond traditional static quantitative analysis to capture progressive relationships between different variables, such as cognition and interaction (Guo et al., 2023). For example, Lye and Koh (2014) suggested that recording the thought processes of students’ verbal expression in the programming process and combining them with students’ programming processes can fully reflect the input thought processes and ability levels of each student in the group when they participate in collaborative programming. However, Scratch has some difficulties in visualizing students’ CT skills growth and exploring specific differences between groups with high and low CT skill levels (Wei et al., 2020). ENA is a reliable tool for analyzing the process of thinking and idea generation (Sun et al., 2022) and can be used to analyze collaborative and exploratory discourse of programming with upper elementary school students (Vandenberg et al., 2021). Thus, ENA has rich potential to analyze the trajectory of primary school students’ CT.
Research Questions
Existing studies have confirmed the effectiveness of PP in terms of cultivating students’ CT from the perspective of summative evaluation (Demir & Seferoglu, 2020; Wei et al., 2020). However, students’ learning behavior patterns in PP are different (Hopcan et al., 2022). The formative evaluation method can be combined with summative evaluation to further explore students’ CT development trajectory in PP learning and to compare the characteristics of students with higher CT levels with those with low CT levels on the CT development trajectory (Lye & Koh, 2014; Ouyang et al., 2022; Zhong, Wang, Chen, et al., 2016). The findings can effectively guide educators to design and implement teaching strategies. Therefore, this study aimed to apply ENA to analyze the development trajectory of pupils’ CT in PP, then to compare the differences between the low- and high-scoring students’ CT. Two following research questions guided this study.
What is the development trajectory of the primary school students’ CT?
Do low- and high-scoring groups of primary school students differ in their CT?
Methodology
Participants
The study was carried out in a fifth-grade course named Information Technology. Visual programming is taught in this course from the fifth grade in this area. The visual programming tool used in their course was Scratch. To ensure the ecological validity of the study and to reduce interference with classroom instruction, we selected an existing complete fifth grade class as the participants (Seel, 2012). A total of 32 students took part in this research.
Learning Materials
Based on the effectiveness of cultivating CT in PP via Scratch (Fagerlund et al., 2022; Wei et al., 2020), the students were taught three Scratch learning topics from the official textbook, “Information Technology,” for fifth-grade students. These Scratch projects have been validated to promote primary school students’ CT skills (Zhao et al., 2022). Each learning topic contains three learning projects of the same type, and the difficulty of the projects increased in turn with the teaching objective to cultivate students’ CT skill. Learning topic 1 consisted of three projects: High Hops (project 1), Draw Regular Polygons (project 2), and Flowers Blooming (project 3). Learning topic 2 consisted of three projects: Go Through the Maze (project 4), Let the Kitten Ask Questions (project 5), and Shark Eats Small Fish (project 6). Learning topic 3 consisted of three projects: Racing Game (project 7), Clone Aircraft Battle (project 8), and Draw a Castle (project 9).
Based on formative assessment and the chronology of Scratch project learning, an initial learning project (Flowers Blooming), an intermediate learning project (Shark Eats Small Fish), and an ending learning project (Draw a Castle) were selected for ENA. They were the last project of each learning topic. We focused on the ENA of peer dialogues in PP, aided by the scoring of Scratch programs to assess the students’ CT level. In this study, the students’ discourse for each group during the discussion of three topics was recorded, namely Flowers Blooming, Shark Eats Small Fish, and Draw a Castle. Students’ Scratch projects for all learning topics were also collected (see Figure 1). With the consent of the students, a recording device was placed on each group’s table. At the start of the discussion, students pressed the recording function on the device. After they finished the team Scratch project, they paused the recording. Ten minutes was available for the team to finish their Scratch project and conduct the group discussion, but in the actual discussion, some groups completed their group project in less than 10 minutes. After collecting all the audio recordings, the audio of the team discussions was transcribed into text. One group’s Scratch project for drawing a castle.
Procedure
The experiment was conducted for a total of 13 weeks. The fifth-grade elementary school students were at the beginning of their programming course, and almost all of them had similar programming experiences. Moreover, learners often show inefficiency when they are paired with a person who is not willing to cooperate during PP learning (Denner et al., 2019; Wei et al., 2020). Therefore, students were told that they could find their own partner to learn pair programming with at the beginning. In this way, students could have a high level of willingness to learn in PP (Tsai et al., 2023). In this way, the 32 students were divided into 16 pairs. They needed to complete all the Scratch programming tasks with their partners. The research procedure is shown in Figure 2. In the 1st to 3rd weeks, the teacher taught the basic operations of Scratch so that students could learn to use Scratch. From the 4th to the 13th week, students were taught with three Scratch learning topics. The learning workflow of the three topics is as follows: (a) The instructor reviews the learning content held in the last topic; (b) The instructor talks about the topic around the learning objectives; (c) The instructor demonstrates to the students how to complete a Scratch project according to the learning goals; (d) The instructor releases the paired programming learning topic. Each group of students discusses and completes the Scratch project and further promotes the Scratch project based on the learning tasks; (e) The instructor gives some guidance to the students to help them cope with their difficulties when finishing the Scratch project; and (f) Some student groups are invited to share their ideas and Scratch projects. The instructor and students share and evaluate peer groups’ Scratch projects. In the end, the instructor summarizes the learning goals and students’ learning performance. Learning procedure.
Instrument
Brennan and Resnick’s (2012) CT framework was adopted to code students' dialogues, and ENA Webkit was used for epistemic network analysis. In addition, the Dr. Scratch platform was used to grade students’ scratch projects to distinguish high- and low-scoring groups. Brennan and Resnick’s (2012) CT framework is closely related to the scoring criteria of Dr. Scratch (Ternik et al., 2017).
The ENA Webkit
In each Scratch programming project, students in pairs discussed how to complete the programming project according to the learning goals. To further evaluate students’ CT skills, ENA was used. ENA Webkit (http://www.epistemicnetwork.org/), developed by the Epistemic Games Group at the University of Wisconsin-Madison, is an online modeling platform for conducting ENA, and was used in the current study. ENA WebKit can quantify and visualize the conversation to facilitate comparison of the differences in the CT of different groups. In addition, Wu et al. (2020) and Ruis et al. (2017) found that setting the window size to 7 generates a more stable interpretation of the ENA model. Thus, we set the window size to 7 to analyze the group discourse. In addition, to further compare the differences in CT skills between different projects and groups, based on the views of Guo et al. (2023) and Paquette et al. (2021), we used a two-sample Mann-Whitney U test to analyze the distribution of ENA spatial projection points of CT skills.
The Dr. Scratch Platform
Dr. Scratch (http://www.drscratch.org/v2) is an accessible web application that aims to provide a free and open tool for instructors and students to analyze the quality of Scratch projects and receive feedback (Moreno-León & Robles, 2015b). Dr. Scratch has been widely used for assessing students’ CT skills (e.g., Gökçe & Yenmez, 2022; Kong et al., 2023; Zhao et al., 2022). After uploading the Scratch project to the Dr. Scratch platform, CT competency was graded according to seven dimensions: abstraction, logical thinking, synchronization, parallelism, flow control, user interactivity, and data representation, according to Brennan and Resnick’s (2012) CT framework (Moreno-León & Robles, 2015b; Wei et al., 2020; Zhao et al., 2022), where the total score reflected the development of the students’ overall CT competency (Moreno-León & Robles, 2015b). Each aspect was evaluated on a scale between 0 and 3 points, where the points corresponded to the level of CT: 0 = none, 1 = basic, 2 = developing, and 3 = proficient (Moreno-León & Robles, 2015b; Troiano et al., 2019). The individuals’ total CT scores are the sum of the scores in all seven aspects: A score of 1–7 is basic, 7–14 is developing, while 15–21 is proficient (Troiano et al., 2019). Dr. Scratch automatically analyzes programming tasks according to the above scoring rules and presents scores to show the quality level of programming projects. There have been many studies that have used Dr. Scratch to assess students’ CT skills (e.g., Moreno-León & Robles, 2015a; Park & Shin, 2019; Troiano et al., 2019; Vourletsis & Politis, 2022). In addition, CT concepts in Brennan and Resnick’s (2012) framework correspond with the Dr. Scratch scoring criteria: abstraction = CT practices, logical thinking = operator, synchronization = event, parallelism = parallelism, flow control = sequence and loop, user interactivity = conditional, and data representation = data (Ternik et al., 2017). Thus, the Dr. Scratch platform was conducted to assess the objective CT scores acquired by primary school students through the Scratch projects.
Coding Scheme of Group Conversations
Coding Scheme of CT.
Data Analysis
To obtain students’ CT development trajectory, the conversation text from the students’ discussions was analyzed using ENA Webkit. Dr. Scratch’s platform was used to evaluate the group students’ Scratch projects. SPSS version 25.0 was used to conduct descriptive statistics on the students’ Scratch project scores to distinguish between the low- and high-scoring groups. Based on views of Reinhold et al. (2020), Wu et al. (2020), and Zhang et al. (2019), the group whose project scores were above average belonged to the high-scoring group and those below average belonged to the low-scoring group.
Results
Analysis of the Development Trajectory of the Students’ CT
Coding Frequency of CT.
The epistemic networks are displayed in Figures 3–5. An average network graph was generated and projected onto a two-dimensional graph to capture the salient features of the network (Csanadi et al., 2018). Epistemic networks analysis adopts the method of Singular Value Decomposition (SVD) to maximize the differences in CT dimensions in the discourse data. The nodes represent the codes, that is, the center of the cognitive network of students’ CT skills in every project; the black dots represent the location of each code (e.g., C1); the lines between the nodes represent the connections, and the thickness of the lines reflects the relative frequency of co-occurrence (Shaffer et al., 2016). A thick line indicates a close relationship, and a thin line indicates a weak relationship. The two dimensions with the largest variance in this data are used to represent the ENA space, with the X-axis showing the dimension with the highest percentage of variance and the Y-axis showing the dimension with the second highest variance (Shaffer et al., 2016). Epistemic network of group discourse in LT1. Epistemic network of group discourse in LT2. Epistemic network of group discourse in LT3.
The epistemic network of students’ LT1 is shown in Figure 3. It presents the data variance ratio for the two dimensions (X, 14.3%; Y, 10.8%). In LT1, there are stronger links between C1, C5, C6, PC1, PC2, PC3, and PC4. That is, students’ CT concepts were closely related to their CT practices. The epistemic network of students’ LT2 is shown in Figure 4. It presents the data variance ratio for the two dimensions (X, 11.0%; Y, 10.2%). In LT2, PC1, PC2, PC3, and PC4 are closely related to C4, C5, and C6, and there is an especially strong connection between C6 and PC1, indicating that students were familiar with the steps or instructions of the Scratch operation and knew which blocks of different functions to implement. The epistemic network of students’ LT3 is shown in Figure 5. It also presents the data variance ratio for the two dimensions (X, 10.5%; Y, 9.1%). There are strong connections between C1, C5, C6, PC1, PC2, PC3, and PC4. Through the ENA for the three learning topics, we found that students’ computational concepts and practices were always closely linked. In addition, PE1 was strongly linked with PC1 in LT3. PC1 emphasizes that students make plans based on their own experience, and plan to constantly change and iterate in the process of problem solving to efficiently achieve problem solving, and PE1 focuses on students’ self-expression in the problem-solving process (Brennan & Resnick, 2012). Hence, students’ PC1 was closely related to PE1 in PP.
The subtracted discourse network can represent the differences in the two clusters’ category connection patterns (Guo et al., 2023; Sun et al., 2022). Figures 6–8 represent the comparison of students’ CT structure among the three learning topics. In Figure 6, the red dots and the blue dots represent the centroids of the students’ CT structure of LT1 and LT2, respectively. The blue or red squares represent the mean centroids of the related dots; the dotted box outside the box indicates a 95% confidence interval. The distance between the centroids shows that the structure of the epistemic network between LT1 and LT2 of CT is different. A total of 6.4% of the data variance is explained by the first dimension (X), and an additional 12.0% by the second dimension (Y). Mann-Whitney U tests were conducted to compare differences in ENA space in students’ CT structures (Guo et al., 2023; Paquette et al., 2021). The result shows that students’ CT in LT1 was significantly different from their CT in LT2 along the X-axis (Mdn LT1 = −.19, Mdn LT2 = .15, U = 600.50, p < .00, r = .64), while the result showed that no significant difference was found between CT skills on the Y-axis (Mdn LT1 = −.05, Mdn LT2 = −.14, U = 1679.50, p = .87, r = −.01). Compared with LT1, blue dots of LT2 were more distributed in C1, C2, C7, PC1, PC2, PE1, and PE2. Subtracted network of group discourse of LT1 (red) and LT2 (blue). Subtracted network of group discourse of LT2 (blue) and LT3 (purple). Subtracted group discourse network for LT1 (red) and LT3 (purple).
In Figure 7, blue dots represent the centroids of students’ CT structure of LT2 and the purple dots represent the centroids of students’ CT structure of LT3. There are differences in students’ CT measured between LT2 and LT3. A total of 4.3% of the data variance was explained by the first dimension (X), and an additional 9.3% by the second dimension (Y). Moreover, students’ CT in LT2 was significantly different from their CT in LT3 along the X-axis (Mdn LT2 = .15, Mdn LT3 = −.13, U = 2715.00, p < .00, r = −.48), while the result showed that no significant difference was found between CT skills on the Y-axis (Mdn LT1 = .16, Mdn LT2 = .15, U = 1865.00, p = .86, r = −.02). Compared with LT2, purple dots of LT3 were more distributed in C2, C3, PC1, PC2, PC3, PE1, and PE2.
Figure 8 shows that there are differences in students’ CT measured between LT1 and LT3. A total of 6.4% of the data variance was explained by the first dimension (X), and an additional 12% by the second dimension (Y). Students’ CT in LT1 was significantly different from their CT in LT3 along the X-axis (Mdn LT2 = .38, Mdn LT3 = .28, U = 384.50, p < .00, r = .77), while the result showed that no significant difference was found between CT skills on the Y-axis (Mdn LT1 = 0, Mdn LT2 = −.07, U = 1667.50, p = .96, r = .01). The dots and connections of LT1 were mainly distributed among C4, C5, C6, C7, PC2, PC3, and PC4, while the dots and connections of LT3 were mainly reflected in PE1, PE2, PC1, C2, and C3. In other words, students’ discourse showed more PE in LT3 than in LT1.
In sum, regardless of the three topics, the computational concepts and practices were always closely linked. However, the computational perspectives could be gradually enhanced after a long period of CT training. As shown in Figure 6, the position of PE1 and PE2 occurred on the side of centroids of LT2 compared with LT1. The position of PE1 and PE2 occurred on the side of centroids of LT3 compared with LT2 in Figure 7. Similarly, the positions of PE1 and PE2 also occurred on the side of centroids of LT3 compared with LT1 in Figure 8. That is, there were more and more segments of PE1 and PE2 in LT1, LT2, and LT3 with the primary school students’ PP learning. However, it needs to be noted that the frequency of PE3 was zero due to the students’ discourses in LT1, LT2, and LT3 not reflecting PE3.
Analysis of the Low- and High-Scoring Groups’ CT
Scores of the 16 Groups’ Scratch Projects on Three Learning Topics.
Note. LT1 = learning topic 1, LT2 = learning topic 2, LT3 = learning topic 3.
The differences between the low- and high-scoring groups’ CT were further explored through ENA Webkit. As shown on the left side of Figure 9, there are stronger links between C1, C5, C6, PC1, PC2, PC3, PC4, and PE1 in the low-scoring groups. Regarding the high-scoring groups (as shown on the right side of Figure 9), there are stronger links between C1, C5, C6, PC1, PC2, PC3, and PC4. In other words, the co-occurrence frequency of these CT fragments is the highest in the network analysis of the high-scoring groups. This finding also proved that students’ CT concepts and CT practices are always closely related. Epistemic network of group discourse of low (red) and high-scoring (blue) group students’ CT.
Besides, Mann-Whitney U tests were conducted to assess differences in ENA space by comparing the low- and high-scoring groups’ CT structures (Guo et al., 2023; Paquette et al., 2021). The results showed that low-scoring groups’ CT structures were significantly different from those of the high-scoring groups along the X axis (Mdn low-scoring group = .08, Mdn high-scoring group = −.07, U = 4581.50, p < .00, r = −.39). Referring to the Y axis, the CT structures of the high- and low-scoring groups did not significantly differ (Mdn low-scoring group = −.04, Mdn high-scoring group = −.02, U = 3279.50, p = .93, r = .01).
To clarify the difference in the CT cognitive network structure of the low- and high-scoring groups, their respective collinear connection coefficients were calculated, and the subtracted network was obtained through ENA. Figure 10 is the subtracted network of group discourse of low- (red) and high- (blue) scoring group students’ CT. It compared the association weights of the two groups, where the red solid line indicates that the link weight of the cognitive network of the low-scoring groups is greater than that of the high-scoring groups, while the blue solid line is the opposite. It also shows that the distance between the centroids of the two groups of cognitive networks is large, indicating that the structure of the corresponding CT cognitive network is quite different. It also shows that the high-scoring group students were more comfortable with CT practices (PC1, PC2, PC3, and PC4) than the low-scoring group students, but the low-scoring group students had more fragments (C1, C4, C5, C6, and C7) of CT concepts than the high-scoring group students did. In addition, it shows that both the low- and high-scoring group students had fragments in PE1 and PE2. However, by comparing the position of PE1 and PE2 for the high- and low-scoring group students, we found that the high-scoring group students had more segments in PE2 compared to the other groups, and the low-scoring groups had more segments in PE1 compared to the high-scoring groups. Subtracted network of group discourse of low- (red) and high- (blue) scoring group students’ CT on the LT3.
Discussion
The Development Trajectory of the Students’ CT
CT is a kind of constructional thinking that requires transforming problem-solving thinking into a series of cognitive processes (Robledo-Castro et al., 2023). CT requires solving problems by analyzing the problem, simplifying the problem, presenting and interpreting the data, formulating the procedure, and finally implementing the procedure (Kong et al., 2019). Therefore, the development trajectory of CT in solving problems is appropriate for students’ cognitive development trajectory. Students should first analyze the requirements to solve the problem, think about the represented ways of the issue through programming, and then go on to analyze specific technical solutions to achieve the study goal. Therefore, students use the learned CT concepts including sequences, loops, parallelism, events, conditionals, operators, and data aspects to analyze the problem, and then use the learned practical CT skills including testing and debugging, being incremental and iterative, abstracting and modularizing to solve the problem, and reusing and remixing CT skills.
This study traced the trajectory of students’ CT by ENA, and deconstructed the three dimensions of CT skills to trace the CT formation process. The results of this study demonstrated that CT concepts and CT practices were always closely related in the LT1, LT2, and LT3. Sáez-López et al. (2016) also confirmed that Scratch could promote students’ understanding of CT practices and CT perspectives while enhancing their CT concepts. Furthermore, students’ CT practices require the usage of CT concepts in programming learning (Brennan & Resnick, 2012). However, this study found that CT perspectives could be gradually enhanced after a long period of CT training. Primary school students’ PE1 (Expressing) and PE2 (Connecting) can be enhanced with the deepening of pair programming training. The ability to express and connect both highlights the importance of socializing and communicating with others to some extent (Brennan & Resnick, 2012). Pair programming provides an opportunity for primary school students to realize full communication and cooperation (Wei et al., 2020). Students can improve their expression ability through communication and discussion among peers in programming learning (Ke, 2014). Thus, their ability to express and connect is enhanced. Compared with solo programming, students interact collaboratively by constantly communicating and expressing their ideas (Wei et al., 2020), which is one of the important ways to help develop plans and solve solutions in PP. This may also be the advantage of PP, which provides insights for cultivating CT perspectives.
In addition, we found that CT perspectives need to be strengthened. They need to be shaped and developed by students’ own personality changes and improved thinking habits after a period of training in CT (Brennan & Resnick, 2012). Compared with CT concepts and practices, CT perspectives as a higher CT skill for elementary school students may be more difficult to develop, and require instructors’ guidance by various programming tasks linked with reality. Students need to be encouraged to promote their CT perspectives through improving the key components of CT perspectives: the ability to connect, the ability to express, and especially the ability to question (Kong & Wang, 2020). In this study, CT discourse data related to PE3 (questioning) were not collected. Questioning emphasizes that learners question the rules that have been formulated and can even respond to the doubts by designing solutions to problems (Brennan & Resnick, 2012). For example, learners question the limitations of Scratch operations during Scratch programming (Brennan & Resnick, 2012). Students are accustomed to teacher-led programming learning activities (Kong et al., 2018). Primary school students seemed to ask questions about areas of knowledge that might challenge prior technology or rules (Kong & Wang, 2020). Teachers can create a variety of meaningful learning situations linked to their experience in real life, encourage and guide students to pose questions and break out of their usual way of thinking to cultivate the ability to question (Mohamad & Tasir, 2023).
The Low- and High-Scoring Group Students Differed in Their CT
The results showed that the high-scoring students had more fragments of CT practices compared with the low-scoring students, while the low-scoring students showed more fragments with the CT concepts in the sequence, conditionals, operator, and data aspects. Moreno-León and Robles (2015a) also found that students scored relatively poorly on data representation (corresponding to data aspects) and user interactivity (corresponding to conditionals) in CT concepts (Ternik et al., 2017). This finding is also similar to the study by Ternik et al. (2017), which showed that a lower level of CT skills means less abstraction and logic as well as CT practices. It is relatively easy to improve CT concepts with Scratch programming activities compared to CT practices (Park & Shin, 2019; Vourletsis & Politis, 2022). It is common to improve CT concepts through learning programming languages such as Scratch (Kong et al., 2023; Lye & Koh, 2014). However, primary school students have a certain challenge in acquiring CT practice, due to CT practice requiring students to pay more attention to the process of programming learning rather than just concept learning (Brennan & Resnick, 2012). Transitioning from CT concepts to CT practices is a process of moving from concrete to abstract, which needs students’ higher-order CT skills, including problem solving, abstract thinking, creative thinking, and critical thinking (CSTA & ISTE, 2011). These skills, such as abstract thinking, can be developed in the practice of solving problems.
In the aspect of CT practices from this study, the students with higher levels of CT practices can get good learning performance and high-order skills in Scratch programming. This verifies the findings of previous studies. For example, some studies found that CT practices made a greater contribution than CT concepts for K-12 students to achieve CT as they provide them with some skills in daily life (Lye & Koh, 2014; Wing, 2006). Programming learners may develop their CT practices (e.g., debugging) in a certain stage (Wohl et al., 2015). For example, learners can develop their programmer’s collaborative problem-solving skills in a collaborative programming learning context (Fawcett & Garton, 2005). Furthermore, high-scoring group students tend to cooperate with each other and develop into cohesive groups in computer-supported collaborative learning (Kong & Wang, 2020). Tacit interaction and contact between high-scoring students can help them develop practical ability via programming participation (Breakwell, 2001). Teachers can intervene in groups when organizing students to engage in pair programming, so as to achieve a more conducive learning environment for students’ collaboration and interaction to acquire CT skills (Fagerlund et al., 2022). For example, teachers can give guidance to each group of students in assigning design tasks, so as to promote them to share and get feedback in PP learning (Tsan et al., 2020).
In addition, this study found that the high-scoring students had more fragments in PE2 (Connecting) of CT perspectives compared with the low-scoring students, while the low-scoring students showed more fragments in PE1 (Expressing) of CT perspectives. Connecting refers to learners’ ability to actually feel that they are collaboratively programming with others in Scratch (Brennan & Resnick, 2012). Students with high scores may improve their connection ability due to the fact that they can promote the generation of more active learning behaviors and the completion of high-quality programming tasks through sufficient discussion and cooperation in the group (Roth & Brooks-Gunn, 2003). According to Brennan and Resnick (2012), PE1 (Expressing) means that learners find it fun to learn programming through Scratch, and they enjoy the process of using Scratch to express new ideas. Students with low scores may regard programming as a playground and learn to code through fun and games (Bers, 2018), but they may stop programming learning when they cannot solve the problem via Scratch (Allsop, 2019). Teachers can guide students to conduct self-assessment and ask questions to identify the crux of the problem, so as to explore and continue to find solutions to the problem (Allsop, 2019).
Conclusions
ENA was utilized to analyze the discourse among primary school students in PP learning and to explore students’ CT level and development trajectory in this paper. In students’ Scratch PP learning, the development trajectory of the students’ CT showed that there were more fragments in the dimension of CT concept in the early stage, with more CT practice related fragments occurring in the later stage. Students’ CT concepts and CT practices were always linked in the process of programming learning. This study also found that CT perspective related fragments were insufficient in all stages, but PE1 (Expressing) was strongly related to PC1 (Being incremental and iterative) after a period of time of programming learning. Besides, the low- and high-scoring group students showed different characteristics of CT structure. Students with high scores had more fragments in CT practices than the low-scoring students, while students with low scores showed stronger connection of the CT concepts. Students with high scores had more fragments in PE2 (Connecting) of CT perspectives compared with low-scoring groups, while students with low scores showed more fragments in PE1 (Expressing) of CT perspectives.
Implications
This study clarified the three dimensions of primary school students’ CT in PP learning, including CT concepts, CT practices, and CT perspectives. Current research provides insights into cultivating primary school students’ CT using Scratch in the context of PP. In addition, this study enriched the assessment research on CT. By using ENA, qualitative conversational communication was quantified, and the capability characteristics and development trajectories of group cooperative programming in the early, middle, and late stages of project programming can be tracked and analyzed. It enables us to obtain the teams’ CT levels and development trajectories.
These findings can provide a reference for programming teachers regarding how to improve students’ overall CT skills in PP learning. The results show that students mainly acquire CT concepts in the early stage, and CT practices in the later stage as well as CT concepts always being linked with CT practices in programming learning, but CT perspectives are always lacking. Computing is seen not only as consumption by computational thinkers, but also as something that they can use to design and express themselves. Computational thinkers look at computing as a medium and say, “I have the ability to create” and “Through this new medium, I have the ability to express my ideas” (Brennan & Resnick, 2012). Thus, teachers should help students develop and enhance their CT perspectives, especially in thinking of computing as a medium for expressing themselves and for functional innovation. Students can be encouraged to express themselves and put into practice real-world problem solving by linking programming projects to their specific personal interests and lives. Furthermore, Kong and Wang (2020) found that developing the ability to question in CT perspectives could stimulate the growth of abilities in other CT perspectives. Hence, teachers can constantly guide students to ask questions so as to enhance their questioning ability and promote the improvement of CT perspectives.
The gap among the two different groups in CT scores was further complemented by the fact that students with high scores had stronger CT practices, while students with low scores had stronger CT concepts. Therefore, instructors can assign students with high and low scores to work together to complete paired programming tasks, depending on their programming level. Besides, in PP, drivers and navigators as the roles of peers can be switched according to the actual situation in the project solution process to help each other finish the programming task efficiently. Instructors could negotiate and guide the discussion process to improve their ability to address problems in programming.
Limitations and Future Study
Firstly, this study did not collect students’ experiences of completing the paired programming tasks, which would have provided a broader perspective to help interpret the findings. Secondly, this study was conducted in an existing class of 32 primary school students; thus, the findings may not represent all primary school students. To expand these findings, future studies could invite more primary school students from different schools or cultures to participate. Finally, the dichotomization and only 10 minutes of discussion data in the ENA may have increased the statistical error and affected the rigor of the findings. Although this study further reflects students’ CT skills through their Scratch project scores, the complexity of different programming tasks will affect the students’ CT performance. The two groups were divided according to the average score; it was difficult to accurately judge the groups of students with scores close to the average. Thus, future research could balance the coding difficulty of the Scratch project and supplement the study with expert observations of the learning process to provide additional support for the study.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the National Science and Technology Council, Taiwan, under grant NSTC 111-2410-H-019-006-MY3 and NSTC 111-2423-H-153-001-MY3.
