Assessing the Distance Teaching-Learning Experience in Higher Technical-Vocational Education in Chile

Abstract

Spanish

This study aimed to validate a multidimensional instrument for assessing distance teaching-learning. It combined factors measuring motivation with instructional material and the teaching-learning experience with peers and teachers. The instrument was administered to 2,984 higher education students in Chile, demonstrating its statistical validity and adequate fit indicators. Consequently, this instrument can confidently evaluate online learning in higher education.

Keywords

online learning higher education vocational and technical education motivation ARCS

Introduction

The health emergency has renewed the attention to distance teaching-learning (Rodríguez-Triana et al., 2020), which has led to an interest in understanding the role of instructional materials (Hamora et al., 2022) and online interaction experiences in teaching-learning experiences (Miao et al., 2022). In this sense, the distance learning modality involves the activation of psychological processes associated with the construction of learning to connect with critical pedagogical aspects, such as a possible reduction of motivation to learn under this modality (Louvigné et al., 2017). Perhaps the most important implication of the constructivist perspective on motivation is the design of learning activities based on individual needs. As Wigfield and Cambria (2010) point out, learning activities linked to students’ learning needs are more effective and produce more meaningful learning. In an e-learning process, it allows students to continue to seek to learn even after formal instruction sessions.

Numerous scholars argue for consistently incorporating learner motivation within teaching-learning endeavors, which holds particularly true in instructional practices within digital media, where motivation assumes a paramount role and merits inclusion as a strategic instructional component (Barger & Byrd, 2011; Keller, 2008, 2016). Traditional considerations of motivation toward online learning have been primarily channeled through the lenses of motivational materials and Keller’s (2008, 2016) model. However, the COVID-19 pandemic’s unique circumstances have underscored the need to expand Keller’s model to accommodate additional dimensions. Moreover, recent research focuses only on the motivation associated with instructional materials. Only a few of them include measures focused on the perceptions of students about relationships with peers (e.g., Douglas et al., 2023; Inan et al., 2023; Pi et al., 2023), and fewer address how students perceive their relationship with teachers (i.e., Makovec Radovan & Radovan, 2023).

Beyond the COVID-19 pandemic-induced shift to emergency remote education, the issue of student motivation within technical vocational higher institutes in Chile remains a challenge (Saadati & Celis, 2022). Drawing from the synthesized background literature, this study proposes a comprehensive model to evaluate the holistic experience of distance courses operating in emergency higher education. This model is applicable across various distance modalities, encompassing synchronous interactions with teachers and peers. The proposed model is structured around two intertwined dimensions: (1) the nexus of motivation and instructional materials, and (2) the assessment of teaching-learning activities. This research strives to provide a framework for assessing and enhancing the distance teaching-learning experience by delving into these dimensions.

Technical Vocational Higher Education in Chile

Technical vocational higher education in Chile focuses on training advanced technical skills, leading to a higher-level technical degree (four semesters), and professional degrees without a bachelor’s degree (eight semesters) (CNA, 2020; MINEDUC-UNESCO, 2017). Since 1981, the Chilean TVHE system has been developed mainly through private institutions (Zapata & Tejeda, 2016), which is associated with the effects of the market model of Chilean education established in the dictatorship era (Bellei, 2015; Valiente et al., 2020). In this context, TVHE has grown enormously in Chile in recent years, reaching 40% of total higher education enrollment (MINEDUC, 2021).

Theoretical Framework

The theoretical framework underpinning this study rests on a constructivist approach to the Expectancy-Value Theory (Keller, 2008, 2016). This framework collectively elucidates the motivation-learning nexus in online educational landscapes, thus supporting the dual levels of the proposed instrument: motivation toward instructional materials, aligned with Keller’s (2008, 2016) model, and motivation toward synchronous online activities, which has proven to have a positive impact in motivation and learning (Wang et al., 2023).

The expectancy-value theory is a particularly relevant approach to studying the relationship between learning and motivation in digital environments. This theory posits that an individual’s motivation to engage in an activity is driven by their expectations of success (expectancy) in the activity and their perception of how satisfying the task is in fulfilling personal needs (values). This approach has been explored in the field of digital learning, yielding pertinent (Eccles & Wigfield, 2002; Keller, 2008) It has also been investigated with relevant results in the field of learning in digital environments (Barger & Byrd, 2011). Along these lines, Keller (2008, 2016) developed the ARCS model of Motivation, a model based on factors influencing students’ motivation. This model identifies four motivational factors that help the development and maintenance of learning: attention, relevance, confidence, and satisfaction. This model shows the effectiveness of instructional designs based on a motivational model where the first factor is to capture students’ attention to arouse their interest and stimulate their curiosity to learn. The second factor, relevance, is about connecting the learning activities with the learners’ personal needs and goals to achieve a positive attitude toward the activity. The third factor, confidence, is about strengthening the learners’ belief and feeling that they will succeed. Finally, satisfaction refers not only to the achievement of the task but also to the satisfaction that the task itself produces, which can be promoted by teachers through rewards, social values, the quality of instruction, and the availability of resources for learning. Evidence has shown that successful work around these factors significantly increases the likelihood of necessary learning outcomes, while underdeveloped work on these factors can have a negative impact on teaching-learning processes in digital environments (Keller, 2010, 2016).

Liao and Wang (2011) applied the ARCS model using an experimental design that allowed them to establish that planning from a motivational perspective has a direct and positive impact on students’ satisfaction with objectives, materials and methods, teacher characteristics, learning climate, assessments, and overall satisfaction with the subject. Furthermore, Reynolds et al. (2017) explored through a multiple case study implementation of the ARCS design model and concluded that motivation influences students’ engagement with learning.

Other scholars indicated that constructivist theory offers a conceptual framework for understanding motivation as a product of social negotiation within group learning contexts. This negotiation of motivation is inseparable from instructional processes and the learning environment, creating a cultural norm of joint activity between learners and their social context. This may result in the interaction between interests, cognitions, and affective engagement and their motivated behaviors (Sivan, 1986).

Recent research in remote education has focused on students’ perceptions and experiences with distance learning during the period of increased health restrictions (Aristovnik et al., 2020; Bashir et al., 2021; Muthuprasad et al., 2021; Vadakalu Elumalai et al., 2020). These studies account for a set of elements necessary in the design of an effective online learning environment. Among the factors that affect the success of online classes are administrative support, course content, course design, instructor characteristics, learner characteristics, social support, and technical support (Vadakalu Elumalai et al., 2020).

Evidence from a self-administered instrument applied to students in India and Saudi Arabia who attended virtual classes during the pandemic period, positive associations are found between these seven independent factors and the quality of online learning reported by students (Vadakalu Elumalai et al., 2020). The administrative capacity of a higher education institution to plan and design online courses and to provide social and technical support to its students and faculty; the technological skills and capacity of the faculty member to provide detailed feedback; and the technological skills and infrastructure of the students are all highly predictive of the effectiveness of an online course. In the same vein, Aristovnik et al. (2020) showed that university students’ overall satisfaction with their institution’s performance during the pandemic is positively related to student’s satisfaction with the digital material used in lectures (recorded videos) and with perceived teacher support (providing sufficient and appropriate feedback on exams). Muthuprasad et al. (2021) explore students’ perceptions of online education and identify key elements for online course success, including the nature of the content, infrastructure, instructor competence, preparation, and student follow-up.

The constructivist perspective posits that student engagement in distance education includes making good use of time and energy for learning through digital instructional materials and engaging in meaningful interactions with the teacher and other students during class (Dixson, 2015). However, the literature on online learning environments must sufficiently emphasize how effectively teachers and students can interact and communicate in virtual contexts (Trespalacios et al., 2021). This is in two senses: (1) to improve class participation and student learning and (2) to develop a sense of school community without the infrastructure that typically brings students together in a face-to-face social context. Evidence shows that, when comparing different study modes, students perceive online learning as ineffective in promoting student participation and interaction while simultaneously being limited in developing a sense of community (Komolafe et al., 2020). Complementarily, research on the effectiveness of learning in virtual environments shows that the intentional and varied use of technologies plays an important role in fostering the development of school community and connectivity. Elements such as the use of interactive learning methodologies, include: multimodal discussion forums; the provision of timely and detailed feedback from the teacher; and the use of shared platforms (i.e., GoogleDocs, wikis, blogs) and social networks to connect with peers and teachers can help improve participation and connectivity in virtual environments (Trespalacios et al., 2021).

To date, most research has preliminary focused mainly on motivation associated with instructional materials, and only a few studies considered the perceptions of students about relationships with peers (e.g., Douglas et al., 2023; Inan et al., 2023; Pi et al., 2023), and a smaller number of studies address how students perceive their relationship with teachers (Makovec Radovan & Radovan, 2023). We argue that this aligns with the pre-pandemic notion of distance education, which often involved minimal interaction with teachers and peers (Shankar et al., 2023).

To sum up, in the post-pandemic era, drawing from the needs and experience, these integrations of two theoretical frameworks furnish a comprehensive scaffold that supports and validates the two levels of the instrument—motivation toward instructional materials and motivation toward synchronous activities—enabling a holistic exploration of the dynamics underlying online learning experiences.

Method

Instrument

The instrument started with a socio-demographic section, inquiring about the participants’ demographic information, including gender, age, commune of residence, year of high school graduation, mode of high school, study day, career, location, working hours per week, internet access (connection and device), previous experience in distance learning courses, among others. The main section of the instrument comprises two parts, each corresponding to distinct facets of distance course evaluation: (1) motivation with instructional materials, (2) effectiveness of course design and communication with peers. The first section draws from the Instructional Materials Motivation Scale (IMMS) questionnaire (Keller, 2010), while the second part integrates a compilation of items sourced from various instruments, including those by Dixson (2015), Komolafe et al. (2020), and Muthuprasad et al. (2021).

In the first part, which evaluates students’ motivation toward instructional materials, we adapted items from the IMMS questionnaire based on Keller’s (2010) ARCS model (Attention, Relevance, Confidence, and Satisfaction) of motivation. This section is intended to measure reactions to self-directed instructional materials. Keller (2010) emphasizes the ARCS model’s adaptability to specific contexts. To provide this adaptation with context, we specifically considered online mathematics courses. This choice was deliberate, as each study program at both institutions incorporates at least one mathematics course, ensuring a more precise contextual fit. Therefore, the IMMS survey was slightly modified to fit the context. The original items were translated into Spanish and worded in the same direction, where 1 is less motivated and 5 means fully motivated. The initial version of this section included 36 items with a five-value Likert-type scale to measure these subscales. The items were distributed as follows: 12 items for attention, 9 items for relevance, 9 items for trust, and 6 items for satisfaction.

In the second part, a total of 12 items were carefully selected and adapted from Muthuprasad et al. (2021), Komolafe et al. (2020), and Dixson (2015). These items evaluated the learning activities within an online mathematics course across two key dimensions: (a) the extent to which it enables students to manage their time and engage with their teacher, and (b) its capacity to facilitate peer-to-peer communication.

The first dimension, encompassing eight items, assessed perceptions of learning, teacher communication, time management, assessment methods, performance, professional expectations, skills development, and the teacher’s availability to address queries during class. In contrast, the second dimension, focusing on communication with peers, consisted of four items that gauged the degree to which the online course fostered connections among students, including knowledge sharing, mutual support, promotion of interaction, and collaboration. As described, the initial version of the instrument comprises a total of 48 items, in addition to the demographic sections.

Participants

A purposive sample of second-year students from two institutions offering 2 and 4 year Higher Technical and Vocational Education (HTVE) programs. These institutions are the larger and prominent institutions of the HTVE education system in Chile, reaching nearly 100,000 students each, which account for more than 30% of the total enrollment of that kind of education.

Considering both institutions, the study population is approximately 40,000 students in the second year of their degree in 2021 (the period in which the data were collected). It was decided to conduct the research with these students to obtain a sample that covers the study cycle of both two-year and four-year degree programs. At the same time, this decision made it possible to include students who, for the most part, were in their first year of studies during 2020, the year to which the questions in the questionnaire refer. It should be noted that this type of large purposive sample design, focusing on one or more institutions with a profile that allows for a broad understanding of an education system, has been frequently used in research on the integration of digital technologies in higher education training processes (Gulbahar, 2008; J. Han et al., 2018; Muianga et al., 2019; Tondeur et al., 2010; Zhang et al., 2020). In the end, a sample of 2,984 students was obtained, corresponding to 7.5% of students with the indicated profile in both institutions. The sample characteristics are presented in Table 1.

Table 1.

Sample Characteristics.

Variable	Category	Frequency	Percentage
Age	19 years-old	1,465	49
	20 years-old	713	24
	27 or more years-old	806	27
Gender	Feminine	1,230	41
	Masculine	1,728	58
	Other	26	1
Region	Metropolitan region	1,356	45
Region	Other region	1,628	55
Shift	Regular	1,983	66
Shift	Night school or both	1,001	34
Teaching-learning area	Business and administration	812	27
	Informatics	370	12
	Design and creation	70	2
	Hospitality and tourism	86	3
	Engineering	1,392	47
	Health	254	9
Work situation	Doesn’t work	1,887	63
	Less than half-day	207	7
	Half or full day	890	30

Procedure and Analysis

Developing and testing the validity evidence of the instrument follows the Standards for Educational and Psychological Testing (AERA et al., 2014; Bostic et al., 2019) as the framework. These Standards establish five sources of validity evidence: test content, response processes, internal structure, relations with other variables, and consequences of testing, which are described in this paper as follows:

Stage 1 . Test content: In the initial phase, the instrument underwent cognitive interviews with two students and two expert teachers from the collaborating institutes. These interviews were conducted with direct support from the central administration of the institutes. During the interviews, the participants were presented with the instrument, and they were asked to read it item by item. They were then invited to provide feedback for each item aimed at enhancing the clarity and relevance of the items within the institutional context. Minor adjustments in grammar and language were made to the first version of the instrument as a result of this stage. This process ensured that the test items were clearly represented and also corresponded with the designated content areas.

Stage 2 . Response processes and piloting: A pilot application of the instruments was conducted with a sample of 97 cases, representing a small group of individuals similar to the target population of one of the collaborating institutions. Exploratory Factor Analysis (EFA) was utilized to assess the support of data for the initial dimensions and to determine if the number of items designed for each dimension in the instrument was adequate or if we needed to add more items. The Kaiser-Meyer-Olkin measure of 0.864 was adequate and the Bartlett’s test of sphericity was statistically significant, with a p value equal to .00. The communalities were analyzed for each item. Following a rule to eliminate those under 0.4, six items were removed based on the analysis results, without a need to add more items. The items were “Doing the course exercises gave me a satisfying sense of accomplishment” (Satisfaction); “The course content will be helpful for me in the future” (Relevance); “As I studied for the course evaluations, I felt confident that I would pass” (Confidence); “The course stimulated my curiosity” (Attention); “During the course the variety of materials helped keep my attention and interest” (Attention); “What I have learned through the online classes during the pandemic will help me to be a good professional” (Teacher Centered Activities). Following these revisions, the second version of the instrument was refined to include 41 items for application in fieldwork and other sources of evidence for validation.

Stage 3 . Fieldwork: This stage aimed to review the last two pieces of evidence of the instrument’s validity, internal structure, and relationships with other variables. In this stage, the institutions collaborated in preparing and implementing the fieldwork. Each institution sent a prior invitation to their students. Subsequently, the survey was sent and made available. In institution A, the instrument was uploaded to a paid survey platform under the direct supervision of the research team. In the case of institution B, the instrument was uploaded onto an institution’s digital platform, making the survey available to all students in the target population. The fieldwork was carried out consecutively over 3 weeks in each institution between September and November 2021.

The internal structure validity examines how the test items are related to the conceptual framework of the construct. This can be evaluated using techniques such as confirmatory factor analysis to determine the consistency of item relationships. Therefore, a Confirmatory Factor Analysis (CFA) was performed, and some possible relationships between variables were explored. CFA provides evidence to support the internal structure validity of a measurement instrument by verifying the number of underlying dimensions and the pattern of item-factor relationships, that is, factor loadings (Rios & Wells, 2014, p. 109). Adequate goodness-of-fit indices should be CFI > 0.90 and RMSEA < 0.08 (Byrne, 2016). In addition, internal consistency reliability was checked to estimate the accuracy coefficient from a set of actual test scores by calculating Cronbach’s alpha.

Evidence-based relationships with other variables include examining the relationship between the test and other measures, as empirical evidence suggests. For this purpose, by the available literature, evidence of convergence of this tool’s variables with the participants’ age criterion was used by examining the level of correlation. Evidence of convergence versus evidence of differentiation is obtained when measures of different constructs show a high correlation.

Following these standards and considering these different sources of narrative evidence, the results of a complete and accurate assessment of the validity of the questionnaire are presented in the findings section. Consequential validity based on the positive or negative consequences of using the results based on this instrument, the positive attributes, the educational system, the school, and the classroom content will be discussed further in the discussion section.

Ethical Aspects

The project was evaluated and approved by an ethics committee of the institution to which the researchers are affiliated. All information provided by the students was recorded in such a way as to maintain total anonymity and confidentiality. The responsible for the care of these measures is the Responsible Researcher, who must keep the data for 36 months from the closing date of the fieldwork. The students read this information and marked their agreement with these conditions on the form before answering; they gave their informed consent to participate and proceed with the questionnaire.

Results

Internal Structure Validity

The instrument used consisted of 41 items assessing students’ experience of learning mathematics online during the pandemic. Following Kline’s (2010) recommendations, the data were regarded as normal since they have skewness between −0.91 and 0.37 and kurtosis between −0.72 and +0.76, in the range within ±3 and ±7, respectively.

Confirmatory factor analysis (CFA) was used to assess a model composed of six latent variables: Attention (Atte), Confidence (Con), Satisfaction (Sat), Relevance (Rel), Teacher-Centered Activities (TCA), and Peer-centered activities (PCA). In this model, TCA were included items like Learning (L), Communication with teacher (CT), Time (T), Evaluation (E), Performance (P), Professional (Pr), Skills (S), Teacher attention to questions (TAQ), and PCA were covered items as Peer communication (PC), Peer questions (PQ), Knowledge exchange (KE), and Peer relationship (PR). The best model fitted with the data was a second-order model in which the six latent variables presented as first-order factors, and there were also two second-order factors called Motivation toward Materials and Evaluation of Activities (Figure A1 at the appendix).

Eleven items with low factor loadings were removed to fit the model with the data. It means, the initial measurement model (model 1) was modified by removing the problematic items. The items removed were: “When I started the course, I was under the impression that it would be easy for me”; “The course content is related to things I already knew”; “There were stories, images or examples in the course that showed me how this material could be important in real life”; “Passing the course was important to me”; “I liked the course so much that I was motivated to learn more about the topic”; “The exercises in the course were too easy”; “I found the course entertaining”; “The course material is interesting”; “The course guides were brief and clear”; “When I passed the course I felt good”; “The online modality facilitated the conversation with my professors.”

As a result, the 30 items remained in the last model (Table 2). The improved model (measurement model 2) adequately fits the data (Chi-Squared = 6,665, df = 614, CFI = 0.93, NFI = 0.93, RMSEA = 0.057, SRMR = 0.033). In addition, all items had high loadings on their respective factors (Table 2), and small correlational residuals were observed (Kline, 2010). This model provides evidence in favor of the proposed theoretical structure.

Table 2.

Standardized Loading Factors and Reliability.

Variables	Item	Standardized loading factors	alpha de Cronbach
Attention (Atte)	Atte_2	0.59	.9
	Atte_8	0.75
	Atte_11	0.82
	Atte_12	0.76
	Atte_15	0.84
	Atte_17	0.87
Confidence (Con)	Con_3	0.65	.82
	Con_4	0.6
	Con_7	0.73
	Con_13	0.7
	Con_34	0.75
Satisfaction (Sat)	Sat_21	0.85	.85
	Sat_27	0.74
	Sat_36	0.83
Relevance (Rel)	Rel_16	0.7	.87
	Rel_18	0.74
	Rel_23	0.85
	Rel_26	0.78
	Rel_30	0.73
Teacher Centered Activities (TCA)	TCA_ L	0.8	.89
	TCA_ CT	0.78
	TCA_ T	0.74
	TCA_ E	0.7
	TCA_ P	0.76
	TCA_ S	0.57
	TCA_ TAQ	0.75
Peers Centered Activities (PCA)	PCA_ PC	0.68	.84
	PCA_ PQ	0.68
	PCA_ KE	0.85
	PCA_ PR	0.8
Motivation toward materials (MtM)	Atte.	0.96	.93
	Con.	0.87
	Sat.	0.99
	Rel.	0.97
Evaluation of Activities (AoE)	TCA	0.97	.78
Evaluation of Activities (AoE)	PCA	0.76	.78

Note. p-values < .001 for all factor loadings.

After identifying the factor structure and eliminating problematic items, items that remained in the confirmatory factor analysis (CFA) model were selected to measure the variables under study, resulting in eight items for Attention, three items for Satisfaction, seven items for Confidence, six items for Relevance, eight items for Learning Experience, and four items for Peer Communication Experience. The final questions can be found in the appendix.

Cronbach’s alpha coefficients, which measure the internal consistency of the items on a scale, were all greater than .85 for all factors. This indicates that the selected items reliably measure the variables under study. Considering the previous results, we concluded that the instrument variables are sufficiently reliable to assess students’ experience of teaching and learning mathematics online during the pandemic.

Figure 1 presents the correlations between the variables measured by the instrument used in the study. It was observed that all correlations between variables were positive and statistically significant. The variables associated with motivation were highly correlated, with values of .79 or more. Similarly, the two experience-related variables, learning experience, and peer communication experience, were highly correlated with a value of .68.

Figure 1.

Correlation between dimensions.

Among the motivation and experience variables, it was found that learning experience showed a higher correlation, around .4, with the four motivation variables. This means that students with a more positive perception of their learning experience during the online course reported a higher level of motivation, including attentiveness, confidence, relevance, and satisfaction with their mathematics course. These findings suggest that a satisfactory learning experience can positively affect students’ motivation in the context of online mathematics teaching and learning during the pandemic.

Evidence of Relationships with Other Variables

To check for evidence of validity in this area, we examined two factors of motivation that the instrument was designed to measure, along with the external variable of participant age. Pearson’s correlation coefficients revealed that motivation toward Materials (r = .05, p < .05) and Evaluation of Activities (r = .11, p < .001) showed a positive and statistically significant correlation with the age of the students (i.e., the older the age, the higher the satisfaction). Statistically significant and positive relationships were also found between the subscales of motivation toward materials (Satisfaction r = .04, p < .05; Relevance r = .05, p < .05; and Attention r = .07, p < .001) and the subscale of Evaluation of Activities (TCA r = .12, p < .001; and PCA r = .08, p < .001) with the age of the participants. These statistically significant and substantial relationships provided evidence of validity based on relationships with other variables consistent with what has been observed in previous research (Morin et al., 2019). This positive correlation between instrument measures and participant age shows evidence of convergence. As for confidence, a positive correlation with participant age was observed, although it did not reach statistical significance (r = .03, p = .07).

Discussion and Conclusion

In crafting the instrument in this study, we considered an integration perspective that entailed the fusion of two theoretical frameworks. This approach involved a method of items pooling, where we wove together items from the instrument developed by Keller (2016) with selected items from questionnaires developed by Muthuprasad et al. (2021), Komolafe et al. (2020), and Dixson (2015). By employing this combined approach, we have created a comprehensive instrument that captures essential elements of instructional design and student learning experiences. The psychometric results of factorial structure and reliability lead us to the conclusion that the questionnaire has a good psychometric quality according to the empirical evidence analyzed. One proof of the suitability of this instrument is that it allows the measurement of differences by age, as the literature in this context has shown in previous research (Morin et al., 2019).

One of the remarkable features of this instrument is its ability to independently assess two distinct aspects of online education when measuring students’ motivation: motivation related to instructional materials and motivation associated with the dynamic interactions of teaching and learning practices in an online course. The latter includes direct communication with both peers and teachers, which is important in the context of “synchronous learning.”

The new contribution of this study is to move forward beyond the mere motivation with instructional online materials to a more complex understanding where interaction and relations are crucial in the teaching-and-learning process (H. Han & Johnson, 2012; Ong & Quek, 2023; Schmitz & Hanke, 2023). This aspect is of paramount importance for higher education institutions in Chile and other countries where distance education has evolved significantly since the global pandemic, driving higher education to adopt many of the teaching-and-learning digital activities in regular courses and creating new online programs (Morales Parada & Nova Rodríguez, 2021).

Our research underscores the positive potential of distance education when autonomous work is complemented by synchronous activities with teachers (Lee et al., 2021) and peers (Luo et al., 2017). Furthermore, it introduces fresh insights into the ongoing discussion about the definition of blended learning (b-learning), as there is a growing need to redefine the concept of “presence” in this context (Phirangee & Malec, 2017, Valverde-Berrocoso et al., 2020).

Within this context, a noteworthy argument emerges, suggesting that a positive learning experience can improve students’ Motivation, influencing their academic performance. If students positively perceive their learning experience, they are more likely to feel motivated to participate in learning activities and put more effort into their work. Previous research has shown that motivation is a key factor for success in online learning (Teng & Yang, 2023). One of the future challenges is to explore what happens to these experiences and their relationship to performance in learning tasks.

Furthermore, the learning experience is significantly related to students’ motivation (Kuo et al., 2014). In this sense, the results of this study suggest that in the specific context of online mathematics teaching and learning during the COVID-19 pandemic, the learning experience can positively impact students’ motivation. While it can be argued that motivation is also related to the learning experience, we suggest that future studies explore a causal relationship between these variables, seeking to test the hypothesis that if students are motivated to learn, they are more likely to pay attention to learning activities, feel more confident in their ability to learn, find the content more relevant, and feel satisfied with their learning.

An inherent limitation of this study lies in its temporal context. The results provide a robust stepping stone for future inquiries within distance education research, where self-learning materials converge with synchronous interactions with peers and teachers. In conclusion, such an instrument for measuring student motivation carries significant weight for educational institutions in Technical and Vocational Higher Education, especially in subjects like mathematics, where students’ motivation has been a concern (Saadati & Celis, 2022). In the post-pandemic era, where online education is anticipated to assume a more prominent role, it becomes crucial for institutions, especially those in the Technical and Vocational Higher Education sector, to evaluate the impact of online learning on their students’ experiences. Armed with this insight, they can design classrooms that integrate self-directed online interactions involving both teachers and peers, ultimately boosting student motivation and, consequently, enhancing academic performance.

Footnotes

Appendix

Questionnaire

Factor	Code	Question
Satisfaction (Sat.)	Sat.21	I really enjoyed studying the course
	Sat.27	In the course the delivery of feedback helped me feel that my effort was recognized.
	Sat.36	It was a pleasure to take a course as well designed as this one.
Relevance (Rel.)	Rel.18	The course provides examples of how to apply the knowledge taught in the course.
	Rel.23	The course material makes it feel worthwhile to learn the content.
	Rel.26	The course was relevant to my learning
	Rel.30	In the course I was able to relate the content to things I have seen, done or thought about in my life.
	Rel.16	The course content is relevant to my interests
Confidence (Con.)	Con.3	The course material was easier to understand than I had expected.
	Con.4	When I read the course syllabus, it was clear to me what I was supposed to learn
	Con.13	When I took the course I felt confident that I could learn the course content
	Con.34	I was able to understand much of the course material and content.
	Con.7	The amount of information in the course was adequate for me to remember what was most important.
Attention (Att.)	Ate.2	At the beginning of the course, we were presented with content and activities that caught my attention.
	Ate.8	The course materials were visually appealing
	Ate.11	The quality of the course material helped me to keep my attention
	Ate.12	The course was applicable to reality, and that helped me to stay focused.
	Ate.15	The course activities gave me an incentive to learn
	Ate.17	The way the course material was organized helped me maintain interest
Teacher-Centered Activities (TCA)	L.	The online classes have allowed me to learn enough
	CT	The online classes have allowed me to communicate well with my teachers.
	T	The online classes have encouraged me to organize my time well.
	E	I consider that the evaluations of the online classes have been adequate.
	P	In the online courses so far I have achieved the performance I wanted to achieve
	S	Through the online classes I believe that my computer and internet skills have improved.
	TAQ	In the online classes I have been able to solve my doubts thanks to my teachers.
Peer-Centered Activities	PC	The online classes have made it easier for me to get to know my classmates well.
	PQ	In the online classes I have been able to solve my doubts thanks to my classmates.
	KE	The online modality encouraged me to participate in exchanges with my classmates.
	PR	In the online modality, I achieved a collaborative relationship with my classmates.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the National Commission of Education (CNED) of Chile. Support from ANID/PIA/Basal Funds for Centers of Excellence FB0003 is gratefully acknowledged.

ORCID iD

Patricio Cabello

Author Biographies

Patricio Cabello, PhD, is an Assistant Professor and researcher at the Center for Advanced Studies in Education, Universidad de Chile. His research focuses on social and educational uses of digital media, collaboration, and online well-being.

Farzane Saadati, PhD, is a Researcher at the Center for Advanced Studies in Education, Universidad de Chile. Her research focuses on profesional development for mathematics teachers and problem solving

Pía Barahona is a research assistant at the Center for Advanced Studies in Education, Universidad de Chile. Her work focuses on collaboration, learning, and well-being.

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). STANDARDS for educational and psychological testing. https://www.testingstandards.net/uploads/7/6/6/4/76643089/standards_2014edition.pdf

Aristovnik

Keržič

Ravšelj

Tomaževič

Umek

(2020). Impacts of the COVID-19 pandemic on life of higher education students: A global perspective. Sustainability, 12(20), 8438. https://doi.org/10.3390/su12208438

Barger

Byrd

(2011). Motivation and computer-based instructional design. Journal of Cross-Disciplinary Perspectives in Education, 4(1), 1–9.

Bashir

Rana

Lambert

Vernallis

(2021). Post-COVID-19 adaptations; the shifts towards online learning, hybrid course delivery and the implications for biosciences courses in the higher education setting. Frontiers in Education, 6, 711619. https://doi.org/10.3389/feduc.2021.711619

Bellei

(2015). El gran experimento: Mercado y privatización de la educación chilena. LOM Ediciones.

Bostic

J. D.

Krupa

E. E.

Shih

J. C.

(Eds.). (2019). Quantitative measures of mathematical knowledge: Researching instruments and perspectives. Routledge.

Byrne

B. M.

(2016). Structural equation modeling with Amos: Basic concepts, applications, and programming (3rd ed.). Routledge, Taylor & Francis Group.

CNA. (2020). Guía para la Evaluación Externa Acreditación Institucional: Centros de Formación Técnica. https://www.cnachile.cl/SiteAssets/Lists/Acreditacion%20Institucional/AllItems/Gui%CC%81a%20para%20la%20evaluacio%CC%81n%20externa%20Centros%20de%20Formaci%C3%B3n%20T%C3%A9cnica.pdf

Dixson

M. D.

(2015). Measuring student engagement in the online course: The online student engagement scale (OSE). Online Learning, 19(4), 1–15.

10.

Douglas

Slusser

Felton

(2023). Academic discourse and peer collaboration in online high school learning environments. Journal of Computer Assisted Learning, 39(5), 1479–1492. https://doi.org/10.1111/jcal.12809

11.

Eccles

J. S.

Wigfield

(2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153

12.

Gulbahar

(2008). ICT usage in higher education: A case study on preservice teacher and instructions. Turkish Online Journal of Educational Technology, 7(1), 32–37.

13.

Hamora

Rabaya

Pentang

Pizaña

Gamozo

M. J.

(2022). Students’ evaluation of faculty-prepared instructional modules: Inferences for instructional materials review and revision. Journal of Education, Management and Development Studies, 2(2), 20–29. https://doi.org/10.52631/jemds.v2i2.109

14.

Han

Johnson

S. D.

(2012). Relationship between students? Emotional intelligence, social bond, and interactions in online learning. Educational Technology & Society, 15(1), 78–79.

15.

Han

Yin

Wang

(2018). A case study of faculty perceptions of teaching support and teaching efficacy in China: Characteristics and relationships. Higher Education, 76(3), 519–536. https://doi.org/10.1007/s10734-017-0223-0

16.

Inan

D. I.

Hidayanto

A. N.

Juita

Hasian

C. Y.

Luvian

Leonardo Ian

S. L.

Pratama

(2023). How personal, technical, social environments affecting generation Z to utilise video-based sharing platform in learning process during crisis? Research and Practice in Technology Enhanced Learning, 19,3. https://doi.org/10.58459/rptel.2024.19003

17.

Keller

J. M.

(2008). First principles of motivation to learn and e3-learning. Distance Education, 29(2), 175–185. https://doi.org/10.1080/01587910802154970

18.

Keller

J. M.

(2010). Motivational design research and development. In Keller

J. M.

(Ed.), Motivational design for learning and performance (pp. 297–323). Springer US.

19.

Keller

J. M.

(2016). Motivation, learning, and technology: Applying the ARCS-V motivation model. Participatory Educational Research, 3(2), 1–15. https://doi.org/10.17275/per.16.06.3.2

20.

Kline

R. B.

(2010). Principles and practice of structural equation modeling (3rd ed.). Guilford Press.

21.

Komolafe

B. F.

Fakayode

O. T.

Osidipe

Zhang

Qian

(2020). Evaluation of online pedagogy among higher education international students in China during the COVID-19 outbreak. Creative Education, 11(11), 2262–2279. https://doi.org/10.4236/ce.2020.1111166

22.

Kuo

Y. C.

Walker

A. E.

Schroder

K. E. E.

Belland

B. R.

(2014). Interaction, Internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. The Internet and Higher Education, 20, 35–50. https://doi.org/10.1016/j.iheduc.2013.10.001

23.

Lee

Hoe Looi

Faulkner

Neale

(2021). The moderating influence of environment factors in an extended community of inquiry model of e-learning. Asia Pacific Journal of Education, 41(1), 1–15. https://doi.org/10.1080/02188791.2020.1758032

24.

Liao

H.-C.

Wang

(2011). Applying the ARCS motivation model in technological and vocational education. Contemporary Issues in Education Research (CIER), 1(2), 53. https://doi.org/10.19030/cier.v1i2.1202

25.

Louvigné

Uto

Kato

Ishii

(2017). Social constructivist approach of motivation: Social media messages recommendation system. Behaviormetrika, 45, 133–155. https://doi.org/10.1007/s41237-017-0043-7

26.

Luo

Zhang

(2017). Effects of different interactions on students’ sense of community in e-learning environment. Computers & Education, 115, 153–160. https://doi.org/10.1016/j.compedu.2017.08.006

27.

Makovec Radovan

Radovan

. (2023). Teacher, think twice: About the importance and pedagogical value of blended learning design in VET. Education Sciences, 13(9), 882. https://doi.org/10.3390/educsci13090882

28.

Miao

Chang

(2022). Teacher–student interaction, student–student interaction and social presence: Their impacts on learning engagement in online learning environments. The Journal of Genetic Psychology, 183(6), 514–526. https://doi.org/10.1080/00221325.2022.2094211

29.

MINEDUC. (2021). Informe 2021. Matrícula de pregrado en Educación Superior. Subsecretaría de Educación Superior. https://www.mifuturo.cl/wp-content/uploads/2021/06/Informe-Matricula-Pregrado-2021_SIES.pdf

30.

MINEDUC-UNESCO. (2017). Estrategia Nacional de Formación Técnico-Profesional. Estrategia Nacional de Formación Técnico-Profesional. http://www.tecnicoprofesional.mineduc.cl/wp-content/uploads/2018/03/Estrategia-Nacional-de-Formación-Técnico-Profesional.pdf

31.

Morales Parada

Nova Rodríguez

. (2021). Percepción de la educación a distancia de emergencia en año de COVID-19 en carreras de auditoría de universidades chilenas. Revista De Investigación Aplicada En Ciencias Empresariales, 10(1), 54–73. https://doi.org/10.22370/riace.2021.10.1.2977

32.

Morin

Safaee Fard

George Saadé

(2019). Understanding online learning based on different age categories. Issues in Informing Science and Information Technology, 16, 307–317. https://doi.org/10.28945/4313

33.

Muianga

Hansson

Suhonen

(2019). The role of ICT in the shift towards student-centred learning in higher education Eduardo Mondlane University, Mozambique: A case study. Department of Computer and Systems Sciences, Stockholm University.

34.

Muthuprasad

Aiswarya

Aditya

K. S.

Jha

G. K.

(2021). Students’ perception and preference for online education in India during COVID -19 pandemic. Social Sciences & Humanities Open, 3(1), 100101. https://doi.org/10.1016/j.ssaho.2020.100101

35.

Ong

S. G. T.

Quek

G. C. L.

(2023). Enhancing teacher–student interactions and student online engagement in an online learning environment. Learning Environments Research, 26(3), 681–707. https://doi.org/10.1007/s10984-022-09447-5

36.

Phirangee

Malec

(2017) Othering in online learning: an examination of social presence, identity, and sense of community. Distance Education, 38(2), 160–172, https://doi.org/10.1080/01587919.2017.1322457

37.

Zhang

Yang

(2023). A familiar peer improves students’ behavior patterns, attention, and performance when learning from video lectures. International Journal of Educational Technology in Higher Education, 20(1), 47. https://doi.org/10.1186/s41239-023-00418-1

38.

Reynolds

K. M.

Roberts

L. M.

Hauck

(2017). Exploring motivation: Integrating the ARCS model with instruction. Reference Services Review, 45(2), 149–165. https://doi.org/10.1108/RSR-10-2016-0057

39.

Rios

Wells

(2014). Validity evidence based on internal structure. Psicothema, 26(1), 108–116. https://doi.org/10.7334/psicothema2013.260

40.

Rodríguez-Triana

M. J.

Prieto

L. P.

Ley

De Jong

Gillet

(2020). Social practices in teacher knowledge creation and innovation adoption: A large-scale study in an online instructional design community for inquiry learning. International Journal of Computer-Supported Collaborative Learning, 15(4), 445–467. https://doi.org/10.1007/s11412-020-09331-5

41.

Saadati

Celis

(2022). Student motivation in learning mathematics in technical and vocational higher education: Development of an instrument. International Journal of Education in Mathematics, Science and Technology, 11(1), 156–178. https://doi.org/10.46328/ijemst.2194

42.

Schmitz

Hanke

(2023). Engage me: Learners’ expectancies and teachers’ efforts in designing effective online classes. Journal of Computer Assisted Learning, 39(4), 1132–1140. https://doi.org/10.1111/jcal.12636

43.

Shankar

Arora

Binz-Scharf

M. C.

(2023). Evidence on online higher education: The promise of COVID-19 pandemic data. Management and Labour Studies, 48(2), 242–249. https://doi.org/10.1177/0258042X211064783

44.

Sivan

(1986). Motivation in social constructivist theory. Educational Psychologist, 21(3), 209–233.

45.

Teng

M. F.

Yang

(2023). Metacognition, motivation, self-efficacy belief, and English learning achievement in online learning: Longitudinal mediation modeling approach. Innovation in Language Learning and Teaching, 17(4), 778–794. https://doi.org/10.1080/17501229.2022.2144327

46.

Tondeur

Cooper

Newhouse

C. P.

(2010). From ICT coordination to ICT integration: A longitudinal case study: ICT coordination in primary education. Journal of Computer Assisted Learning, 26(4), 296–306. https://doi.org/10.1111/j.1365-2729.2010.00351.x

47.

Trespalacios

Snelson

Lowenthal

P. R.

Uribe-Flórez

Perkins

(2021). Community and connectedness in online higher education: A scoping review of the literature. Distance Education, 42(1), 5–21. https://doi.org/10.1080/01587919.2020.1869524

48.

Vadakalu Elumalai

Sankar

J. P.

Kalaichelvi

Ann John

Menon

Salem

Alqahtani

Abdulaziz Abumelha

. (2020). Factors affecting the quality of E-learning during the COVID-19 pandemic from the perspective of higher education students. Journal of Information Technology Education: Research, 19, 731–753. https://doi.org/10.28945/4628

49.

Valiente

Zancajo

Jacovkis

(2020). The coordination of skill supply and demand in the market model of skill formation: Testing the assumptions for the case of Chile. International Journal of Lifelong Education, 39(1), 90–103. https://doi.org/10.1080/02601370.2019.1678692

50.

Valverde-Berrocoso

Garrido-Arroyo

M. D. C.

Burgos-Videla

Morales-Cevallos

M. B.

(2020). Trends in educational research about e-learning: A systematic literature review (2009–2018). Sustainability, 12(12), 5153.

51.

Wang

Wen

Quek

C. L.

(2023). Engaging learners in synchronous online learning. Education and Information Technologies, 28(4), 4429–4452.

52.

Wigfield

Cambria

(2010). Students’ achievement values, goal orientations, and interest: Definitions, development, and relations to achievement outcomes. Developmental Review, 30(1), 1–35. https://doi.org/10.1016/j.dr.2009.12.001

53.

Zapata

Tejeda

(2016). Publicidad engañosa en educación superior: Definiciones, regulación y una revisión de la publicidad en prensa escrita. Calidad En La Educación, 44, 197–242. https://doi.org/10.4067/S0718-45652016000100008

54.

Zhang

S.-N.

Y.-Q.

Liu

C.-H.

Ruan

W.-Q.

(2020). Critical factors identification and prediction of tourism and hospitality students’ entrepreneurial intention. Journal of Hospitality, Leisure, Sport & Tourism Education, 26, 100234. https://doi.org/10.1016/j.jhlste.2019.100234