Effectiveness of online learning in non-online classes during the pandemic

Abstract

The coronavirus disease 2019 (COVID-19) pandemic that has been going on since the beginning of 2020 has forced governments to take a policy to implement online learning. This policy is an effort to break the chain of virus spread in education clusters. For schools and universities that are not used to carrying out online learning, this condition is a problem. Therefore, measurement is needed to ensure effectiveness. In this study, the effectiveness was measured using the variables specified in the Indonesian Online Learning System (Sistem Pembelajaran Daring (SPADA)) 2019 set by the Indonesian government. Data collection was carried out by quantitative and qualitative methods. Quantitative analysis was carried out through questionnaires administered to students while qualitative was carried out by interviewing students and lecturers. The data were processed using cross-tabulation analysis and multiple linear regression. Strengths, Weaknesses, Opportunities, and Challenges were analyzed to assess online learning. As research results, it is found that online learning is quite effective, but there are still many things that can be improved to increase the effectiveness of online learning.

Keywords

COVID-19 effectiveness online learning SWOC university

Introduction

The coronavirus disease 2019 (COVID-19) was a problem faced by almost all countries in the world. With the severity of the consequences caused by COVID-19, the governments of countries have taken actions, such as imposing large-scale social restrictions and lockdowns.

Indonesia is a country in Southeast Asia, which has also been affected by the COVID-19 pandemic. Therefore, the Indonesian government has also taken large-scale social distancing. Face-to-face meetings and activities of students were prohibited in colleges and universities. Activities for all students are now conducted online. Initially, staff and lecturers were still permitted to conduct activities on campus or at school, but after a certain amount of time, the campus was completely shut down (Ministry of Higher Education of the Republic of Indonesia, 2020). This decision was unexpected and presented challenges for classes that had been doing face-to-face learning (non-online learning) before the pandemic.

This shift made the institutions worried about whether the learning process was as effective as face-to-face learning. This is a problem since the institution never used online learning; as a result, there are no supporting guidelines, and no ready-to-use equipment and neither the lecturers nor the students have the necessary abilities to deal with it.

X University's Industrial Engineering Study Program is one study program that initially uses non-online learning methods. This study program's learning method suddenly needs to change to online learning. There is a need to research how successful online learning is and how institutions and teachers prepare for this abrupt transformation in light of all the limitations and challenges. This research utilized the study program in question as a case under such circumstances.

Various studies have been carried out on the effectiveness of online learning, both for classes that are indeed conducted online (such as Buttner & Black, 2014; Means et al., 2013), as well as classes that were originally non-online but were forced to go online (such as Agarwal & Dewan, 2020; Baber, 2022; Satyawan et al., 2021). Research has also been carried out to identify barriers to the effectiveness of online learning, (such as Heng & Sol, 2021; Ramli et al., 2020) and the positive outcomes resulting from online learning (Lorenzetti, 2013; Nguyen, 2015). Some studies compare online learning with non-online learning methods (such as Chang et al., 2021; Darkwa & Antwi, 2021).

Even though there have been numerous studies on online learning, research on the effectiveness of online learning for initially non-online classes is still scarce. Additionally, neither publications discussing how the institution made this abrupt transition nor studies on the efficiency of online learning based on the five components of the government-mandated Indonesian Online Learning System (Sistem Pembelajaran Daring (SPADA)) could be discovered. This study helps evaluate the efficacy of online learning that is implemented abruptly, figuring out how to make it better, and knowing how efforts are being made for this improvement.

Literature review

Several researchers have proven that online learning classes opened are effectively conducted learning processes (Buttner & Black, 2014; Means et al., 2013; Nguyen, 2015). Some researchers have found the positive things obtained from online learning, namely cost-effective implementation, ease to form world class (Lorenzetti, 2013), enhances better students’ test scores and access to learning materials, improves learning perceptions, reduces failures, and improved relations between students (Nguyen, 2015).

However, institutions in Indonesia were obliged to switch from face-to-face study to online learning almost immediately. This occurred unexpectedly, and no plans for online learning had been established. Considering the implementation of online learning which is very different from face-to-face learning, there are concerns about online learning's effectiveness.

Research has been carried out before by several researchers and it is found that online learning is effective (Agarwal & Dewan, 2020; Baber, 2022; Satyawan et al., 2021; Suprianto et al., 2020), However, research on the efforts undertaken to effect change is still limited. On the other hand, other academics point out challenges with online learning, including lack of digital skills, socioeconomic variables, workload, evaluation and supervision, and incompatibility with certain courses (Heng & Sol, 2021). Other researchers find that online learning is boring, unengaging, needs personal attention, lack practical experience (Dhawan, 2020), and causes stress for students (Argaheni, 2020). However, no article was found using the five aspects of the SPADA.

SPADA is a program of the Indonesian government to increase equitable access to quality learning in Higher Education (Sekretariat Direktorat Jenderal Pendidikan Tinggi Kementrian Pendidikan dan Kebudayaan, 2015). SPADA was designed to support Independent Learning Independent Campus called MBKM sets up by the Indonesian government. This MBKM encourages students to learn outside of the classroom, whether on the campuses of other institutions, corporations, or even in the community.

Methodology

As described earlier, the assessment of online learning performance is carried out based on five aspects of SPADA. The five aspects of the performance consist of:

Lesson plan.

Learning activities.

Delivery/delivery strategy.

Learning media and technology.

Study assistance services.

The Likert scale was employed as shown in Table 1 to evaluate the elements and efficacy. As shown in Table 2, these 5 factors are applied to 28 variables. A questionnaire is created using these factors. While four criteria are used to evaluate the efficacy of online learning as shown in Table 3.

Table 1.

Likert scale.

Score weight	4	3	2	1
Performance rate	Strongly agree	Agree	Disagree	Strongly disagree
Effectivity	Strongly agree	Agree	Disagree	Strongly disagree

Table 2.

Online learning performance.

Var. No.	Variable statements	Score
X1	Online learning materials are oriented toward student independence	3.19
X2	Online learning materials emphasize student creativity and innovation in learning	3
X3	Online learning materials are arranged systematically	3.01
X4	Online learning materials are in accordance with the predetermined learning plan	3.08
X5	Online learning is carried out according to a predetermined schedule	3.25
X6	It is easy to collect assignments/reports online	3.12
X7	Online group work can be done easily	2.31
X8	Presentation of assignments online is easy	2.91
X9	The online exam process can be done easily	2.89
X10	The process of monitoring and evaluating online learning can guarantee the quality of learning	2.64
X11	The online learning plan for one semester is clearly conveyed at the beginning of the lecture	3
X12	Online learning materials are presented in an interesting way	2.75
X13	Online learning materials are delivered clearly	2,89
X14	The interaction between lecturers and students in online learning takes place actively	2.81
X15	Delivery of online learning materials (lectures/practicum) facilitates students to learn actively (practice questions, case studies, discussions)	2.99
X16	The lecturer/assistant has an adequate level of technology and media mastery	3.12
X17	Lecturers always accompany during online learning	3.03
X18	Feedback from lecturers in online learning can be used by students to better understand the lecture material	2.99
X19	Online learning provides an opportunity to reflect on material understanding through self-assessment	2,95
X20	The characteristics of the material delivered are in accordance with the online learning media used	3,04
X21	Easy-to-use online learning platform	2.93
X22	Online learning media (Whatsapp, Line, Google Meet, Zoom, etc.) can be used easily	3.29
X23	Online learning materials can be downloaded easily	3.24
X24	Administration provides administrative service support during online learning	3.08
X25	Lecturers provide support in overcoming difficulties in online learning	3
X26	Information about student learning progress and success can be accessed easily in online learning	3
X27	Study program provides a means of submitting input/complaints about students related to online learning	2.97
X28	Internet quota is always available when participating in online learning	2.83
Average		2.93

Table 3.

Online learning performance effectiveness.

Var. No.	Variable statements	Score
Y1	Online learning improves my understanding of course material	2.52
Y2	Online learning increases my motivation to study	2.43
Y3	Online learning increases my independence in learning	2.99
Y4	Online learning increases my opportunities to actively participate in learning activities	2.72
Average		2.66

Seventy-five Industrial Engineering students from the Class of 2017–2020 were given surveys to complete using Google Forms. The survey asks about student characteristics as well as the usefulness and performance of online learning.

Multiple linear regression was used to examine the effectiveness of online learning and performance evaluation. The strength, weekness, opportunity, challenge (SWOC) analysis was then carried out based on the outcomes of the multiple linear regression and the cross-tabulation analysis. Five students, three professors, and three university administrators were interviewed as part of studies on how institutions adopt change. The suggestion for improvement was based on the SWOC study. The research was conducted following the Figure 1 flow diagram.

Measurement of online learning performance is carried out using the following formula:

\bar{X} = \frac{\sum_{i = 1}^{k} X_{i}}{n}

(1)

where

$\bar{X}$ = average score of performance level;

$X_{i}$ = the score of ith respondent's performance level;

n = number of respondents.

To find out the online learning performance variables that affect the effectiveness, the multiple linear regression method is used with the following equation:

Y = a + b_{1} X_{1} + b_{2} X_{2} \dots + b_{n} X_{n}

(2)

where

Y = online learning process effectivity;

$X_{1}, X_{2}, \dots, X_{n}$ = online learning performance variables;

a = constant;

$b_{1}, b_{2}, \dots, b_{n}$ = regression coefficient.

SWOC analysis can be used to analyze Strengths, Weaknesses, Opportunities, and Challenges (Dhawan, 2020). Interviews with both students and lecturers were performed as part of the SWOC study. Based on all the data acquired, improvement ideas were developed.

Result and discussion

The results of the questionnaires filled out by respondents, namely students of the Industrial Engineering Study Program can be seen in Tables 2 and 3.

Validity and reliability test

The measurement device's accuracy was validated by the validity test. All construct scores were legitimate since they were positively correlated with the overall question score (count r value > r table value). The accuracy of the measurement instrument was examined using the Cronbach’s alpha (α) statistical test. Both questionnaires’ scores—0.915 for the learning process performance evaluation questionnaire and 0.845 for the learning effectiveness assessment questionnaire—indicate their reliability.

Classic assumption test

Classical assumption tests need to be carried out to ensure the validity of the use of multiple linear regression which consists of:

The normality test: The significant value Asym. Sig. (two-tailed) of 0.842 > 0.05, indicating that the data is normally distributed.

The multicollinearity test: The value of the variance inflation factor (VIF) of all independent variables is less than 10, hence there is no multicollinearity.

A heteroscedasticity test: There are two variables (X25 and X5) that have a significant value p < 0.05, which indicates that heteroscedasticity occurs, then the two variables are excluded.

A linearity test: From the test results, there are four variables (X4, X9, X11, and X14) that have deviation from linearity significant values p < 0.05, which indicates that there are no significant linear correlations, then these variables are excluded.

Multiple linear regression

To determine the performance factors of the online learning process that influence learning effectiveness, multiple regression analysis was used. The average efficacy of the online learning process is utilized as the dependent variable as shown in Table 3 while the 22 performance characteristics for the online learning process are used as independent variables as shown in Table 2.

The effect of the independent variables together on the dependent variable is carried out by conducting a simultaneous significant test (F-test), with the following hypothesis formulation:

H₀: The regression coefficient is not significant.

H₁: The regression coefficient is significant.

The level of significance (α) used is 5%.

Decision-making criteria:

Reject H₀ if the significance value p ≤ 0.05.

Accept H₀ if the significance value p > 0.05.

Based on the processing results, the value of significance of 0.000 (p < 0.05), so the decision is to reject $H_{0}$ . This means that the regression coefficient is significant, so the performance of the online learning process has a significant effect on the effectiveness of learning.

The effect of each independent variable on the dependent variable is carried out by performing a partial significant test (t-test) with the following hypothesis formulation:

$H_{0}$ : the online learning process performance variable does not affect the effectiveness of online learning.

$H_{1}$ : the online learning process performance variable does affect the effectiveness of online learning.

Based on the results of processing, the values of significance for X19, X7, and X20 are less than 0.025, then it is declared to have a significant effect on the dependent variable, with the following regression equation:

Y = 0.011 + 0.388 X 19 + 0.249 X 7 + 0.308 X 20

(3)

Information:

Y = effectiveness of online learning.

0.011 = constant.

Three variables have a direct impact on how successful online learning is according to the multiple regression study. These factors can account for up to 50.2% of the efficiency of online learning (shown by the adjusted R2 value). This shows that 49.8% are still excluded and provides a chance for more study. The dependent variable and independent variable have a significant positive relationship as indicated by multiple correlation values (R) of 0.723. These three influencing factors may be taken into account to increase the efficiency and efficacy of online learning.

Cross-tabulation analysis (crosstab)

Seventy-five students from the 2017 to 2020 academic year completed the questionnaire; their profiles are shown in Figure 2. As a probability value of 0.809 > 0.05, there is no correlation between the class and the efficiency of online learning.

The proportion of the GPA of students who fill out the questionnaire is shown in Figure 3. Cross-tabulation analysis was performed between this data and the effectiveness of online learning, which yielded the chi-square probability value of 0.451 > 0.05. That means that there is no correlation between them.

Figure 4 shows the online learning media chosen by students. Video conference is the most popular. The chi-square with the online learning efficacy was 0.48 > 0.05. This demonstrates that there is no correlation between them.

As shown in Table 4, the factors X19, X7, and X20 that the multiple regression analysis indicated had an impact were used in the cross-tabulation to determine whether there were variations in effectiveness across classes. Only X7 (ease of working in groups) of these factors indicates a difference between groups (probability value: 0.04–0.05). Because they have not had much time to get to know one another, new students who are required to work in groups online face more challenges. Students who have been in college for 2 to 3 years have no or little challenges because they already know one another, but students who have just started their second year are still having some difficulty.

Table 4.

The results of the cross-tabulation of influential variables.

		X19					X7					X20
		1	2	3	4	Total	1	2	3	4	Total	1	2	3	4	Total
CLASS	2017
	Count	0	2	11	3	16	2	5	6	3	16	1	1	11	3	16
	Peecentage of total	0.0	2.7	14.7	4.0	21.3	2.7	6.7	8.0	4.0	21.3	1.3	1.3	14.7	4.0	21.3
	2018
	Count	0	4	20	3	27	9	11	6	1	27	0	3	19	5	27
	Percentage of total	0.0	5.3	26.7	4.0	36.0	12.0	14.7	8.0	1.3	36.0	0.0	4.0	25.3	6.7	36.0
	2019
	Count	1	1	8	0	10	0	2	8	0	10	0	1	9	0	10
	Percentage of total	1.3	1.3	10.7	0.0	13.3	0.0	2.7	10.7	0.0	13.3	0.0	1.3	12.0	0.0	13.3
	2020
	Count	0	2	19	1	22	4	10	6	2	22	0	1	18	3	22
	Percentage of total	0.0	2.7	25.3	1.3	29.3	5.3	13.3	8.0	2.7	29.3	0.0	1.3	24.0	4.0	29.3
	Total
	Count	1	9	58	7	75	15	28	26	6	75	1	6	57	11	75
	Percentage of total	1.3	12.0	77.3	9.3	100	20.0	37.3	34.7	8.0	100	1.3	8.0	76.0	14.7	100
	Pearson chi-square Asymp. Sig. (two-sided)	0.32					0.04					0.64

SWOC analysis

According to the findings of a study on change implementation efforts, institutions look for the most suitable online learning platform when there are rapid changes. Trial and error were used to conduct this search. The most appropriate and effective platform can eventually be identified. The learning management system (LMS) and online learning platforms are quickly mastered by lecturers, who subsequently turn their lectures into videos or narrated slides and deposit them there. Additionally, instructors strive to make their lectures as effective as in-person instruction. Additionally, the platform and LMS had to be learned by the students, and they had to adjust to any new procedures that emerged. The college promotes online learning by creating different policies and providing a more adequate LMS.

The SWOC analysis was performed to determine the Strengths, Weaknesses, Opportunities, and Challenges associated with the present online learning process. The study was based on the value and efficacy of online learning, as well as the findings of multiple linear regression analysis. Classification of value quality is carried out based on the established criteria as shown in Table 5.

Table 5.

Quality criteria.

Range	Quality
1 ≤ value < 1.75	Very bad
1.75 ≤ value < 2.5	Bad
2.5 ≤ value < 3.25	Good
3.25 ≤ value ≤ 4	Very good

Strength

The average score of online learning performance is 2.93 (good). This can be seen as a strength. However, the overall achievement of the variables is still far from the maximum value, so there is still an opportunity for improvement.

In terms of the X5 variable, the findings of the interviews with students and instructors show that online lectures are implemented 50% more promptly than face-to-face lectures. First, concerning the X22 variable, the LMS has not been actively socialized or emphasized as a tool for learning source materials for face-to-face interaction. But in the second semester, instructors and students received training on how to utilize it. The creation of learning resources suitable for online courses and their availability on a student-accessible LMS were prerequisites for lecturers.

Variables X1, X3, X4, X6, X16, X17, X20, and X24 gain scores above 3, which means that they are considered very good by students. The learning materials have been specifically designed for online learning. Lecturers have studied technology and media related to online learning so that online learning can be carried out properly. On the other hand, administrative officers are also prepared to support online learning implementation.

Weaknesses

Based on student assessments, there is a variable that is considered bad, namely X7. One of the important things for Industrial Engineering graduates is to be able to work in groups. Therefore, many activities are conducted in group work to learn to lead, be led, communicate, relate, make decisions together, resolve conflicts, etc. Working in a group is uncomfortable for students, especially new ones since the learning must be done entirely online. Since group projects were a regular part of in-person lectures, this surprised the students. They complain about the hefty quota fees, the unavailability of gadgets, and the fact that the students did not know each other yet. Additionally, because group members occasionally engage in other activities, the dynamics of the workgroup are occasionally restricted.

Opportunity

The effectiveness of online learning led to the possibility of hybrid or online learning in the future. Because instructors produced learning materials and consistently uploaded them to the LMS, some students can even achieve superior results while learning online. Although it is still a required LMS from the government, SPADA is still mandatory, with superior and more efficient. The information is freely accessible to the students, who may then discuss it with the lecturers.

By proving that online learning was effective and also becoming support for universities in Indonesia to conduct MBKM. The MBKM program is supported because:

Students can take lessons in far locations.

Allow the university to collaborate with other parties, for example, micro, small, and medium enterprises (MSMEs) to be involved in lectures and businesses.

University can outreach students in remote areas.

Cooperation with overseas universities can easily be accommodated.

Challenges

In contrast to the opportunities previously outlined, several obstacles materialized. On the student side, signal, quota, and gadget availability are issues that many students encounter. The third issue is that students’ eye health is impacted due to prolonged and frequent usage of gadgets (such as Mohan et al., 2021; Pachiyappan et al., 2021). Next, issues related to motivational issues arise. The informal learning environment at home makes learning less efficient. The last difficulty is the tension that students undergo since they are unable to socialize and communicate with peers.

For lecturers, online learning is tough because they must quickly create relevant learning materials, and develop ways to conduct lectures while missing out on interaction with students. While the challenge for the study program is to design practicums so that they can be done online, rather than in a laboratory where students get experience by doing experiments.

Online learning has the potential to replace or mix with the traditional classroom in the future, given its advantages, strengths, and effectiveness. With the shortcomings and difficulties come the need to solve them, particularly for Indonesian universities that must swiftly switch from in-person to online instruction. There are other options, including:

To carry out group assignments, lecturers must provide a time slot. Students get clarity of time and lecturers can supervise them.

Planning synchronous learning, and assignments in ways that improve relations between students, for example, by holding games, providing time to chat together, etc.

Allocate funds to help students who are less well off financially. With online learning, the university's operational costs have decreased. These cost savings should be diverted to help students in need of online learning support equipment.

Create work units to help lecturers prepare learning materials, such as making interesting learning videos, presentation slides.

The study program is exploring creating a virtual reality laboratory so that students can still get an experience like in a laboratory.

These concepts can be used in locations or nations other than Indonesia that have comparable settings and situations.

Conclusion

Online education has become essential as a result of the COVID-19 epidemic. The unexpected necessity to perform online learning caused some distress and misunderstanding at the beginning of the epidemic, but these issues may eventually be handled. However, online learning has proven to be a challenge for students, lecturers, study programs, and administrative personnel. Online and hybrid education can then be alternatives to learning methods.

Inadvertently, the epidemic that forces online learning speeds up the use of digital technologies in the classroom. The employment of digital technology in online learning can be advantageous for the Indonesian government's MBKM, international, and outreach activities.

Further study may be focused on searching for other variables that impact learning effectiveness as the variables discovered can only account for 50.2% of the variance in learning efficiency. More research may be conducted by reassessing the learning efficacy once the online learning system has been improved and put into use. Additional research may be done to examine certain student difficulties including motivation, well-being, and stress levels.

Footnotes

Acknowledgments

The authors would like to thank Universitas Kristen Maranatha for contributing to this research. The authors would also like to thank Ms Prameshwara Anggahegari for proofreading this manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank Maranatha Christian University for funding this research.

ORCID iDs

Jimmy Gozaly

Yulianti Talar

Christina Wirawan

Author biographies

Jimmy Gozaly is a senior lecturer who actively teaches at the Bachelor Program of Industrial Engineering, Maranatha Christian University, with a scientific field of Industrial Management. He currently serves as Head of the Manufacturing Processes Laboratory. He studied Bachelor of Industrial Engineering at Maranatha Christian University and earned his Master's degree in Industrial Engineering at Bandung Institute of Technology. His research interest is in topics related to additive manufacturing technology, marketing, and quality improvement of learning.

Yulianti Talar is a senior lecturer who actively teaches at the Bachelor Program of Industrial Engineering, Maranatha Christian University, with a scientific field of Industrial Management. She currently serves as Head of the Industrial Management and Systems Design Laboratory. She studied Bachelor of Industrial Engineering at Maranatha Christian University and earned her Master's degree in Industrial Engineering at Bandung Institute of Technology. Her research interest is in topics related to marketing and quality improvement of learning.

Christina Wirawan is a lecturer at Industrial Engineering, Maranatha Christian University. She earned her doctorate from the School of Business Management, Bandung Institute of Technology. Her research interest includes industrial estate firm, product design, service science, quality engineering, and entrepreneurship. Currently, she likes to combine engineering study with social and management science in her research.

Axel Varian Kurniawan is a practitioner who actively works at PT. Mayora Indah, Tbk., with passion and knowledge of Industrial Management. He currently serves as Supply Planner Overseas. He studied Bachelor of Industrial Engineering at Maranatha Christian University. His research interest is in service quality improvement and supply chain management.

References

Agarwal

Dewan

(2020) An Analysis of the Effectiveness of Online Learning in Colleges of Uttar Pradesh During the COVID-19 Lockdown. Journal of Xi’an University of Architecture & Technology 12(5): 2957–2963.

Argaheni

(2020) A Systematic Review: The Impact of Online Lectures During the COVID-19 Pandemic Against Indonesian Students. PLACENTUM: Jurnal Ilmiah Kesehatan Dan Aplikasinya 8(2): 99–108.

Baber

(2022) Social Interaction and Effectiveness of the Online Learning – A Moderating Role of Maintaining Social Distance During the Pandemic COVID-19. Asian Education and Development Studie 11(1): 159–171. Retrieved from: http://www.anitacrawley.net/Articles/OnlinelearningSocialinteractionandthecreation.pdf#page = 78.

Buttner

Black

(2014) Assessment of the Effectiveness of an Online Learning System in Improving Student Test Performance. Journal of Education for Business 89(5): 248–256. Retrieved from: http://observatorio.epacartagena.gov.co/wp-content/uploads/2017/08/metodologia-de-la-investigacion-sexta-edicion.compressed.pdf

Chang

JY-F

Wang

Lin

, et al. (2021) Comparison of Learning Effectiveness Between Physical Classroom and Online Learning for Dental Education During the COVID-19 Pandemic. Journal of Dental Sciences 16(4): 1281–1289.

Darkwa

Antwi

(2021) From Classroom to Online: Comparing the Effectiveness and Student Academic Performance of Classroom Learning and Online Learning. Open Access Library Journal 8: 1–22.

Dhawan

(2020) Online Learning: A Panacea in the Time of COVID-19 Crisis. Journal of Educational Technology Systems 49(1): 5–22.

Heng

Sol

(2021) Online Learning During COVID-19: Key Challenges and Suggestions to Enhance Effectiveness. Cambodian Education Research Journal 1(1): 3–16.

Lorenzetti

(2013) Academic administration-running a MOOC: Secrets of the world’s largest distance education classes. Wisconsin: Magna Publication.

10.

Means

Toyama

Murphy

, et al. (2013) The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature. Teachers College Record 115(3): 1–47.

11.

Ministry of Higher Education of the Republic of Indonesia (2020) Circular of the Director-General of Higher Education of the Republic of Indonesia Number 1.

12.

Mohan

Sen

Shah

, et al. (2021) Prevalence and Risk Factor Assessment of Digital eye Strain among Children Using Online e-Learning During the COVID-19 Pandemic: Digital eye Strain among Kids (DESK Study-1). Indian Journal of Ophthalmology 69(1): 140.

13.

Nguyen

(2015) The Effectiveness of Online Learning: Beyond no Significant Difference and Future Horizons. MERLOT Journal of Online Learning and Teaching 11(2): 309–319.

14.

Pachiyappan

Kumar

Mark

, et al. (2021) Effects of Excessive Usage of Electronic Gadgets During COVID-19 Lockdown on Health of College Students: An Online Cross-Sectional Study. Asian Journal of Pharmaceutical Research and Health Care 13(2): 139–145.

15.

Ramli

Majid

Badyalina

(2020) Impeding Factors Towards the Effectiveness of Online Learning During Covid-19 Pandemic among Social Sciences Students. International Journal of Learning and Development 10(4): 37–49.

16.

Satyawan

Wahjoedi

Swadesi

IKI

(2021) The Effectiveness of Online Learning Through Undiksha E-Learning DurSatyawan. . Journal of Education Technology 5(2): 191–199.

17.

Sekretariat Direktorat Jenderal Pendidikan Tinggi Kementrian Pendidikan dan Kebudayaan (2015) SPADA Indonesia. Retrieved February 25, 2022, from: https://spada.kemdikbud.go.id/berita/apa-itu-spada-indonesia#:∼:text=SPADA Indonesia adalah kependekan dari Sistem Pembelajaran Daring Indonesia.

18.

Suprianto

Arhas

Mahmuddin

, et al. (2020) The Effectiveness of Online Learning Amid the COVID-19 Pandemic. Jurnal Administrare: Jurnal Pemikiran Ilmiah Dan Pendidikan Administrasi Perkantoran 7(2): 321–330.