Metacognitive skills in low-code app development: Work-integrated learning in information systems development

Work-integrated learning with low-code platforms

WIL is an educational approach that 'integrate[s] theory with the meaningful practice of work as an intentional component of the curriculum' (Wood et al., 2020, p. 331). Students draw on theoretical concepts through a purposefully designed curriculum to solve practical problems. WIL focuses on authentic and meaningful work tasks that demand timely completion (Patrick et al., 2009). The integration of theory and practice seeks to emulate the functions of a workplace to provide real-life experiences (Jackson, 2017). Consulting projects are often undertaken in which students, industry stakeholders, and educators cocreate valuable outputs from the learning activities (Cooper et al., 2010). A simulated WIL is ' … an immersive WIL experience in a context created to emulate the functions of a workplace with input by the workplace/community, educational institution, and the student' (Wood et al. 2020, p. 333). The WIL is considered ‘simulated’ because the ISD team members are students in the classroom and not temporarily placed in the company as employees, which might be during a placement or internship. A course is considered a WIL course when it involves three stakeholders: students, academics, and industry/external stakeholders. Without the external element, the WIL becomes a full simulation; albeit technology-supported simulations may be a pedagogic practice within a WIL (Patrick et al., 2009). Active stakeholder involvement accelerates the project work and facilitates the outputs that are of value to the business (Leidner & Jarvenpaa, 1995). Prior research shows that WIL increases the work readiness of students, and thus, universities worldwide are increasing their WIL offerings.

An ISD project aiming to develop a minimal viable product is an authentic and meaningful work task for WIL in an IS course. Organizations and customers demand the delivery of tailored IS at an increased speed to meet heightened pressures from digital innovations and short-time-to-market (Wiesche, 2021). Thus, the WIL needs to mimic the demands for fast delivery, frequent changes, and complex requirements. IS educators attempt to provide such learning environments through agile simulation games (Hefley & Thouin, 2016) and by providing technology-enhanced scaffolds (Janson et al., 2020). The next advances in learning would mean that students work with inputs from external partners (i.e. industry). Such an advancement would ensure that students are provided with meaningful theory-practice integrations (Wood et al., 2020). In particular, with the risks of failing to deliver an output (e.g. software), the relevance and utility of WIL are unlimited (Effeney, 2020).

Low-code ISD and the respective platforms ¹ are development environments increasingly used in industry for 'developing apps fast [while also] changing them fast and inexpensively' (Phalake & Joshi, 2021; p. 690). The low-code platforms provide declarative, high-level programming abstraction and model-driven development (Richardson & Rymer, 2016), thereby extending the functionalities of past WYSIWYG editors (e.g. markup (HTML) editors). Both low-code platforms and WYSIWYG editors have easy-to-use graphical interfaces that create automatic code; albeit scholars reject the notion that markup is a programming code (Phalake & Joshi, 2021; p. 690). Furthermore, some research argues that low-code platforms are not a new technology and that the components of these platforms have existed for some time. However, the innovation of low-code platforms comes from the integration 'in one environment, [of] multiple well-known and traditional system design components' (Bock and Frank, 2021, p. 739). The advancements are integrated low-code platform functionalities such as automated database initiation and extensions, and automated testing (setting up the test environment and running the test continuously) (Beranič et al., 2020; Litman & Field, 2018). The advancements of low-code platforms compared to traditional systems are particularly relevant when considering the differences between no-code and low-code platforms. The latter provides expert developers with an integrated ISD environment, whereas no-code platforms are mainly for business users with little IT skills (Hurlburt, 2021).

Low-code platforms are increasingly used as teaching technology in higher education (Adrian et al., 2020; Metrôlho et al., 2020; Vikebø & Sydvold, 2019). The literature presents several reasons why low-code platforms are deemed suitable teaching technologies, including component integration, easy platform access for students and educators (through cloud solutions), and availability of rich training materials. The decision to use low-code platforms as a teaching technology needs to be made considering limitations raised about them (Luo et al., 2021). For example, studies find the lack of design and layout customization are unique challenges for low-code developers (Alamin et al., 2022). Similarly, practitioners find high pricing models and steep learning curves of low-code platforms to be barriers to adoption (Luo et al., 2021). These limitations demand that educators use free-of-charge platforms and engage in deep curriculum preparation to personally master the use of low-code platforms for teaching and learning. The benefits of the platform can be complemented with agile ISD method principles (Harvey et al., 2019), particularly autonomy (Matook et al., 2016), so that learners have discretion in making decisions on low-code platform aspects, for example, to overcome customization challenges in their interface design.

In sum, with low-code platforms, IS students can develop apps for real-world clients quickly (within the timeframe of a semester), especially should their ISD expertise be that of a non-IT expert. The low-code WIL provides an environment that simulates a real-world ISD project, allowing students to develop and deliver working software quickly.

Educational information systems research and metacognition

Research on digital learning and teaching has a long tradition in the IS discipline with seminal papers dating back 20–30 years (see, for example, Gupta and Bostrom (2009); Leidner and Jarvenpaa (1993, 1995); Piccoli et al. (2001)). This body of work shows that technology improves learning processes and can enhance the effectiveness of a teaching method ² (Leidner & Jarvenpaa, 1993). Ongoing advancements in new technologies provide more integrated and immersive learning in simulated environments (Gupta & Bostrom, 2013). For example, educational research began with examining IBM technologies that projected results to overhead screens (Leidner & Jarvenpaa, 1993) but advanced to web-based training and collaboration methods that had greater positive effects on multiple learning outcomes (Gupta & Bostrom, 2013).

Technology is also used for learning simulations and game-based learning (Leemkuil & De Jong, 2012), including gamified learning (Schöbel et al., 2023) and serious games (López et al., 2021). Especially in management education, gamified online training programs with badges and points create positive learning outcomes (such as better problem-solving) through the increased emotional engagement of learners. Because such learning environments are simulations without needing inputs from an industry/external stakeholder (Wood et al., 2020), teaching and learning are flexible while still realistic. Fewer accounts exist in the literature on technology-rich learning environments that involve all three stakeholders (i.e. students, academia, and industry/external stakeholders) to cocreate outcomes (learning outcomes and product outcomes).

Prior research describes different IS educational models to present the determinants of learning (see, for example, Gupta and Bostrom (2009); Leidner and Jarvenpaa (1993, 1995); Piccoli et al. (2001)). Depending on the learning context, its authenticity (realism), and the learner’s control over the technology, the learning manifests differently (observational repetition, own creation, affective responses) (Leidner & Jarvenpaa, 1995). The models place the learner as a central factor within the learning process (Leidner & Jarvenpaa, 1993). The human dimension – and its specifics – is critical for learning success within a given teaching method (Piccoli et al., 2001). Theoretical diversity exists in other design determinants to account for the differences in learning structures, content, interaction, and technology (Gupta & Bostrom, 2009; Piccoli et al., 2001). Learning outcomes in digital settings are the extent to which learning goals are achieved following the means-end understanding (Matook, 2013). The goals include skill, cognitive, effective, and metacognitive factors (Anderson & Sosniak, 1994). Especially, metacognition is highlighted as an important indicator of students’ ability to learn and to understand their own learning, albeit skills and cognitive goals are most often studied. Educational scholars consequently called for research on metacognition in digital learning and teaching (Gupta & Bostrom, 2009, p. 692).

Metacognition is conceptualized as 'one’s knowledge concerning one’s own cognitive processes and products or anything related to them' (Flavell, 1979, p. 232). At the core, metacognition is about reflections of a person’s cognition and is thus referred to as 'thinking about thinking' (Dinsmore, 2017). In the context of learning, metacognition captures how learners think about how to acquire new information (Flavell, 1979). Contrasting traditional views of learning, which put the educator at the centre of learning, metacognition accredits the learner with an active role in learning (de Fátima Goulão & Menedez, 2015). Metacognitive learners think critically about their learning and adapt when perceived improvements are required (Flavell, 1979). Therefore, learners assume responsibility for the learning outcome and take ownership of the learning process (Flavell, 1979). Through metacognition, learners plan their learning, monitor their progress, and self-assess their performance (Tanner, 2012).

Metacognition draws on reflections as deliberate and focused thinking processes. The reflection aids in understanding one’s own capabilities, learning strategies, and the requirements of a task (Coutinho & Neuman, 2008). Without reflective abilities, learners struggle with planning, and the cognitive monitoring and evaluation of their actions. When learners are given time and opportunity for reflection, they can consciously register when learning occurs and derive conclusions for improvements (Boor & Cornelisse, 2021). Reflexivity makes learning more personal (Clem et al., 2014; Dafei, 2007). Through regular reflective behaviours, learners better internalize the planning, monitoring, and evaluation of learning. Thus, metacognitive skills serve as a ‘surplus value’ to students’ intelligence while contributing to their academic success (Veenman et al., 2014).

Metacognition and thinking about thinking have a focus on the self (Dunlosky & Metcalfe, 2008; Schleifer & Dull, 2009). Scholars, including cognitive psychologists and educational researchers (Livingston, 2003; Schraw et al., 2006; Veenman et al., 2006) suggest that 'metacognition is related to learners’ perceptions of tasks, self, and learning situations (Flavell 1979), (Çini et al., 2023, p. 2). Within the dimensions (detailed in Table 1), attributes of the learner, beliefs, dispositions, motivation, and specifics of the learning task (knowing what) are theorized.

OPEN IN VIEWER

Table 1.

Metacognitive learning theorizing dimension of person and task for a situation.

Dimension	Meaning
Person	It captures attributes of the learner and knowledge about oneself that are indicative of the learner’s abilities and efforts the learner undertakes (Schraw et al., 2006). Relevant factors include self-beliefs of learners in their abilities and factors that capture how much the learners desire to learn (motivation, goal achievement desire).
Task	It captures the new learning assignment and the knowledge about the process of undertaking the task, including strategies a learner applies to execute instructions and complete the task (Schmitt & Newby, 1986).
Situation	It captures the learning context and the contextual experiences of the learner with the task (Çini et al., 2023). Situation-specific responses to task-oriented factors and situational aspects vary among learners.

Differences in metacognitive skills can be explained by examining variations in the knowledge of a learner and the learning tasks (Livingston, 2003). For example, differences in metacognitive skills were due to learners’ differences in language learning (Goh & Vandergrift, 2021; Kessler, 2021), mathematic problem-solving (Lee et al., 2019; Tian et al., 2018), and programming (Chetty & van der Westhuizen, 2014; Prather et al., 2020). Metacognition comes to fruition when a learner is strategic in learning new knowledge (knowing how) and has an understanding of when and why certain behaviours are to be selected (Paris et al., 1983). Flavell (ibid) dimensions serve as influencing factors for metacognitive skills, but scholars must contextualize these factors for unique learning situations (such as learning programming or math problem-solving).

Metacognition is an important factor for education because it can explain how students effectively learn new information (Tanner, 2012). Students acquire current knowledge through their university education but they face an ever-changing workplace as graduates (Wiesche et al., 2019). The environment demands ongoing learning and students must have obtained metacognitive skills as part of their graduate learning. Metacognitive students can plan, monitor and evaluate their learning, which is useful in mastering workplace demands. For example, metacognitively skilled students can reflect and reframe their thinking to new learning processes. Drawing on metacognition, the learning becomes deeper and more effective (Schraw & Dennison, 1994). However, the learning outcomes (e.g. grades or confidence in learned knowledge) vary among students depending on their metacognitive skills (Zhao & Ye, 2020).

Metacognition is investigated in the literature of computer science and IS, but focuses predominantly on programming, which is only one ISD activity (see Appendix A, and references such as Chetty & van der Westhuizen, 2014; Lopez, 1997). In this context, metacognition is theorized as affecting programmers’ choices and the ISD activity of programming for creating software artefacts (Ambrose, 2003; Prather et al., 2018; Shaft, 1995). Metacognitive skills of programmers can impact their comprehension of code (Shaft, 1995). When programmers are given metacognitive instructions during programming, they improve their understanding of the program and can better describe the strategies used (Loksa et al., 2016), albeit only through specific metacognitive heuristics (Shaft, 1995). In addition, novice programmers benefit from using metacognitive strategies to enhance their coding expertise (Prather et al., 2018).

Through reflections and engaging in higher-order thinking, novice programmers understand their needs for scaffolding and knowledge gaps and subsequently, ask for assistance to perform better. Even for senior programmers, their competency depends on metacognitive factors, which in turn influence programming behaviours, resulting in different performance outcomes (Ambrose, 2003). Consequently, this body of research provides evidence of metacognition's influential role in software programming (Havenga et al., 2013) due to the highly intellectual nature of creating and orchestrating these complex systems (Maruping & Matook, 2020). However, programming is only one of many activities during the ISD process; others including planning, analysis, and design have received limited attention (Matook et al. 2021). Two noteworthy exceptions are Santoso et al. (2017) and VanDeGrift et al. (2011) which examine metacognition during the analysis and design activities of an ISD project, but with little attention paid to programming and delivery.

More recently, IS studies examine metacognitive influences in the context of technology adoption, including AI algorithmics and social media analytics (Kessler, 2021; Sagara et al., 2020), but also as general learning aids in IS courses (Lang, 2018; Prather et al., 2020). Collectively, these studies illustrate that students with metacognitive skills think more often about how to acquire new knowledge and are more aware of their abilities to learn (Vu et al., 2000). However, past research is exploratory using qualitative methods such as reflective journals of users and learners (Kessler, 2021; Shaft, 1995), and metacognitive mental maps that elicit reflective episodes and thinking processes (Lee & Baylor, 2006). To a lesser extent, we find quantitative studies (e.g. surveys) with large samples in IS that examine antecedents and impacts of metacognition in IS learners, except for one large experiment study (Prather et al., 2018). Notably, as Appendix A shows, past research mainly focuses on programming. Because prior research remains unclear about metacognition in other contexts than programming, especially learner-centric ISD contexts with real-world outputs and the three-stakeholder model (i.e. WIL), more research is needed.

Hypotheses development

Methodology of quantitative Study 1

In Study 1, we mainly use data from a cross-sectional survey of IS student in a low-code WIL course.

Participants and data collection

The survey participants were students in the low-code WIL course. The study received ethics/IRB approval from the authors’ university ethics board (2021/HE000464) in accordance with national guidelines and the Australian Code for the Responsible Conduct of Research. All 479 enrolled students were invited to participate, from which we received 463 responses. After deleting incomplete and invalid responses (e.g. straight-ticket), we arrived at a final sample of 417 respondents (response rate of 87.1%). The sample size is satisfactory as the power analysis indicates. We used G*Power 3.1 (Faul et al., 2009) to calculate our minimum sample size. Results show we need a minimum of 85 participants to detect a medium effect size (ƒ² = 0.15) with 80% statistical power (alpha = 0.05, two-tailed). Table 2 shows the student demographics and details of the student’s app.

OPEN IN VIEWER

Table 2.

Participants' demographics and app details.

Variable	Category	Frequency	Percentage
Age (mean = 25 years)	Under 23	31	7.43%
	23–30	382	91.61%
	Older than 30	4	0.96%
Gender	Male	149	35.73%
	Female	268	64.27%
Study time (mean = 8 hours/week)	Less than 10	301	72.18%
	10–20	98	23.50%
	More than 20	18	4.32%
University program	Master of Commerce	271	64.99%
	Master of Business	133	31.89%
	Master of Computer Science	13	3.12%
App delivery top 5 criteria included for: 1. Requirements completeness  2. User interface design  3. Implementation complexity	Requirements completeness
	Welcome page to support app navigation	417	100%
	About Us page to share details about the NGO	408	97.84%
	Event information display	401	96.16%
	Possibility for program coordinator to enter information about an event	392	94.00%
	Possibility for program coordinator to delete information about an event	378	90.65%
	User interface design
	User can complete a task within three clicks	415	99.52%
	Page titles and headings	409	98.08%
	Only relevant elements and design choices included (no design overload)	389	93.29%
	Navigation menus	375	89.93%
	Consistent gestalt of the app’s pages	367	88.01%
	Implementation complexity
	Custom layout template	411	98.56%
	Rich data input	394	94.48%
	Multiple, role-based app view	390	93.53%
	Multi-layer microflow design	370	88.73%
	Custom login process	336	80.58%

Results of quantitative Study 1

The research model was tested using covariance-based structural equation modelling (CB-SEM) via LISREL 10.30. We chose CB-SEM because the literature notes that it avoids 'statistical blind spots' (e.g. the multicollinearity and measurement error) that cause Type I errors for path estimates (Goodhue et al., 2017). CB-SEM sidesteps the blind-spot by accounting for random measurement error and construct correlations (Goodhue et al., 2012). Further, CB-SEM is more robust for a large data sample such as ours (N = 417), and it is recommended as the technique of choice when the sample size exceeds 250 (Hair et al., 2012).

Before presenting the results, we detail the results of a comparative calculation of metacognitive skills pre-WIL and post-WIL. If we can demonstrate an increase in the value of metacognitive skills, then it can be suggested that students’ metacognitive skills have increased. As part of the internal university reporting, we captured students’ metacognitive skills at the beginning of the semester (i.e. t₁). The measurement in t₁ serves as a baseline for determining a possible average increase in metacognition. Thus, we compared the mean for metacognitive skills in t₁ = 3.93 with the mean at t₂ = 4.19 (end of the semester, after app completion). An independent t-test shows a significant difference between the two times (p < 0.001). Next, we present first the results for the measurement model and then the structural model (i.e. hypothesis testing) results (Anderson & Gerbing, 1988).

Measurement model assessment

We conducted a confirmatory factor analysis to test the measurement model using a maximum likelihood approach. As per Table 3, the goodness-of-fit indices suggests a reasonable fit of the models to the dataset (Straub et al., 2004).

OPEN IN VIEWER

Table 3.

Measurement model fit evaluation.

Fit Index	Acceptable Levels	Measurement Model
χ2	N/A	609.17
Df	N/A	384
χ2/df	< 3	1.59
Goodness-of-fit (GFI)	> 0.80	0.91
Normed Fit Index (NFI)	> 0.90	0.93
Comparative fit index (CFI)	> 0.90	0.97
Standardized root mean square residual (SRMR)	< 0.10	0.03
Root mean square error of approximation (RMSEA)	< 0.08	0.04

The measurement model was also assessed for reliability and validity, following the guidelines by Fornell and Larcker (1981). Internal consistency reliabilities assess the extent to which the items measuring the same construct correspond highly with each other (Straub et al., 2004). We evaluated the internal consistency using Cronbach’s alpha (α) and the composite reliability measure (pc). As per Table 4, both measures are within the recommended range of 0.70–0.95 for all constructs (Straub et al., 2004), thus suggesting adequate internal consistency. Convergent validity was tested based on three criteria (e.g. Fornell & Larcker, 1981): 1) all indicator factor loadings (λ) should be significant and surpass 0.50; 2) composite reliability (pc) should surpass 0.70; and 3) average variance explained (AVE) should surpass 0.50. Table 4 shows that all three criteria are met, thus demonstrating adequate convergent validity.

OPEN IN VIEWER

Table 4.

Reliability and validity of measurements.

Latent Variable	Mean	SD	Item	Mean	SD	Factor Loadings [p< 0.001]	Cronbach’s α	Composite Reliability	AVE
Knowledge Confidence KNCO	3.89	0.71	KNCO_1	3.76	0.85	0.64	0.89	0.90	0.68
			KNCO_2	3.93	0.81	0.89
			KNCO_3	3.91	0.82	0.92
			KNCO_4	3.95	0.80	0.82
App Delivery APPD	3.98	0.75	APPD_1	4.03	0.80	0.85	0.88	0.90	0.74
			APPD_2	3.99	0.84	0.80
			APPD_3	3.93	0.86	0.92
Metacognitive Skills MECO	4.17	0.66	MECO_1	4.22	0.82	0.84	0.92	0.92	0.62
			MECO_2	4.18	0.79	0.80
			MECO_3	4.06	0.83	0.68
			MECO_4	4.16	0.86	0.75
			MECO_5	4.24	0.77	0.84
			MECO_6	4.22	0.78	0.86
			MECO_7	4.12	0.81	0.71
Self-efficacy EFFI	4.17	0.62	EFFI_1	4.17	0.68	0.71	0.90	0.90	0.65
			EFFI_2	4.15	0.72	0.93
			EFFI_3	4.23	0.70	0.74
			EFFI_4	4.17	0.73	0.84
			EFFI_5	4.14	0.81	0.79
Motivation MOTI	4.24	0.68	MOTI_1	4.17	0.79	0.89	0.88	0.89	0.74
			MOTI_2	4.29	0.74	0.90
			MOTI_3	4.25	0.72	0.79
Strategic Approach STRL	4.14	0.58	STRL_1	3.93	0.78	0.84	0.80	0.83	0.57
			STRL_2	4.01	0.78	0.92
			STRL_3	4.22	0.71	0.68
Autonomy AUTO	4.24	0.61	AUTO_1	4.37	0.68	0.71	0.88	0.81	0.53
			AUTO_2	4.26	0.69	0.69
			AUTO_3	4.24	0.72	0.90
			AUTO_4	4.07	0.83	0.57

Discriminant validity evaluates the extent to which measurement items differ from each other (Campbell & Fiske, 1959). Fornell and Larcker (1981) recommend testing whether the square root of AVE for each construct exceeds the correlations of the construct with other constructs. Perusal of Table 5, the construct correlation matrix indicates that the square root of each construct’s AVE is greater than the off-diagonal correlations. Thus, we believe discriminant validity is satisfactory.

OPEN IN VIEWER

Table 5.

Inter-construct correlations.

Latent Construct	Latent Construct
	1	2	3	4	5	6	7
1: KNCO	0.83
2: APPD	0.47	0.86
3: MECO	0.67	0.67	0.79
4: STRL	0.49	0.43	0.49	0.76
5: EFFI	0.55	0.43	0.53	0.63	0.81
6: MOTI	0.54	0.50	0.59	0.49	0.59	0.86
7: AUTO	0.55	0.47	0.58	0.59	0.73	0.64	0.73

Note: Square root of the average variance extracted (AVE) is the diagonal.

As per Table 5, we noticed the correlation between autonomy and self-efficacy (r = 0.73) is close to the 0.8 cut-off (Franke, 2010; Shrestha, 2020), which may point to multicollinearity. We checked the severity of potential multicollinearity issues via variance inflation factors (VIFs) and the tolerance via condition indices and variance proportions (Shrestha, 2020; Thompson et al., 2017). Results in Appendix F show that multicollinearity is less likely an issue in our data.

One threat to validity is common method biases (CMBs). To control for CMB, we followed established guidelines for procedural and statistical remedies (Podsakoff et al., 2003). Procedural remedies included proximal separation, increasing the physical distance between the measures of the independent and dependent variables. This was achieved by distributing the questions on different pages of the survey. Statistical remedies included Harman’s single factor test and a partial correlation procedure (e.g. marker variable). First, we conducted Harman’s single-factor test (Harman, 1976). We loaded all variables into an EFA and examined the unrotated factor solution. Six factors that emerged from the dataset accounted for 70.39% of the variance, and no single factor explained more than 50% variance. Then, we performed a partial correlation test using a marker variable to examine the influence of CMB on the observed relationships between constructs (Lindell & Whitney, 2001). The results show changes in the partial correlation were non-significant. Overall, these results indicate that CMB is unlikely to pose a strong threat to our dataset.

Structural model assessment

A structural model evaluates the relationship between the theoretical constructs. To test our hypotheses, we assessed the structural model. The results are shown in Figure 3 and Table 6. The structural model presented adequate fit (χ²/df = 1.54, GFI = 0.90, NFI = 0.91, CFI = 0.97, SRMR = 0.05, and RMSEA = 0.04). Appendix G presents the model fit testing results.OPEN IN VIEWER

Figure 3.

Results of research model.

OPEN IN VIEWER

Table 6.

Structural model results.

Path	Path Coefficient	T-statistic	Outcome	Effect Size
H1: Self-efficacy → Metacognitive skills	−0.01ns	−0.06	Not supported	0.002 (not sig.)
H2: Motivation → Metacognitive skills	0.36***	5.23	Supported	0.116 (small to medium)
H3: Strategic approach → Metacognitive skills	0.22***	5.05	Supported	0.043 (small)
H4: Autonomy → Metacognitive skills	0.25*	2.46	Supported	0.028 (small)
H5: Metacognitive skills → Knowledge confidence	0.73***	15.26	Supported	1.198 (large)
H6: Metacognitive skills → App delivery	0.75***	17.04	Supported	1.290 (large)
H7: Metacognitive skills → Grade	0.17**	2.76	Supported	0.258 (medium to large)
R2	Metacognitive skills R2 = 0.536; knowledge confidence R2 = 0.567; app delivery R2 = 0.569; grade R2 = 0.479

Notes: **p < 0.01, ***p < 0.001. ^ns p > 0.05; effect size ƒ² = 0.02 = small, ƒ² = 0.15 = medium, ƒ² = 0.35 = large.

Following Cohen (1988), we calculated effect sizes with the formula: ƒ² = (R²included − R²excluded)/(1 − R²included).

The coefficient of determination (R²), which is the amount of explained variance of the endogenous latent variable, should be above 0.20 to indicate acceptable explanatory power (Zikmund, 2003). Structural model tests results (see Table 6) indicate that our model explained 56.7% of the variance in knowledge confidence (R² = 0.567; a large effect size), 56.9% of the variance in app delivery (R² = 0.569; a large effect size), 47.9% of the variance in grade (R² = 0.479; a large effect size), and 53.6% the variance in metacognitive skills (R² = 0.536; a large effect size).

As per Table 6, motivation (γ = 0.36; t = 5.23, p < .001), strategic approach (γ = 0.22; t = 5.05, p < .001), and autonomy (γ = 0.25; t = 2.46, p < .05) significantly influence metacognitive skills. Thus, H2, H3, and H4 are supported. As the effect of self-efficacy on metacognitive skills is not significant (γ = −0.01; t = −0.06, p > .05), H1 is not supported. The results show that the effects of metacognitive skills on knowledge confidence (β = 0.73; t = 15.26, p < .001), app delivery (β = 0.75; t = 17.04, p < .001), and grade (β = 0.17; t = 2.76, p < .01) are significant. Thus, H5, H6, and H7 are supported.

Among the controls, gender significantly influences knowledge confidence (β = −0.14; t = −3.39, p < .001) and grade (β = −0.29; t = −4.58, p < .001). Studying time (β = 0.40; t = −6.26, p < .001), development expertise (β = 0.20; t = 3.16, p < .01), and prior academic performance (β = 0.46; t = 7.28, p < .001) are significant on grade. App delivery is only significantly influenced by prior academic performance (β = 0.10; t = 2.18, p < .05). Age is not significant with any dependent variable.

To check the robustness of the results, we examined the effect sizes (ƒ²) (Table 6). The results show that metacognitive skills have a large effect (Cohen, 1988) on both knowledge confidence (ƒ² = 1.198) and app delivery (ƒ² = 1.290), and a medium to large effect on grade (ƒ² = 0.258). Strategic approach and autonomy have a small effect of 0.043 and 0.028. Motivation has a small to medium effect of 0.116. Self-efficacy does not have a significant effect.

Post-hoc analysis – mediation tests

Although this study does not hypothesize mediation effects, the research model embeds such effects. Results for mediation tests following a bootstrapping approach are shown in Table 7 (Hayes, 2009; MacKinnon et al., 2012; Vance et al., 2015). More detailed bootstrapped CI test results are reported in Appendix H. Our results show that metacognitive skills fully mediate the effects of strategic approach and autonomy on knowledge confidence, respectively, and partially mediate the effects of motivation on knowledge confidence. Metacognitive skills also fully mediate the effects of strategic approach and autonomy on app delivery, and partially mediate the effects of motivation on app delivery. Metacognitive skills, however, mediate the effects of neither independent variable on grade. There were no mediating effects of self-efficacy on either dependent variable.

OPEN IN VIEWER

Table 7.

Mediation test results.

Relationship	Effect			Outcome
	Direct	Indirect	Total
Self-efficacy → MC skills → Knowledge confidence	0.184**	0.040ns	0.224**	No mediation (direct-only)
Motivation → MC skills → Knowledge confidence	0.112*	0.154**	0.266***	Partial mediation
Strategic approach → MC skills → Knowledge confidence	0.102ns	0.083*	0.185**	Full mediation
Autonomy → MC skills → Knowledge confidence	0.060ns	0.113*	0.173*
Self-efficacy → MC skills → App delivery	−0.001ns	0.052ns	0.051ns	No mediation (no effect)
Motivation → MC skills → App delivery	0.123*	0.200***	0.323***	Partial mediation
Strategic approach → MC skills → App delivery	0.115ns	0.108*	0.223**	Full mediation
Autonomy → MC skills → App delivery	0.036ns	0.146*	0.183*
Self-efficacy → MC skills → Grade	0.007ns	−0.019ns	−0.012ns	No mediation (direct-only/no effect)
Motivation → MC skills → Grade	0.327***	−0.024ns	0.303***
Strategic approach → MC skills → Grade	0.230*	0.007ns	0.240**
Autonomy → MC skills → Grade	0.174**	0.011ns	0.185**

Notes: MC skills = metacognitive skills; ^ns p> 0.05, *p< 0.05, **p < 0.01, ***p < 0.001.

Motivation

The quantitative analysis in Study 1 provides insights into learner-oriented and task-oriented influences in low-code WIL on metacognition and learning outcomes. The results determine that motivation, autonomy, and the strategic approach significantly impact on metacognition, but self-efficacy does not. Thus, we conducted this Study 2 to examine the non-significant finding (Venkatesh et al., 2016) while also enhancing the understanding of how the design choices regarding technology aspects (low-code platform, agile method, and development task) impact on learning. In our qualitative engagement with the data, we sought for deeper insights into low-code WIL with respect to Study 1 factors, and their interplay with the technology. We aimed to provide a better understanding of how (in which ways) learner-oriented and task-oriented factors (as part of Study 1 model) impact students’ metacognition and the role the technology aspects play. We benefited from the literature on scaffolding (Lajoie, 2005; Wood et al., 1976) to delineate students’ individual learning experiences.

Scaffolding is the temporary support by experts (human and objects) for learners to accomplish a task that is beyond the learner’s individual capabilities (Wood et al., 1976). Prior research conceptualizes different scaffolding approaches that guide and facilitate learning (Hannafin et al., 2004), viz. procedural (knowing how), conceptual (knowing what), metacognitive (reflecting on the how and what), strategic (identifying alternative solutions), and technology-enhanced scaffolds (IT for learning and teaching) (Janson et al., 2020; Sharma & Hannafin, 2005). For a low-code WIL, the technology-enhanced scaffolds are most interesting because the platform and the ISD method can serve as the supporting expert for students. Technology-enhanced scaffolds are technological tools and resources to support learning (Kim & Hannafin, 2011), including static web-based materials (e.g. online books, learning portals) and dynamic learning environments (e.g. artificial intelligence (AI)–based tutors (Wambsganss et al., 2021)).

The design of any scaffolds – timing, intensity, and richness – impacts learning processes, giving rise to individual experiences of learners. In Appendix D, we detail the learning activities in the low-code WIL regarding metacognitive skill building. In each low-code development stage, multiple ISD activities are undertaken. For example, in the app creation stage, students participate in model-driven development, a retrospective, and feedback activities. For the students’ learning, the low-code experts are educators, industry partners, and technology aspects. The scaffolding provided by human experts fades over the teaching time when students gain ISD capabilities. In contrast, the technology-enhanced scaffolds remain in place (e.g. Mendix learning paths, platform error messages); albeit these support structures are dynamic, feedback on learning varies depending on learning activities. As the detailed account of the designed low-code learning in Appendix D shows, scaffolds and educational efforts are directed towards building metacognition in low-code WIL. However, one needs to involve the learners and gather their perspectives to better understand the learning experience. Thus, our guiding question for Study 2 is: 'How do students experience learning in low-code WIL regarding metacognitive skills and technology-enhanced scaffolds as well as the learner-oriented and task-oriented factors?'

Methodology of qualitative Study 2

The reflective essays are a written account of students’ learning. We treated the essays as qualitative data that provides insights into student learning (challenges and mastery) in the low-code WIL. The reflections describe a student’s process of learning; how to learn new knowledge and how to learn low-code app development within the WIL pedagogy. Of particular interest are the factors in our research model (Figure 1). In this study, we followed a realist ontology and approached the data with a positivist mindset (Sarker et al., 2018). We sought to triangulate the non-significant finding from Study 1 while also gaining deeper insights into students’ learning experiences in the low-code WIL environment.

Two coders, experienced in qualitative research, coded the 417 student essays. An initial coding list guided the analysis with a focus on the learner-oriented factors, task-oriented factors, and their relationship to metacognitive skills. The coders first coded 35 essays independently and then discussed their coding to understand agreement and address coder differences. Subsequently, each coder coded 191 students’ essays and they regularly met to discuss the coding findings. Through creative collaboration, the coders arrived at a shared understanding.

Results of qualitative Study 2

The students describe the WIL as a self-learning experience that demands their active participantion in the learning. The acknowledgement of the Self shifts the locus of responsibility from the teacher to the learner. This means learning accountability for creating and delivering the low-code app is placed with the students. By accepting the learning responsibilities, students show that they possess educational readiness for WIL. In the essays, students use terms such as ‘self-study’ and ‘self-learning’. One student explained:

[Mendix development] … is self-learning and exploration. Before, I only obtain knowledge from lectures instead of participating in mining knowledge and exploring knowledge by myself. For example, I was unaware of how to convert the user requirement into an app. After I undertook a self-study, I learned the different type of user and transfer their needs into business value by searching from the learning paths of Mendix, I have successfully solved the problem and created the app.

We also found various examples in the data describing how technology-enhanced scaffold of the low-code platform helped with metacognitive skill building. For example, students explain that implementation errors stimulated thinking about better ways to learn low-code development. As Appendix D illustrates, in the stage of 'App creation with Mendix low-code platform', feedback is provided by the platform as part of the app creation process before deployment. As this technology-enhanced scaffold depends on the quality and errors of the app, it is student-dependent and dynamic feedback. Importantly, the error message also explains the type of error and provides a link to the online handbook with tips on how to debug it.

Mendix itself also provides a hint of all kinds of error messages, which helped me get into the habit of thinking about problems.

In analyzing the qualitative data, we particularly sought to explain the non-significant impact of self-efficacy on meta-cognition from Study 1. In their essays, students highlight how the development on the low-code platform and the interactions with the app improved their beliefs in themselves and their abilities to develop a working app in a short time (a few weeks of the semester). Students’ confidence extends beyond the classroom, indicating that they applied the skills gained further. The technology-enhanced scaffolds in the form of materials on low-code development (i.e. the Learning Path as part of the Mendix Academy) are online accessible. Thus, support was readily available, albeit students need to search for the respective section in the materials:

My teacher recommended the learning path on the Mendix website, where I was able to enhance my skills and upgrade what I had learned, as well as increase my motivation to learn on my own and gain confidence in my future work and studies. This event also prompted me to want to learn more beyond the course and to start researching published software that I could learn from, including typography, design, and user stories.

The reflections performed by students during the app development also strengthen their self-efficacy beliefs. Students benefited from the teacher demonstrations of the low-code activities, but then continued with the ISD work on their own, in the belief they could deliver an app to the client’s satisfaction.

I should first understand what each step is doing, what is the role and purpose, and then try to complete it by myself, taking the teacher's operation as a correction version, self-picking errors and then correcting them.

We also found evidence in the data that students at times were not self-efficacious in their development work, such that they felt overwhelmed and did not trust their abilities. Students lacking self-efficacy sought out various learning resources (technology-enhanced scaffolds) to allow them to continue with the app development, for example, teaching instructions, Mendix Academy materials, and lecture video recordings. Through these activities, they still engaged in metacognitive thinking because they reflected on how to learn low-code development and adjusted accordingly. Again, the technology-enhanced scaffolds supported the students’ learning and contributed to their high perceptions that they could deliver the app.

I thought that I was not competent enough to do the job. But in the process …, I also checked a lot of things on the Internet and in the library and conducted research. But when I encounter a problem that cannot be solved, I will watch the teacher’s video to improve it. I think this is not good, because I haven’t done a thorough thinking and reflection, so next time I should complete it independently and encounter problems.

The qualitative data also provides details on the convergent results, esp. the learner-oriented factors of motivation and the task-oriented factors of autonomy and strategic approach. Regarding motivation, the student voices explicitly mentioned it as enabling ‘thinking about their own thinking' (i.e. metacognition) because the feedback received in the showcase presentations (see Figure 2) stimulated their motivation and a meta-thinking process:

In the WIL process, I do have some thinking for myself. Because of unique WIL learning, sometimes I just finished the basic work at the beginning and do not have enough motivation to learn by myself. But in the following process, I notice other classmates do really well in this course and much useful knowledge has been taught, I made the decision I need to adapt to the learning, so I try all the styles of learning and find the most suitable way for myself to learn the information system design and Mendix.

In addition, the technology-enhanced scaffolds, esp. the drag-and-drop coding of the low-code platform, facilitated students’ motivation. For example, Mendix provides an effortless, easy-to-use visual user interface that motivates students to low-code. Similarly, the online Mendix Academy offers teacher and classroom independent learning resources. Using the material enables students to grow their mastery of low-code development. The quotes below suggest that students were surprised about their self-motivation to develop the low-code app, while the quotes also show that the availability of the technology-enhanced scaffolds motivated them to learn and continue to learn, maybe even more so than traditional scaffolds:

What I found more surprising was that this motivation to learn was spontaneous, by constantly taking in new knowledge independently, practicing encountering problems, and then problem-solving and summarising them, which I think is the best way to learn.

The experience of with the technology-enhanced scaffolds also benefits the task-oriented factors of autonomy and strategic approach. Using a strategic approach for low-code app development was seen as the best way to deliver the app and to build metacognitive skills. Students explored various platform-related materials (e.g. Mendix Academy materials) on how to use the low-code platform:

During the Mendix development, I try to determine which things I do understand well and adjusted my learning strategies accordingly. Most of the time, I tried to figure out what the logic is behind every step? However, sometimes, I didn't adjust my learning strategy in time, and I just follow the steps of the lecturer.

Autonomy cultivated in students while undertaking the development task was repeatably mentioned as a benefit by the students. The teacher-recommended solution for the app was perceived only as an alternative option. At the same time, students embraced the freedom to decide on their own, engaged freely with the platform and its features, and tried various development possibilities.

Making almost the same app according to the steps taught by the teacher actually limits our imagination and understanding of Mendix in a certain way. Because the freedom to use Mendix can help us to explore the platform better. For example, if we encounter new problems when designing a new board, we can deepen our understanding of Mendix in the process of solving these problems.

Although students appreciated the autonomy, they expressed the need to enhance further their skills and knowledge about low-code ISD and metacognitive knowledge of learning processes.

Then followed the given instructions, I tried to autonomously build a real App according to different requirements. In this learning experience, I found that clear logical thinking was highly required for creating Microflow. To design high-level custom functions in other App, I think I still need to practice more to strengthen my logical thinking.

Subsequently, metacognitive skills emerged, which the students enjoyed building as it was a personal challenge that enhanced their employability:

The whole learning experience is enjoyable to me because it enhances my problem-solving capability and upskills my ability for my future career.

Similarly, a student explained that the authentic nature of the WIL – having an actual client in the classroom – furthered their thinking and enhanced their workplace readiness:

The WIL this semester taught me to accurately collect the required information from clients and interact with clients online. How to memorize clients’ needs and use them in my own work task. I hope I can skillfully use this knowledge to work efficiently and reduce errors in my future positions.

In summary, as per Table 8, the student voices indicate that the design of the low-code WIL experience (i.e. specific low-code development tasks, platform features, and ISD practices) enhanced their motivation, self-efficacy, autonomy, and their strategic approach, which in turn, improved their metacognitive skills. Details of metacognitive skills building are presented in Appendix D. The ISD work and the low-code platform functioned as technology-enhanced scaffolds for learning. Notably, the scaffolds did not fade but were available to students permanently, esp. the low-code platform.

OPEN IN VIEWER

Table 8.

Essay-based relations between low-code platforms, independent variables, and metacognitive skills.

ISD and Low-Code Platform AspectsFeatures, Practices, and Tasks	Motivation	Self-efficacy	Strategic Approach	Autonomy	Metacognitive Skills
Features
Automated testing and deployment (error handling, publishing, previewing)	x		x	x	x
Application logic (microflows)	x	x	x	x	x
Visual modelling/drag-and-drop interface	x	x		x
Data structure (entities/attributes)		x		x	x
Role-based user access	x			x	x
Agile practices
Requirement gathering (client interviews)	x			x	x
Design sprint planning (e.g. formulating user stories)			x	x	x
Retrospectives		x		x
Feedback session/showcases (app presentation to client, teachers feedback)				x
Development tasks
Creation of business logics (e.g. microflows)	x			x	x
Self-directed user resources (Mendix Academy/learning paths)	x	x	x	x	x
Relationship building across app components (esp. between data structure and other app elements)	x	x			x
User-centric interface design	x		x		x

Learning and ‘learn to learn’ in low-code WIL environments

The research examined how students in low-code WIL build metacognitive skills, which are essential skills for lifelong learning and for future workplaces. Study 1 investigated the impact of learner-oriented and task-oriented factors (Flavell 1979) on metacognitive skills leading to relevant learning outcomes. The empirical results indicate that students’ motivation, autonomy, and the strategic approach aid in developing metacognition, albeit students’ self-efficacy perceptions are not relevant. Study 2 focused on students’ experiences in the low-code WIL environment aiming to understand how learning takes place and the role of technology-enhanced scaffolds in supporting learning and metacognitive learning. Findings show that students use extensively different technology-enhanced scaffolds that allow building metacognition at different stages of the work (see Appendix D). Through the results of Study 2, we can address the divergent finding regarding self-efficacy.

In Study 1 and Study 2, we show that motivation impacts metacognition. When students are highly motivated to learn the skills to deliver a valuable and working business app to the NGO, they more often encounter problems because they are learners, not professional developers with expert knowledge. Thus, students need to think and reflect on the development work and explore alternatives to find solutions to their ISD problems. These reflections also encourage them to think about their development approaches, the methods used, and their past development activities. For some requirements, additional knowledge was necessary and that was not given to them by the lecturer. Only when the students reflected on how they had planned and executed their development can they realize mistakes, errors, and oversights, and subsequently, fix them. For example, the NGO did not mention the feature of password recovery in the planning meeting, but in a later showcase, it became an important non-functional requirement. When confronted with the missing feature, students re-designed their interface while capturing other non-functional requirements. The reflection in the showcases and a strong motivation stimulated their thinking about how to learn better to undertake ISD activities. Thus, reflection can improve the low-code ISD work through metacognition.

Both studies provide convergent results for the task-oriented factor. We propose that students can acquire metacognitive skills when they strategically learn how to develop apps with low-code platforms. The WIL course used a low-code platform and the agile ISD method, which is known for exploration opportunities (Vidgen & Wang, 2009). Thus, students were able to be reflective when exploring innovative ways. For example, the WIL allowed students to challenge their interface design, reflect on alternative data structures, and think about a better way to implement a user story by adopting different agile practices, esp. showcase presentations, retrospectives, and planning meetings. The technology-enhanced scaffolds, which relate to planning (user stories) and the ISD process (testing and deployment), support the students in achieving their development results, such as Mendix Microflows. Our results demonstrate that students can translate their strategic and result-oriented ISD efforts into metacognitive learning and thinking about better ways to develop the app.

Autonomy – a second task-oriented factor – is an important aspect of the low-code WIL and we obtained convergent findings for it through the two studies. Prior research shows that autonomy is essential within the agile ISD method (Lee & Xia, 2010). Students had control over their ISD decisions, guided only by the client’s requirements. Our results support a positive relationship between autonomy and metacognition. Students used the autonomy to plan, monitor, and assess not only their ISD activities but also their thinking about how to develop the app. Because the WIL took place in a blended learning setting (Zhang et al., 2016), many ISD activities had to be done outside class hours, and without the teaching staff’s input. In utilizing their autonomy, students engaged in thinking about how to best use the Mendix platform, obtain solutions to error fixing, and monitor their development work. Consequently, our findings provide empirical support for educators who promote students’ independent ISD work as it creates metacognitive skills. Without traditional human scaffolding (e.g. procedural and conceptual), students need to think on their own about how to advance their skills in planning, monitoring, and assessing the intermediary outputs (app). However, the technology-enhanced scaffolds were always available to them and did not fade over time.

In this research, we encountered one divergent finding. In Study 1, the relationship between self-efficacy and metacognitive skills was not significant. However, in Study 2, we found evidence in the qualitative data that self-efficacy contributes to metacognitive skill building. In consulting prior research, we find that differences in fading of human and technology-enhanced scaffolds (Collins et al., 1988) explain the divergent result. The low-code WIL curriculum was designed as a rich practical experience with extensive ongoing support, including technology-enhanced scaffolds (platform’s features, learning materials), the agile ISD method practices, and support from teaching staff. However, the literature on scaffolds suggests that they should be only temporarily available and must fade so that learners take responsibility for their learning (Sharma & Hannafin, 2005). The continuing availability of the scaffolds might result in students’ overconfidence in their abilities (Littrell et al., 2020; Zamary et al., 2016) while also preventing them from beneficial moments of confusion (Lodge et al., 2018).

Our meta-inference about self-efficacy and metacognitive skills is that students who used technology-enhanced scaffolds and esp. used them more or in addition to human scaffolds were able to build metacognitive skills. We suggest this meta-inference because (1) both types of scaffolds did not fade contributing to students feeling self-confident about their abilities for the tasks, (2) human scaffolds were individualized and calibrated providing students with immediate and correct help, and (3) technology-enhanced scaffolds were not calibrated and not tailored to each specific student motivating reflections. Considering these conditions, the findings from Study 2 need to be supported which illustrate the building of metacognition related to high levels of self-confidence. This conclusion is supported by the theory of metacognition in the field of programming, which provides empirical evidence on how metacognitive skills form when learners write software code and are supported by scaffolds (Loksa et al., 2016). Thus, students drawing on technology-enhanced scaffolds need to reflect (i.e. engage, plan, and monitor) on learning with the low-code platform, the agile method, and the ISD task. For example, when the testing and deployment fail, they need to reflect on why this happened, what they know about how to rectify it, and how to proceed. Therefore, different technology-enhanced scaffolds are available, such as links to the Mendix handbook, lecture recordings, and Mendix developer discussion group. In this process, students build metacognitive skills. Consequently, we suggest that students, who used the human scaffolds more than the technology-enhanced scaffolds, had fewer opportunities to build metacognitive skills.

Finally, our results show that metacognitive skills impact multiple learning outcomes. Indeed, students’ metacognitive skills enhanced their confidence, the app delivery quality, and the grade, all of which are important aspects for becoming workplace ready. When students are confident in what they learned and deliver an app to the satisfaction of the clients, they can effectively undertake the ISD activities that eventually enhance their employability in IS careers (e.g. Business Analyst). A high grade achieved in the course is important because it signals knowledge and skills to future employers. The three factors in concert contribute to creating deep metacognitive thinking processes and the knowledge to deliver low-code apps. The mediation testing further illustrates that metacognitive skills also partially or fully mediate the relationships between the independent variables (except for self-efficacy) and dependent variables (except for grade). The test provides however no evidence that metacognitive skills mediate the effects of the independent variables on grade, albeit strategic approach, motivation, autonomy, and metacognitive skills alone impact grade. A plausible reason for this is that there may exist other factors that mediate the relationships between the independent variables and grade.

Implications and contributions

This research provides several contributions to the literature. First, it enriches our understanding of the use of digital educational technologies, viz. low-code platforms (Bock & Frank, 2021), on students’ ability to learn how to learn (i.e. metacognition) (Bennett, 2017; Flavell, 1979; Tanner, 2012). This study provides evidence of the antecedents of metacognition in ISD environments where problems are ill-structured and complex (Matook et al., 2021). For students with little experience in authentic ISD workplaces, app development may appear to be overwhelmingly challenging. Important ISD factors aid in building metacognition that in turn, influences learning outcomes. Because the hypothesized relationships were significant (except for self-efficacy), this study is one of the first to demonstrate how low-code WIL also promotes learning and skills to learn-to-learn.

The study provides an example of how digital educational technologies enhance practical experiences for learners in IS education. The use of technology in student learning was examined in blended learning environments (Dang et al., 2016; Zhang et al., 2016) and programming (Prather et al., 2020). In these settings, the technology is a support media, whereas in a low-code WIL, the technology is the focal point of learning. The platform and the app manifest as one artefact such that an enhanced understanding of the platform translates into a feature-rich app, while feedback on app improvements stimulates advanced knowledge of the platform. We believe the study provides new knowledge on the catalyst role of technology for effective learning (Leidner & Jarvenpaa, 1995).

Second, we also contribute to the IS literature on low-code development as an ISD method for non-IT professionals (Bock & Frank, 2021; Frank et al., 2021). Prior research shows how low-code platforms utilize visual interfaces, model-driven development, and automatic code generation to create apps with minimal hand-coding (Litman & Field, 2018). Extensive research is currently conducted to theoretically investigate low-code methods and cloud-based platforms regarding adoption, use, and benefits. Our study builds on these findings to demonstrate how students in business school IS programs experience low-code development. The business students can be contextualized as non-IT developers, and their development tasks can be a trigger to revive the discussion of user-led ISD initiatives (Lawrence & Low, 1993). Similarly, the findings contribute to the long tradition in ISD research on democratizing IS development (Bjerknes & Bratteteig, 1995; Muller & Kuhn, 1993). Our empirical results provide evidence that non-IT professionals can be empowered to develop valuable solutions while limiting the control of the ISD process and outcome by IT departments and experts.

Third, we also contribute to the literature on learning approaches in the field of education. A lively discussion surrounds the concept of learning styles with strong views about their validity (Nancekivell et al., 2021). While we believe this is an ongoing discussion, we contribute to it with new knowledge about the balanced approach. Particularly, as we do not subscribe to the strict position that a learning style is learners’ disposition valid across different learning situations (Dembo & Howard, 2007). Our results demonstrate that within low-code WIL, the strategic approach with a result-oriented focus enables achieving both objectives, viz. gaining workplace experiences and metacognitive skills. With an achievement focus, students learn how to learn and have practical experiences in the future workplace. Our findings provide new knowledge suggesting that integrating theory and practice in contemporary IS teaching affords educational skill building. Through metacognition, students acquire skills that support the mastery of future innovations in the workplace.

Fourth, we contribute to the literature on motivation, particularly on learners’ motivation to acquire skills for delivering results to the ISD clients (NGO business in the case of this WIL environment). Motivation theory contextualized to the field of education (Colquitt et al., 2000) and studies on motivation of learners (Esnaashari et al., 2020; Klein et al., 2006) demonstrate the importance of students taking responsibility for their learning. In the low-code WIL, we focused on the motivation of students to learn the skills for producing a valuable, working product and their willingness to engage in ISD activities (Noe & Schmitt, 1986). Especially in WIL, when learners believe in the authenticity of the project, they seek to become more skilful at their development. This goal can be achieved through the reflection of learning approaches, the planning, monitoring, and improvements of ISD tasks. Our study enriches prior literature by showing that when learners invest time and efforts in creating apps, they build metacognitive skills that enhance their learning outcomes. Thus, motivation is an internal factor of a learner and to a large extent, under the learner’s control. The factor’s mean (Table 4) was among the highest (4.24/5), which provides evidence that real-world practical experiences indeed stimulated students’ motivation.

Finally, we contribute to research on autonomy, a concept highly valued in agile ISD method (Lee and Xia, 2010). Educational scholars show that students with large degrees of decision freedom succeed at a learning task (Chene 1983). Similarly, our study shows that low-code WIL also supports students working autonomously with the creation of metacognition. Consequently, research that proposes to treat students as active learning partners rather than passive knowledge recipients is also supported by our study (Wang et al., 2013). Observations from our own experiences as educators suggest that through the autonomy afforded by low-code WIL, students’ metacognitive skills (e.g. reflective thinking) improved, evidenced by the less demanding weekly consultations compared to previous years. Through building metacognitive skills, students explored and implemented better processes for their app development.

Findings impact on teaching practices in higher education, on ISD work using low-code platforms with citizen developers (i.e. students), and on learners’ understanding of how to become workplace ready. At universities, the WIL teaching benefits from a focus on metacognition as a means for enhancing employability. It is of crucial importance to build skills at the ‘meta’ level to encourage learners’ reflections on their thinking processes. Our findings also impact on ISD practices by providing insights into the use of low-code platforms by novice and citizen developers. The students serve as a proxy for practitioners who lack programming expertise but have received ISD analysis and design training. The WIL shows how non-IT professionals can self-direct the development of apps. Finally, learners can draw on the findings to understand how they can control their success in learning and, ultimately, their future careers. When learners are motivated to acquire skills for delivering results, engage in an output-oriented strategic approach, and develop apps autonomously, they build metacognitive skills, which are useful for mastering the demands of today’s digital workplace.

Limitations and future research

Some limitations of the current research need to be acknowledged. First, we have used the proxy measures (knowledge confidence, app delivery, and grades) for employability. The extent to which WIL helped enhance students’ workplace readiness and improved success in their future careers can only be determined in a longitudinal study. A future research study (qualitative diary study) should accompany learners from ‘being students’ to ‘becoming graduates’ and follow them through the beginnings of workplace life. Such a study would provide details on the effects of university education, esp. WIL, on employability.

A second limitation may be the delivery mode of the course being exclusive to online teaching due to the global health pandemic. All students and educators interacted virtually but in real-time. In fact, all teaching was delivered as live lectures, and recordings were only used as an additional learning aid. Still, the virtuality may have impaired some low-code WIL activities and how metacognition was integrated into the curriculum. However, virtual work was the new norm during the years 2020–2021, and the teaching settings simulated even this real-world aspect of remote working and distributed software development. Thus, a future research study is encouraged to compare low-code WIL in exclusively virtual teaching to that in a hybrid mode.

It may be that self-efficacy perceptions are impacted by the differences in the technology-enhanced scaffolds in virtual and hybrid learning (Thatcher & Perrewe, 2002). Whereas our focus was on WIL self-efficacy, future studies may want to theorize the more specific concept of computer self-efficacy. Regarding the divergent finding of self-efficacy, it needs to be noted that we did not collect an additional dataset, but instead provide empirical and theoretical explanations for the difference. Such an approach is possible following guidelines by Venkatesh et al. (2016).

Further research may theorize WIL elements and examine how specific agile practices and low-code experiences (e.g. agile retrospectives, talk out loud about interface design) contribute to building metacognitive skills. Another study may take an employer’s perspective to evaluate metacognitive teaching practices undertaken in the classroom. In designing a qualitative study, interviews with practitioners may provide insights into the relevance and effectiveness of the learning. A resulting taxonomy of metacognitive ISD teaching practices would be useful for educators.