Abstract
Listening plays a critical yet undertheorized role in English-medium instruction (EMI), where subject courses are taught through English rather than learners’ first language, and access to disciplinary knowledge relies heavily on real-time comprehension of lectures and academic tasks not designed for language development. For low-proficiency students enrolled in science, technology, engineering, and mathematics (STEM) programmes, this creates a high-stakes learning condition in which assessment-driven accountability can intensify cognitive and affective pressure. While outcome-based education (OBE) offers curricular structure and transparency, it is frequently criticized for privileging measurable outcomes over sustained engagement and long-term skill development. This study examines how accountability in EMI listening courses can be structurally balanced with ungraded pedagogical supports to foster engagement and learning sustainability. Drawing on a three-year longitudinal reform of a compulsory elective EMI listening course, the study analyses how a design approach integrating graded accountability with collaborative learning, voluntary practice, and reflective work reshaped learners’ engagement and learning trajectories. Using a mixed-methods design combining descriptive and inferential quantitative analyses with thematic analysis of learner reflections, integrated through triangulation, the analysis synthesizes performance data, engagement logs, and learner reflections collected from 533 low-proficiency STEM learners across three cohorts. Findings indicate that assessment-driven designs alone were insufficient to support learning progression in EMI listening. When graded accountability was complemented by ungraded, low-risk supports, learners demonstrated reduced failure risk, sustained voluntary engagement, and heightened reflective awareness of listening strategies and goals. Listening was increasingly construed not as a short-term course requirement, but as a transferable academic skill. The study contributes to EMI and language teaching research by theorizing EMI listening as a pedagogically constrained condition and by illustrating how OBE can be enacted as a supportive, rather than compliance-oriented, design. The findings offer transferable design principles for balancing accountability with ungraded pedagogical supports in EMI listening courses beyond the immediate institutional context.
Keywords
1. Introduction
English-medium instruction (EMI) is commonly defined as the delivery of academic subjects through English rather than through students’ first language (Macaro et al., 2018a), and has been widely examined in diverse higher education contexts (Lasagabaster & Sierra, 2010). In such contexts, listening plays a critical yet often underexamined role, particularly in higher education in science, technology, engineering, and mathematics (STEM). Learners’ access to disciplinary knowledge depends largely on real-time comprehension of lectures, instructions, and academic tasks delivered orally in English. However, EMI subject courses are typically designed to prioritize disciplinary content rather than explicitly develop learners’ language skills. As a result, EMI listening functions as a pedagogically constrained learning condition in which students are required to sustain attention, process dense academic input, and monitor comprehension in real time while simultaneously managing the linguistic demands of a second language (Field, 2019; Siegel, 2020). For low-proficiency learners, these conditions frequently lead to comprehension breakdowns, reduced participation, and limited engagement with learning tasks (Galloway et al., 2021; Macaro et al., 2018a; Rose et al., 2019).
Although listening is central to learning in EMI settings, curricula have largely focused on content transmission rather than on pedagogical support for listening development. Prior research indicates that EMI courses rarely articulate explicit language-learning aims, leaving learners to cope independently with the linguistic challenges embedded in disciplinary instruction (Duran & Sert, 2021). While recent studies have begun to explore listening strategies and learner adaptation in EMI contexts (e.g. Fung & Lo, 2023), there remains limited empirical research examining how EMI listening can be systematically designed and supported within existing curricular and assessment structures, particularly for low-proficiency learners in STEM disciplines.
This challenge is further intensified by the widespread adoption of outcome-based education (OBE) in higher education. OBE emphasizes constructive alignment among learning outcomes, teaching activities, and assessment, alongside transparency and measurable performance indicators, as key mechanisms for ensuring academic accountability (Biggs & Tang, 2011; Harden, 2007; Spady, 1994). In EMI contexts, such structures can offer curricular clarity and procedural consistency by making expectations explicit for both instructors and learners. However, OBE has also been criticized for encouraging assessment-dominant practices that prioritize short-term performance indicators over sustained engagement and longer-term learning capacity (Yu, 2016). When learning is framed primarily through graded outcomes, instructional attention may shift toward compliance with assessment requirements rather than meaningful participation in learning processes. For learners who already face linguistic vulnerability in EMI listening, such assessment-heavy designs can intensify anxiety and reinforce surface participation, rather than supporting the development of confidence, engagement, and sustainable listening skills.
Recent pedagogical discussions therefore emphasize the need for balanced instructional designs that integrate accountability with supportive learning conditions. From this perspective, the core issue in EMI listening is not whether accountability should be present, but how accountability can be pedagogically configured to promote engagement, functional skill development, and learning sustainability under conditions of linguistic constraint. Addressing this issue requires moving beyond isolated teaching techniques toward principled design approaches that coordinate graded requirements with low-risk opportunities for preparation, collaboration, practice, and reflection.
The present study responds to this gap by examining an enhanced outcome-based education – balanced collaborative and individual learning (OBE–BCIL) design implemented in a compulsory elective EMI listening course for low-proficiency STEM learners. Rather than treating OBE–BCIL as a fixed or context-bound model, the study conceptualizes it as a design approach that balances graded accountability with complementary ungraded pedagogical supports. Over three years, the course evolved from an assessment-dominant configuration to a balanced model integrating collaborative learning and individual accountability, and finally to an enhanced design incorporating ungraded supports aimed at reducing learning risk and sustaining engagement.
To examine how these design shifts shaped learners’ experiences and learning trajectories in EMI listening, the study focuses on three interrelated constructs: engagement level (EL), skill development (SD), and learning awareness (LA). EL reflects learners’ willingness and capacity to participate actively in listening tasks under accountability conditions. SD refers to learners’ functional ability to process spoken academic input, including comprehension, strategy use, and task performance. LA captures learners’ metacognitive understanding of listening as a transferable and ongoing academic skill, including awareness of strengths, weaknesses, strategies, and long-term learning goals. These constructs represent a developmental learning trajectory rather than independent outcomes. In EMI listening, engagement is a necessary condition for sustained skill development, while skill development alone does not ensure continued learning beyond course requirements. Learning awareness, therefore, becomes central to understanding whether instructional designs support learners’ capacity to navigate future EMI learning demands independently.
By theorizing EMI listening as a pedagogically constrained condition and examining how accountability can be productively balanced with supportive learning designs, this study contributes to ongoing debates on OBE implementation and EMI pedagogy, while articulating transferable design principles for fostering engagement, skill development, and learning sustainability in EMI listening courses across diverse higher education settings. Section 2 reviews relevant literature to position this study within existing EMI and OBE research and to clarify the gap addressed.
2. Literature Review
2.1. Outcome-Based Education and Accountability in EMI Higher Education
Outcome-based education (OBE) has played a significant role in shaping curriculum design in higher education by foregrounding constructive alignment among learning outcomes, teaching activities, and assessment (Biggs & Tang, 2011; Harden, 2007; Spady, 1994). Advocates argue that OBE enhances transparency, fairness, and academic standards by clarifying expectations and making achievement measurable. When effectively implemented, OBE can provide clear instructional direction and accountability for both instructors and learners. However, critiques have cautioned that narrowly implemented OBE may become overly assessment-driven, encouraging short-term performance and compliance rather than meaningful engagement and sustained learning (Yu, 2016). In such cases, learning outcomes risk being reduced to grading targets rather than serving as developmental guides for learning progression.
These tensions are particularly salient in English-medium instruction (EMI) contexts. EMI learners often face additional linguistic and cognitive demands, and accountability mechanisms based primarily on assessment outcomes may disproportionately disadvantage low-proficiency students (Dafouz & Smit, 2020; Macaro et al., 2018a). EMI implementation has also been discussed at the policy–pedagogy level, highlighting tensions between institutional goals and classroom realities, particularly in Asia-Pacific higher education contexts (Macaro et al., 2018b). In EMI settings, accountability therefore requires careful pedagogical design to ensure that it supports, rather than undermines, learner engagement, skill development, and longer-term learning capacity.
2.2. EMI Listening as a Pedagogically Constrained Condition in STEM
Listening plays a central role in EMI, where students access disciplinary knowledge primarily through spoken English. Unlike traditional language courses, EMI programmes typically do not include explicit language-learning objectives, leaving learners to manage comprehension challenges independently (Duran & Sert, 2021). As a result, listening in EMI functions under highly constrained conditions, particularly in content-heavy STEM disciplines.
Research consistently shows that EMI learners experience difficulty processing lectures, following instructions, and sustaining attention during extended spoken input (Field, 2019; Galloway et al., 2021; Siegel, 2020). These challenges are intensified for low-proficiency learners, who must process technical content while simultaneously managing unfamiliar academic language forms. For low-proficiency STEM learners, the simultaneous demands of disciplinary knowledge and second-language processing can increase cognitive overload, reduce confidence, and discourage participation (Macaro et al., 2018a; Rose et al., 2019). While recent studies have begun to document listening strategies and learner adaptation in EMI settings (e.g. Fung & Lo, 2023), relatively little research has examined how EMI listening courses can be deliberately designed to mitigate these constraints through pedagogical support rather than leaving learners to cope individually.
2.3. Engagement Through Collaborative Learning Under Accountability Conditions
Learner engagement is widely understood as a multidimensional construct encompassing behavioural, emotional, and cognitive dimensions (Fredricks et al., 2004; Mercer & Dörnyei, 2020). In assessment-intensive contexts such as EMI, behavioural engagement is often driven by grading requirements, while emotional and cognitive engagement depend more strongly on instructional design. Collaborative learning has been shown to enhance engagement by distributing responsibility, encouraging peer interaction, and supporting comprehension through dialogue (Gillies, 2019; Johnson & Johnson, 2009). Well-structured collaborative activities can promote accountability while reducing individual performance pressure, particularly for learners facing linguistic challenges (Slavin, 1995).
In EMI STEM contexts, collaborative learning may serve a dual role. It maintains participation under accountability conditions while providing linguistic scaffolding and shared problem-solving opportunities. However, collaborative structures alone do not guarantee sustained engagement, especially when participation remains primarily assessment-driven. Without complementary supports, collaboration risks becoming another graded requirement rather than a mechanism for deeper learning.
2.4. Reflection, Self-Regulation, and Sustained Learning Awareness
Beyond immediate engagement and performance, sustained learning in EMI requires learners to develop self-regulatory capacity. Self-regulated learning involves goal setting, monitoring, strategy use, and reflection, all of which are associated with long-term language development (Oxford, 2017). Reflective practices allow learners to identify strengths and weaknesses, evaluate learning strategies, and develop awareness of learning processes (Schön, 1983; Zhang et al., 2022). In listening contexts, reflection supports the development of metacognitive strategies such as planning, monitoring comprehension, and evaluating understanding, which are essential for coping with fast-paced academic input (Vandergrift & Goh, 2012). However, reflective awareness does not emerge automatically. Studies emphasize that reflection must be systematically embedded within instructional design and supported by meaningful learning experiences to foster sustainable learning orientations (Oxford, 2017).
In EMI settings, reflection is particularly important because learners often perceive listening difficulties as fixed or externally imposed. Structured reflection can help learners reconceptualize listening as a developable skill, but only when learners have sufficient opportunities to engage, practise, and experience progress.
2.5. Voluntary and Self-Directed Listening as Supportive Practice
Listening development also depends on extensive exposure beyond classroom instruction. Voluntary and self-directed listening provides learners with opportunities to engage with authentic input, refine strategies, and build confidence (Peng & Woodrow, 2010; Zhang et al., 2023). Such practice is especially valuable in EMI contexts, where classroom exposure alone may be insufficient to meet learners’ listening needs.
Despite its potential benefits, research indicates that participation in voluntary listening practice remains limited unless it is supported by structured pedagogical frameworks. Without guidance or integration into course design, optional activities are often neglected, particularly by low-proficiency learners who may lack confidence or strategic awareness. Integrating voluntary listening within guided learning cycles or reflective tasks can increase participation and transform optional activities into meaningful learning experiences (Graham, 2017; Zhang et al., 2023). In EMI contexts, these approaches may help learners develop autonomy and resilience while reducing reliance on assessment-driven motivation alone.
2.6. Integrating Accountability with Ungraded Supports in EMI Listening Course Design
Considered together, existing research highlights several elements that are individually beneficial for EMI listening: accountability through OBE structures, collaborative learning, reflective practice, and voluntary self-directed engagement (Dafouz & Smit, 2020; Galloway et al., 2021; Macaro et al., 2018a). However, prior studies have tended to examine these elements in isolation, offering limited insight into how they might be systematically integrated within a single course design for low-proficiency STEM learners. In EMI listening contexts, accountability is necessary to ensure participation and fairness, particularly in large and diverse cohorts. However, accountability alone often reinforces compliance-oriented engagement and fails to address the pedagogical constraints learners face when processing academic spoken English. Collaborative learning can mitigate some of these pressures by distributing responsibility and providing peer support, but its impact remains limited when participation is driven primarily by grades. Similarly, reflection and voluntary practice are widely recognized as essential for sustained learning, but they are unlikely to take hold without structured opportunities for engagement and low-risk experimentation.
What remains underexplored is how these elements can be coordinated as part of a coherent, outcome-oriented design that balances graded accountability with ungraded, supportive learning conditions. For EMI listening, such integration is particularly important because engagement, skill development, and learning awareness are interdependent rather than independent outcomes. Learners are unlikely to develop reflective awareness or self-directed habits unless they first experience sustained engagement, and engagement itself depends on instructional environments that reduce risk and support participation. Addressing this issue requires moving beyond isolated pedagogical techniques toward design-oriented approaches that foreground how accountability and support interact over time. Examining how these interactions shape learner engagement trajectories, listening skill development, and awareness of listening as a transferable academic skill remains a significant gap in EMI and OBE research.
2.7. Research Gap
Building on the preceding discussion, there remains a limited understanding of how these elements can be systematically integrated within a single instructional design to support sustained engagement in linguistically demanding EMI contexts. While each element has been shown to support aspects of learning, their coordinated implementation within a unified design remains underexplored. In particular, prior research has not sufficiently addressed how accountability-driven OBE structures can be balanced with pedagogical supports that reduce cognitive and linguistic pressure for low-proficiency learners.
As a result, EMI listening is often treated either as a content delivery mechanism or as an individual skill to be assessed, rather than as a pedagogically structured process that requires coordinated support over time. This gap is especially evident in relation to learner engagement, which has been widely recognized as a key mediator of learning outcomes but is rarely operationalized as a design outcome shaped by the interaction of instructional components. Existing studies tend to report engagement as a by-product of specific interventions, rather than examining how engagement can be intentionally designed, sustained, and linked to skill development and learning awareness.
Addressing this gap, the present study investigates how an OBE–BCIL instructional framework coordinates accountability with structured pedagogical supports to create conditions for sustained learner engagement in EMI listening. In doing so, the study contributes a design-oriented perspective that emphasizes the sequencing and alignment of instructional elements. In order to address this gap, this investigation is guided by the following research questions:
Research question 1: How does balancing graded accountability with ungraded supports affect learner engagement in EMI listening? This research question was examined through analysis of engagement logs and students’ reflection survey responses.
Research question 2: To what extent does this accountability–support balance influence listening skill development among low-proficiency learners? This research question was investigated through analysis of students’ listening performance across course assessments.
Research question 3: How does this design shape learners’ awareness of listening strategies and long-term learning goals in EMI contexts? This research question was explored through qualitative analysis of reflection narratives and group interview data.
3. Methodology
In Thailand, EMI has expanded rapidly across higher education as universities increasingly pursue internationalization and global competitiveness. In STEM programmes specifically, disciplinary courses are often delivered in English, requiring students to process complex academic content through a second language. Nevertheless, a number of learners enter university with limited English proficiency, making sustained comprehension of lectures and academic tasks challenging. Within this context, listening functions simultaneously as a medium through which students access disciplinary knowledge and as a skill that must be developed and evaluated within language-focused curricula, often under linguistically demanding learning conditions.
This study adopted a descriptive, developmental mixed-methods design to examine how successive course design refinements shaped learner engagement, listening skill development, and learning awareness in an EMI listening course. The study followed three student cohorts (2023–2025) enrolled in the same compulsory elective course, allowing systematic comparison of learning outcomes under differing accountability–support configurations within a stable curricular context over a three-year period of iterative pedagogical refinement.
3.1. Participants and Context
A total of 533 undergraduate students participated across three cohorts: 236 in 2023, 192 in 2024, and 105 in 2025. All participants were STEM majors, including engineering, computer science, and applied sciences, enrolled in degree programmes delivered partially or fully through English. Institutional placement records indicated that most students entered the course at CEFR A2–B1 levels, reflecting limited readiness to process extended academic input in English. Despite disciplinary differences, all cohorts followed the same EMI listening syllabus, learning outcomes, instructional sequence, and assessment framework. This consistency ensured that differences observed across cohorts could be attributed primarily to instructional design changes rather than disciplinary variation, while still reflecting the diversity typical of EMI STEM populations.
In 2025, the course was delivered in two instructional cycles (n = 285). The present study reports data from the first cycle (n = 105), which was the first cohort to complete the fully enhanced course design. Data from the second cycle were excluded because instruction was still ongoing at the time of analysis. This decision ensured that only complete datasets capturing engagement, performance, and reflection were analyzed, while also demonstrating the scalability of the reform across repeated course implementations.
3.2. Course Design Trajectory
The course was refined across three stages in response to observed learner difficulties and emerging evidence from prior cohorts. Prior to these refinements, the course followed a conventional EMI listening format centred on comprehension-based input and assessment. Students typically completed individual listening tasks and quizzes based on short audio or lecture-style materials, with limited opportunities for preparation, collaboration, or reflection. Instruction focused primarily on checking correct answers rather than supporting how learners approached listening tasks, and engagement was largely driven by assessment requirements rather than sustained participation.
In 2023, the course employed an accountability-heavy OBE design, with assessment weighted 70% individual quizzes and 30% collaborative tasks. This design emphasized transparency and fairness but offered limited pedagogical support, resulting in surface participation and uneven engagement among low-proficiency learners.
In 2024, the design shifted to an outcome-based education – balanced collaborative and individual learning (OBE–BCIL) configuration. Assessment was equally divided between Collaborative Learning and Learning from Mistakes (CLLM, 50%) and Individual Listening Quizzes (ILQ, 50%). This revision aimed to scaffold weaker learners through collaboration while preserving individual accountability. Performance improved under this design; however, engagement remained largely grade-driven.
In 2025, the design was further enhanced by retaining the balanced grading structure while embedding three ungraded supports: Knowledge-Building Cycles (KBC), Voluntary Self-Directed Listening (VSDL), and Reflective Practice (RP). These supports were introduced to reduce performance risk, encourage voluntary engagement, and cultivate awareness of listening as a sustainable academic skill.
Across these stages, the design progression can be understood as a shift from assessment-dominant practice (2023) to balanced accountability and collaboration (2024) and, finally, to an integrated model combining accountability with structured, low-risk supports for engagement (2025). The three-year timeframe was not arbitrarily determined but reflects the iterative nature of pedagogical refinement across successive cohorts. Each phase of the design was implemented over a full academic cycle, allowing sufficient time for learners to engage with the instructional configuration and for observable patterns of engagement, performance, and learning awareness to emerge. This longitudinal approach enabled the study to examine how progressive design adjustments shaped learner outcomes over time, rather than capturing short-term effects from a single instructional intervention.
3.3. Research Instruments
Data collection varied across cohorts. For the 2023 and 2024 cohorts, only final grade distributions were available, reflecting student performance under accountability-heavy and early OBE–BCIL conditions. For the 2025 cohort, a mixed-methods design was implemented using five complementary instruments designed to capture behavioural engagement, listening performance, and learning awareness across the enhanced course design.
The Performance Assessment (PA) served as the primary graded indicator of listening skill development across cohorts. It functioned as the central graded measure across all cohorts, though its structure reflected evolving course designs. In 2023, the PA was weighted heavily toward quizzes (70%) with fewer collaborative tasks (30%). In 2024, the structure shifted to an equal balance of 50% CLLM and 50% ILQ, reflecting the initial OBE–BCIL model. In 2025, the enhanced OBE–BCIL framework retained this 50–50 split while embedding ungraded supports such as KBC, VSDL, and RP. Across all years, the PA functioned as the consistent benchmark for tracking skill development and allowed meaningful comparisons of outcomes.
The Engagement Log (EL) captured behavioural participation in both graded and voluntary course activities. Introduced in 2025, the EL systematically monitored participation across graded and ungraded components. It documented attendance at KBC and CLLM sessions, along with voluntary participation in VSDL, enabling distinctions between accountability-driven and intrinsically motivated engagement. As such, these logs provide an objective, non-self-report measure of behavioural engagement based on observed participation patterns, thereby reducing reliance on self-report data alone.
The Reflection Survey (RS) measured learners’ perceived engagement, skill development, and learning awareness. A 9-item survey was administered in 2025 to capture students’ perceptions of engagement, skill development, and learning awareness. Items were grouped into three dimensions and rated on a 5-point Likert scale. Reliability testing showed high internal consistency (Cronbach’s α = .87), confirming the instrument’s stability.
The Reflection Narratives (RN) provided qualitative accounts of learners’ experiences with the enhanced course design. All students in 2025 were invited to submit short written reflections describing their experiences with the course, focusing on motivation, strategy use, and awareness. These narratives provided qualitative insight into how learners engaged with the enhanced OBE–BCIL design.
The Group Interview (GI) generated dialogic accounts that elaborated learners’ engagement and reflective learning experiences. This semi-structured interview was conducted with eight volunteer students from the 2025 cohort, with questions exploring participation, skill development, and reflective practice, thereby extending insights beyond survey and narrative data. An overview of these instruments, their targeted constructs, and their analytical purposes is presented in Table 1.
Research Instruments, Constructs, and Analytical Purposes Across Cohorts.
3.4. Procedure and Data Analysis
Each cohort completed the course under its respective design. In 2023 and 2024, data collection focused on summative outcomes. In 2025, instruments were embedded throughout the instructional process. Engagement data were recorded throughout KBC, CLLM, and VSDL activities, performance data were captured through ILQ assessments, and awareness data were collected through the reflection survey, reflection narratives, and group interviews administered at the end of the course. Table 1 summarizes the instruments used across cohorts, their targeted constructs, and their analytical purposes. These instruments collectively enabled the examination of learner engagement, listening skill development, and learning awareness under differing accountability–support designs.
A mixed-methods approach was employed (Creswell & Plano Clark, 2018). Quantitative analyses included descriptive statistics for grade distributions, engagement frequencies recorded in the Engagement Log, and survey responses. Cohort differences in grade distributions were examined using chi-square tests with Cramér’s V as an effect size estimate. Within the 2025 cohort, paired-samples t-tests compared Collaborative Learning and Learning from Mistakes and Individual Listening Quiz scores to examine transfer from collaborative to individual performance, ensuring that engagement in group work did not artificially inflate outcomes. Qualitative data from Reflection Narratives and Group Interviews were analyzed thematically following Braun and Clarke’s (2006) framework. Coding was conducted inductively with analytic attention to learner engagement, strategy use, and learning awareness. A second coder independently analyzed approximately 25% of the reflection narratives, and discrepancies were resolved through discussion to enhance interpretive reliability.
Integration of findings was achieved through triangulation (Dörnyei, 2007). Data from multiple sources, including engagement logs, performance outcomes, reflection surveys, reflection narratives, and group interviews, were analyzed in combination to enable methodological triangulation across behavioural, performance, and self-reported dimensions. Learner-reported perceptions of engagement were interpreted alongside behavioural indicators independent of self-report, including participation rates in graded and ungraded activities, depth of voluntary engagement in self-directed listening, and performance transfer from collaborative to individual assessments. This triangulated approach reduced reliance on attitudinal data alone and strengthened the validity of interpretations regarding engagement and learning outcomes across design conditions.
4. Results
This section reports on findings from a three-year reform of a compulsory elective EMI listening course, with learner engagement as the central analytic focus. Quantitative and qualitative findings are presented in an integrated manner to explain not only what changed across course designs, but also how and why these changes occurred. Additional patterns were observed across cohorts; however, results are selectively reported here to foreground engagement as the primary analytic condition and to maintain coherence with the study’s design-oriented contribution. Listening skill development and learning awareness are therefore reported as outcomes that emerged through sustained engagement rather than as independent result strands.
4.1. Engagement Patterns Under the Enhanced Accountability: Support Design
Learner engagement was most clearly observable in the 2025 cohort, when the enhanced course design combined graded accountability with ungraded pedagogical supports. Unlike the 2023 and 2024 cohorts, where engagement could be inferred primarily from performance outcomes, the 2025 cohort generated direct evidence of participation across instructional components. This allows a detailed examination of how learners engaged with listening tasks when accountability was balanced with opportunities for preparation, collaboration, autonomy, and reflection. Table 2 summarizes learner participation across graded and ungraded components in the enhanced design.
Learner Engagement Across Graded and Ungraded Components in the Enhanced Design (2025, n = 105).
Participation rates indicate that engagement extended well beyond compliance with assessment requirements. As expected, graded components attracted near-universal participation. More striking, however, was the similarly high participation in ungraded activities. Preparatory discussions in KBC involved 98% of students, 95% engaged in VSDL, and 88% completed RP tasks. These patterns suggest that low-proficiency learners did not restrict their engagement to credit-bearing activities but participated broadly across the instructional cycle, including tasks that carried no formal assessment weight.
4.1.1. Depth of Voluntary Engagement
While participation rates capture the breadth of engagement, the depth of engagement is most evident in VSDL activity. Because VSDL was entirely voluntary, the number of completed tasks provides insight into learner motivation in the absence of grade incentives. Students completed an average of 18.6 listening tasks, with substantial variation across individuals. Some learners completed few or no tasks, whereas others engaged intensively, completing more than 70 activities. This pattern indicates that the design allowed engagement to scale according to learners’ readiness and motivation rather than enforcing uniform participation. For a cohort of low-proficiency STEM learners, the normalization of voluntary listening practice represents a shift away from minimal compliance toward self-initiated engagement.
4.1.2. Perceived Quality of Engagement
Survey data further clarify how learners experienced engagement in the enhanced course design. Students reported strong agreement that collaborative activities increased motivation, fostered a positive learning atmosphere, and reframed mistakes as opportunities for improvement (all mean ratings above 4.6 on a five-point scale). These results indicate that engagement was not only frequent but also experienced as meaningful rather than merely compliance-driven. Qualitative evidence from reflection narratives and group interviews provides deeper insight into how this engagement was sustained at an experiential level. Learners frequently described a reduction in performance pressure during in-class activities, alongside a sense of reassurance gained through peer interaction. For many, collaborative preparation created a psychologically safer space in which they could rehearse responses, test understanding, and build confidence prior to independent performance. As one participant reflected, ‘Group work reduced the pressure of answering alone. After practicing together, I felt more confident when working independently’ (GI, P4).
In addition to confidence building, learners emphasized a gradual shift in responsibility for learning. Engagement was not framed solely as a response to grading requirements, but as a self-directed commitment that extended beyond formal assessment. Voluntary practice, in particular, was perceived as a means of strengthening personal learning habits rather than fulfilling external expectations. This shift is illustrated in one reflection, where a student noted, ‘I joined activities not only because of grades, but because I wanted to improve. Practicing on my own made me feel more responsible for my progress’ (RN, S22).
Collectively, these accounts suggest that the quality of engagement in the enhanced design was shaped by learners’ experiences of psychological safety, shared effort, and opportunities to prepare for independent performance. Engagement thus functioned not only as active participation, but also as a developmental condition through which confidence and responsibility for learning could emerge under accountability constraints.
4.2. Engagement as a Pathway to Listening Skill Development
Listening skill development was examined using scores from the Performance Assessment (PA), the sole assessment instrument administered consistently across all three cohorts. Because the PA remained unchanged across course iterations, it provides a stable basis for examining how differences in instructional design were associated with variations in listening performance. Table 3 summarizes performance outcomes under each course design.
Listening Performance Outcomes Across Course Designs (2023–2025).
Notes. OBE = outcome-based education; BCIL = balanced collaborative and individual learning.
As the balance between accountability and ungraded pedagogical support increased across designs, failure rates declined steadily and higher achievement became more widely distributed. In 2023, more than 15% of students failed, and fewer than 10% achieved B or above. In 2025, failures were eliminated entirely and nearly half of the cohort achieved B or higher. A chi-square test confirmed that differences across cohorts were statistically significant, χ²(14) = 92.16, p < .001, with a medium effect size (Cramér’s V = .29).
In order to examine whether collaborative engagement inflated performance outcomes, scores from Collaborative Learning and Learning from Mistakes (CLLM) were compared with Individual Listening Quizzes (ILQ) within the 2025 cohort (see Table 4). ILQ scores were significantly higher than CLLM scores, indicating that collaborative engagement functioned as a scaffold rather than a substitute for individual accountability. Qualitative evidence helps explain this transfer effect. Learners described using collaborative activities to rehearse strategies before applying them independently, as reflected in one account, where a student stated, ‘Working with my group helped me practice strategies first. When I took the quiz alone, I already knew how to approach the listening’ (RN, S18).
Transfer from Collaborative to Individual Performance (2025, n = 105).
Notes. CLLM = Collaborative Learning and Learning from Mistakes; ILQ = Individual Listening Quizzes.
These findings suggest that improvements in listening performance are best understood as outcomes of sustained engagement distributed across both graded and ungraded learning activities.
4.3. Engagement and the Emergence of Learning Awareness
Beyond performance outcomes, sustained engagement was associated with the development of learning awareness in the 2025 cohort. Survey results indicated high agreement that learners became more aware of their listening strengths and weaknesses, reflected more deliberately on learning processes, and felt prepared to continue developing listening skills beyond the course (M ≈ 4.5). Qualitative evidence elaborates these patterns. Reflection narratives and interviews showed that learners used reflective practice to identify specific difficulties, adopt targeted strategies, and plan continued learning beyond immediate course requirements, as illustrated in one reflection, where a student explained, ‘I realized my weakness with long sentences and practiced more intentionally to improve. Listening is no longer just for passing this course’ (RN, S30).
Across accounts, learning awareness emerged only after sustained engagement had been established. Learners explicitly linked reflection and future planning to repeated participation in collaborative and voluntary activities. Learning awareness, therefore, did not function as a separate or antecedent outcome, but as a developmental consequence of engagement sustained over time.
4.4. Integrated Summary of Findings
Across the three-year reform, engagement emerged as the pivotal condition through which instructional design influenced learning outcomes. When graded accountability was balanced with ungraded supports in 2025, learners engaged broadly and deeply with listening tasks. This engagement enabled measurable improvements in listening performance and supported the emergence of learning awareness oriented toward sustained EMI learning. Instead of operating as independent outcomes, skill development and learning awareness emerged as downstream effects of engagement. Viewed together, these findings demonstrate how balancing accountability with supportive, low-risk learning opportunities can transform EMI listening from an assessment-driven requirement into a meaningful and sustainable learning experience for low-proficiency STEM learners.
5. Discussion
This study examined a three-year reform of an EMI listening course for low-proficiency STEM learners, focusing on how different configurations of accountability and pedagogical support shaped learning outcomes. Instead of viewing listening difficulties in EMI as fixed learner deficits, the findings demonstrate that engagement in EMI listening is highly sensitive to instructional design, particularly to how graded accountability is balanced with low-risk opportunities for preparation, collaboration, voluntary practice, and reflection. Across the three course iterations, engagement emerged as the central condition through which listening skill development and learning awareness were enabled.
5.1. Re-conceptualizing Engagement in EMI Listening
Engagement in EMI listening is often assumed rather than deliberately designed. This section, therefore, reframes engagement in EMI listening as a pedagogical outcome shaped by instructional design rather than a stable learner characteristic. In many EMI contexts, listening is positioned as a passive prerequisite for content learning, with limited pedagogical attention to how learners manage linguistic load, cognitive demands, and performance pressure. Under such conditions, engagement is implicitly expected to emerge through assessment compliance. Evidence from accountability-heavy designs reflects this pattern: although assessment ensures fairness and transparency, learner engagement remains minimal and largely compliance-oriented.
The introduction of collaborative learning has been shown to increase visible participation, aligning with EMI research demonstrating that peer interaction can reduce anxiety and support comprehension (Gillies, 2019; Johnson & Johnson, 2009). However, participation driven primarily by collaboration alone often remains extrinsic and grade-dependent. The present findings indicate that engagement in EMI listening cannot be sustained through accountability or collaboration in isolation. Instead, it requires deliberately designed learning conditions that allow learners to prepare, experiment, and recover from errors without immediate evaluative consequences.
Ungraded preparatory and recovery spaces, such as Voluntary Self-Directed Listening (VSDL) and Knowledge-Building Cycles (KBC), fulfilled this role. Their high uptake demonstrates that low-proficiency learners are willing to engage voluntarily when EMI listening is framed as manageable and developmental rather than punitive. This supports the view that engagement is not an inherent learner trait but a design-mediated outcome, particularly in EMI environments characterized by linguistic risk and cognitive overload (Mercer & Dörnyei, 2020).
5.2. Engagement as a Mediator of Listening Performance in EMI STEM
Improvements in listening performance across cohorts were closely aligned with changes in engagement conditions. While a balanced accountability–collaboration design reduced failure rates and improved average performance, the most substantial gains emerged only after ungraded supports were introduced. These patterns suggest that accountability establishes minimum participation thresholds, whereas engagement mediates how performance is distributed and sustained across learners, particularly in linguistically demanding EMI STEM contexts. This relationship is not assumed to be linear or deterministic, but highlights engagement as a key enabling condition within performance-oriented EMI environments.
Comparison of collaborative (CLLM) and individual (ILQ) assessments further clarifies this mechanism. Higher individual quiz scores indicate that collaborative engagement functioned as a scaffold rather than inflating outcomes. This finding is particularly important in EMI STEM education, where collaborative approaches are sometimes criticized for diluting academic rigor. Instead, the results show that when engagement is supported through structured collaboration and voluntary practice, learners are better positioned to transfer strategies to independent performance rather than becoming reliant on group support.
Empirically, these findings align with EMI research calling for pedagogical approaches that integrate language support without compromising academic standards (Macaro et al., 2018a; Rose et al., 2019). More importantly, they identify engagement, rather than collaboration per se, as the mechanism through which low-proficiency learners are able to meet the listening demands of EMI courses.
5.3. Engagement and the Emergence of Learning Awareness in EMI Listening
Learning awareness was largely absent in earlier course designs but became visible once engagement was sustained across both graded and ungraded activities. In the enhanced design, reflective practice did not merely prompt learners to recount experiences; it enabled them to interpret their own engagement, explicitly linking effort, strategy use, and learning outcomes.
In EMI contexts, this form of awareness is particularly significant. Learners often experience EMI listening as externally imposed and beyond their control. Reflective Practice, supported by sustained engagement, helped learners reconceptualize listening as a skill that can be analyzed, improved, and maintained beyond immediate course requirements. This aligns with research on self-regulation and metacognitive awareness (Oxford, 2017), while extending it by demonstrating that learning awareness emerges as a developmental consequence of engagement, not as its prerequisite.
Positioning awareness in this way also clarifies why isolated strategy instruction or stand-alone reflection tasks frequently show limited impact in EMI settings. Without sustained engagement, awareness lacks experiential grounding. When engagement is established through supportive design, reflective practice consolidates learning into a longer-term orientation toward EMI listening.
5.4. Implications for EMI Listening Course Design
The findings offer several implications for EMI STEM education. Theoretically, they extend critiques of outcome-based education (OBE) by demonstrating that accountability is necessary but insufficient for learning under linguistically constrained conditions. When complemented by ungraded pedagogical supports, OBE can be enacted as a flexible design framework that promotes engagement rather than reinforcing compliance-oriented participation.
Empirically, the study shows that improvements in EMI listening outcomes are mediated by engagement conditions rather than by assessment structures alone. This shifts attention from assessment design to learning design, helping to explain inconsistent findings in prior EMI interventions that focus narrowly on testing formats or isolated strategy instruction without addressing how learners are prepared and supported to engage under accountability pressure. Practically, the enhanced OBE–BCIL framework should not be interpreted as a fixed or context-bound model. Instead, it articulates a set of adaptable design priorities that can be applied across diverse EMI STEM settings, even where class size, assessment weightings, instructional formats, or resource availability differ.
Building on these findings, EMI listening can be conceptualized as a design problem that requires balancing accountability with these core pedagogical supports. These include establishing accountability that sets participation thresholds without dominating learning spaces, embedding preparatory collaboration to reduce linguistic and cognitive risk, structurally supporting voluntary practice rather than leaving it incidental, and using reflective practice to consolidate engagement into sustained learning awareness.
Rather than prescribing specific tasks or formats, this approach highlights how instructional elements can be purposefully sequenced to support learner engagement under EMI conditions. Such sequencing may be adapted across disciplinary contexts, for example, by aligning preparatory activities with lecture formats in engineering, problem-solving discussions in science, or case-based input in applied fields. In this sense, the contribution of the study lies not only in the specific configuration implemented but in demonstrating how accountability and pedagogical support can be systematically coordinated to enable engagement, skill development, and learning awareness in EMI listening. This design perspective enables instructors to adapt these principles across contexts by coordinating accountability with structured learning supports. In this way, learners are guided through a developmental process in which they progressively move toward more independent and sustained engagement with academic listening.
In contexts where EMI expansion outpaces students’ English proficiency, these principles may be particularly relevant for supporting learners who must process disciplinary knowledge through a second language. This is especially salient in Thailand and similar expanding EMI systems, where instructional designs must support learners in navigating disciplinary content through English while developing sustained engagement and longer-term learning awareness. The findings, therefore, underscore how EMI listening courses within language programmes can play a strategic role in developing the listening competence required for participation in English-medium academic environments.
5.5. Transferability of the Study
This study extends the applicability of its findings across contexts by articulating transferable design principles rather than presenting a context-bound instructional solution. It shows how accountability structures can be systematically coordinated with ungraded pedagogical supports to sustain learner engagement in linguistically demanding environments. To support adaptation in different settings, it identifies key design elements that are not tied to specific institutional conditions or assessment structures. These include structured preparation to reduce cognitive load, opportunities for low-risk participation to lower affective barriers, integration of collaborative learning to support meaning-making, provision of voluntary practice to extend engagement beyond formal tasks, and reflective activities to consolidate learning awareness. These elements can be flexibly adapted according to class size, learner proficiency, and instructional format. Importantly, the value of the study lies not in replicating the exact course design, but in adapting the underlying design logic to different EMI contexts.
To support practical adoption, these elements can be operationalized through the ASE (accountability–support–engagement) sequence, offering a clear pathway for instructors to translate the findings into their own contexts. Firstly, accountability structures establish clear participation expectations and align learning activities with measurable outcomes, consistent with principles of constructive alignment in OBE (Biggs & Tang, 2011). Secondly, pedagogical support is embedded through structured preparation, collaborative learning, low-risk practice, and reflective activities, all of which are widely recognized as important for reducing cognitive load and supporting language development in EMI contexts (Mercer & Dörnyei, 2020; Vandergrift & Goh, 2012). Thirdly, these combined conditions enable sustained learner engagement, which functions as a key mediator linking instructional design to skill development and longer-term learning awareness (Fredricks et al., 2004).
In this way, the ASE sequence provides a portable design logic rather than a fixed framework, allowing instructors to adapt the balance and sequencing of these elements according to their own EMI teaching conditions. This perspective shifts transferability from replicating specific practices to reconfiguring underlying design principles, making the approach applicable across diverse EMI settings where learners must process disciplinary content through a second language under varying levels of linguistic readiness.
6. Conclusions and Future Directions
This study examined a three-year reform of an EMI compulsory elective listening course for low-proficiency STEM learners, demonstrating how different configurations of accountability and pedagogical support shaped learner engagement, listening performance, and learning awareness. Across the three course designs, a clear developmental pattern emerged. An accountability-heavy approach ensured fairness but failed to sustain engagement. A balanced accountability–collaboration framework improved performance yet remained largely grade-driven. The enhanced OBE–BCIL framework, which integrated ungraded supports alongside graded components, fostered sustained engagement that enabled both stronger listening skill development and heightened learning awareness.
These findings contribute to rethinking outcome-based education (OBE) in EMI contexts. Accountability remains essential, particularly in large and academically diverse STEM cohorts, but it is insufficient on its own to support low-proficiency learners. When accountability is deliberately balanced with opportunities for preparation, voluntary practice, and reflection, EMI listening can shift from short-term task completion to sustained, self-directed learning. This reframing positions engagement not as a by-product of instruction, but as a central design outcome.
Several contextual constraints should be considered when interpreting the findings. The study was conducted within a single institutional setting, which may limit direct generalization to other EMI environments. Cohort sizes varied across years, with the smallest cohort in 2025, potentially affecting proportional comparisons of achievement outcomes. In addition, fine-grained data on engagement and learning awareness were available only for the final year of the reform, limiting longitudinal comparison beyond performance measures. Participation in Voluntary Self-Directed Listening also relied partly on self-reported logs, which may introduce reporting bias. Finally, the study focused exclusively on listening, leaving questions about applicability to other language skills in EMI contexts.
These considerations suggest several directions for future research. First, multi-institutional studies could examine whether similar engagement trajectories emerge when comparable design principles are implemented in different EMI STEM settings. Second, future research could investigate the mechanisms through which collaborative engagement transfers to independent performance, particularly in content-intensive disciplines. Third, longitudinal studies could track whether learning awareness fostered through reflective practices leads to sustained self-directed language use after course completion. Extending the framework to speaking, writing, or integrated skills would further clarify its scope and adaptability across EMI curricula.
Overall, this study shows how an accountability-heavy EMI listening course can be systematically redesigned into a balanced and enhanced framework that supports engagement, performance, and learning awareness. By foregrounding engagement as the condition through which other outcomes emerge, the enhanced OBE–BCIL framework offers transferable design principles for EMI STEM education and provides a coherent foundation for future pedagogical and research development.
Footnotes
Acknowledgements
The author would like to express sincere appreciation to the editors of Language Teaching Research, Hossein Nassaji and Maria del Pilar García Mayo, as well as the anonymous reviewers for their time, careful reading, and constructive feedback. Their insightful comments significantly strengthened the rigor and clarity of this manuscript. The author is especially grateful for the opportunity to learn from their guidance throughout the review process.
Ethical Considerations
This study received ethical approval from the Institutional Review Board of King Mongkut’s University of Technology Thonburi, Thailand (certificate number KMUTT-IRB-COA-2025-041) prior to data collection.
Informed Consent
Informed consent was obtained from all participants, and participation was voluntary. Participant confidentiality was maintained throughout the study.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Artificial Intelligence Tools
AI tools were used solely for language refinement and clarity. The study’s conceptualization, design, data analysis, interpretation, and reporting were conducted entirely by the author.
Data Availability Statement
The data supporting the findings of this study are available from the corresponding author upon reasonable request. Documentation confirming professional language editing of the manuscript can also be provided upon request.