Realizing the Sustainability of Portfolio Assessment in Second-Language Writing

Portfolio Assessment in L2 Writing and Sustainability

In the past two decades, assessment has increasingly emphasized learner-centred assessment that enhances active involvement and self-regulated learning, whereby learners monitor and regulate actions towards their learning goals (Clark, 2012). Complementing this shift, PA, whether paper-based or web-based, has gained popularity worldwide in educational settings due to its emphasis on student empowerment within the assessment process (Duong et al., 2011). In L2 writing classrooms, portfolios that dovetail with the process-oriented approach are developed over time and serve as a means of communicating about student growth and development (Stiggins and Chappuis, 2012), hence making progress and improving performance. Notwithstanding a robust body of literature that has substantiated the potential merits of PA, there is slow progress in its widespread implementation due to interacting influences at the personal, institutional and cultural levels (Fulmer et al., 2015). At the personal level, literature documents a lack of knowledge and skills from teachers, resulting in students becoming passive in the assessment process and unable to make productive use of assessment information to improve their work (see Deeley and Bovill, 2017). For instance, Bader et al.'s (2019) study investigated students’ perception of formative feedback from teachers and peers as part of PA at two teacher education institutions in Norway. Findings revealed that students appreciated teacher feedback given its specificity and concrete guidelines on how to improve their writing. Some students, however, admitted they lacked the capacity to make improvement due to inadequate knowledge and skills in understanding and processing feedback, reflecting students’ low feedback literacy. Beyond individual factors, there are additional pervasive influences such as institutional mandates and cultural norms and values on the implementation of PA. In Duong et al.'s (2011) study, researchers constructed a framework for assessing English-as-a-foreign-language (EFL) students’ competence in writing portfolios at Vietnam National University in collaboration with teachers of the institution. Despite process-oriented portfolios being established for six years, norm-referenced interpretation held sway, with the absence of clarity on the assessed construct. This institutional constraint hampered the effectiveness of the PA. Likewise, Lee et al.'s (2019) study investigated two teachers’ attempts to integrate PA into two Hong Kong primary writing classrooms, with a focus on teachers’ and students’ perception of the benefits and challenges in its implementation. Results indicated a lack of critical mass in the teaching context as other colleagues were neither involved in the assessment innovation nor prepared to practise PA, illustrating inadequate support from the school as a whole. Relatedly, Chen (2006) investigated two 7th grade English classrooms in Taiwan, where PA was implemented for two semesters. Despite positive effects of PA on students’ confidence and ownership of the assessment process, teachers’ PA practice was hindered by test-driven CHC, teachers’ self-perceived inadequacy in assessment literacy, and students’ perception of PA as an unreliable measure. Previous research studies thus illuminate the challenges of implementing alternative assessment practices and the short-lived nature of many assessment initiatives. In assessment literature, empirical research on the conditions for sustaining assessment initiatives in L2 writing is underexplored, and the success in sustaining PA remains largely elusive. As noted by scholars (Hyland and Wong, 2013), sustainability in innovation is crucial to enhance students’ learning. To leverage the learning potential of PA, more research is needed to understand PA sustainable implementation from a multi-dimensional perspective that addresses the influence of key factors. This study thus aims to explore how the sustainability of PA implementation can be brought into fruition.

Framework of the Study

This study draws on the conceptual framework of Fix et al. (2021), informed by literature investigating factors influencing the sustainability of educational innovations. Given the multi-faceted nature of innovations involving the interplay between individuals, institutional infrastructure and larger sociocultural systems (e.g., King, 2016; Rikkerink et al., 2016), this framework provides a useful lens for understanding the intricacies of this study's assessment innovation (see Figure 1), drawing specifically from distributed leadership, context-conscious leadership, vision and goals and flows of learning. Distributed leadership does not confine leaders to those with formal leadership positions; rather, influence and agency are shared among members at different levels of the school. Context-conscious leadership concerns the management of contextual influence and involves striking a balance between leadership decisions, policy and teachers’ actions. Vision and goals inform the collective behaviour of teachers and administrators, aligning team members’ actions with the objectives of the school. Finally, flows of learning centres teacher learning processes as the backbone of innovations, involving sense-making at an individual and collective level to ensure feed-forward flow of learning (i.e. exploration of new practices) and feedback flow of learning (i.e. reflection on one's practice). Teacher learning could be reinforced through teachers’ perceived autonomy, competence and relatedness, which call for teachers to make volitional choices of their practice, to achieve mastery of tasks relevant to their goals and to join a supportive environment to give teachers a sense of connectedness. Moreover, teacher learning undergoes four interrelated sub-processes: intuiting, interpreting, integrating and institutionalizing. Intuiting occurs at an individual level, which concerns personal interpretation of experience. Interpreting involves the refinement of intuitive insights, where the individual learning process is explicated, demonstrated and shared with colleagues. Integrating encompasses the development of shared understanding among individuals, resulting in coordinated practice. Institutionalizing or sustainable development occurs when the processes are systematically supported and endorsed by the school organization.

OPEN IN VIEWER

Figure 1.

Conceptual Framework of Sustainable Innovation (Fix et al., 2021).

Drawing on the theoretical framework and informed by existing studies on the sustainability of educational innovations, this study addresses one central research question: what influential forces mediate the sustainability of portfolio assessment implementation in the elementary L2 writing classroom?

Context of the Study

The present study involves one elementary school in Hong Kong. Since two Primary 6 teachers recognized that students lacked autonomy and reflective capability in the writing process, they began PA as an assessment innovation on an individual level four years ago (at the time of the study) after receiving professional training from a university educator. PA was carried out within a process writing approach, where the whole portfolio procedure comprised goal setting, self and peer assessment, teacher feedback, an error log, and reflection to develop students’ regulated capacities (see Figure 2). Delayed evaluation was employed, where the whole portfolio was graded summatively at the end of the academic year for reporting purposes. With accelerating technological advancements and the outbreak of the COVID-19 pandemic, the school incorporated some digital elements into the writing portfolio, like flipped learning videos assigned as homework to familiarize students with the target text type before writing. Teachers used a virtual whiteboard to collaboratively brainstorm in the instructional scaffolding phase, and Google Classroom provided students with content-related feedback on their written drafts.

OPEN IN VIEWER

Figure 2.

Portfolio Assessment Within a Process Writing Approach (Adapted from Mak and Wong, 2018).

During the initial year of PA implementation, the two teachers showcased the work of PA to other Primary 4 to 6 English teachers through English team meetings and workshops. The practice of PA then extended to all Primary 4 and 6 classes, led by the two teachers who were assigned to teach at each grade level, in the second year. From the third year until now, all Primary 4 to 6 classes have implemented PA in their English classrooms.

Data Collection and Analysis

This paper represents part of a larger study exploring the implementation of PA in L2 writing and the learning trajectories of both teachers and students. Due to the scope of this paper on the sustainability of PA, we relied on qualitative data gathered in the final year of the study, including individual semi-structured interviews with the principal, English Panel head and four teachers (Allison, Caitlyn, Megan and Violet – pseudonyms) who possess the most extensive experience in PA accumulated in this school, six rounds of classroom observations focusing on different phases of the portfolio process, and six observations of teachers’ team meetings. The interviews, lasting approximately 40 minutes, were carried out with each of the participants in their first language, Cantonese, according to their wishes, and audio-recorded for transcription and analysis. Classroom observation and team-meeting data were reviewed and triangulated with interview data to yield contextual information on teachers’ design and use of pedagogical material as well as PA pedagogy.

Data was analysed following a recursive and iterative process (Miles et al., 2013), where the researchers moved back and forth through the different sources of data, research question and theoretical framework with reference to the literature on educational innovation sustainability (e.g. Fix et al., 2021; Rikkerink et al., 2016). The interview transcripts were read and re-read to identify codes that reflected successes and challenges in sustaining PA. The identified codes were further brought into analysis by drawing on the features of sustainability educational innovation proposed by Fix et al (2021). These emerging codes were then examined by scrutinizing the transcribed classroom observation and teacher meeting data to add richness to the interpretation. To ensure the trustworthiness of findings, member checking was employed by inviting participants to examine our interpretations and conclusions of the interviews. The two researchers performed data analysis independently, followed by collective discussion until consensus was reached.

Distributing Leadership at Different Levels as the Catalyst for Sustainability

The principal approved a distributed approach to leadership practices, where she dispersed decision-making authority to teachers. Recognizing that no single leader possesses the necessary knowledge and skills to address educational innovations, she expressed how she ‘trusts the teachers’ professionalism’ and values their ‘strong subject knowledge’. Changes to the English writing curriculum, therefore, were not at the prerogative of the principal as teachers’ expanded leadership roles allowed the whole English team to decide on necessary changes. As she put it: ‘If they [the English team] have their rationale and we think that it [the innovation] is worth pursuing, then by all means they can go ahead!’ (principal).

Considering that autonomous development of PA innovation was part of teachers’ jobs, the administrators boosted teachers’ ‘motivation to work on the change’ (Megan), making them more committed to the innovation. To support positive distributed leadership in practice, the principal afforded teachers infrastructural essentials including offering extra manpower, adjusting teaching load and allocating additional resources to writing classrooms. Two teachers were assigned to each class where they worked collaboratively to deliver instruction and provide detailed feedback to students. On the whole, distributed leadership fostered a sense of ownership, boosted teachers’ motivation to engage in the change process and encouraged individual and collective learning, which altogether cultivated sustainable development of the innovation (Meyer et al., 2022).

Buffering Threats and Hurdles to the Innovation Through Context-Conscious Leadership

Another essential element leading to sustainability is the strategic leadership role that the principal assumed. Acknowledging the potential pressure of embracing innovation processes, the principal actively exploited opportunities in the environment to facilitate the innovation. As such, there were professional development (PD) opportunities within and beyond the school, which enhanced teachers’ autonomy and developed teachers’ continuous growth in their professional competence. Sharing good practices and concerns among colleagues during co-planning sessions, peer observations and regular English team meetings fostered meaningful interaction and professional learning, and instilled a shared commitment to PA that could see the team through threats or hurdles to the innovation. Allison summarized:

We have sharing within the grade level, peer observation, co-planning and English team meetings. During the meetings, we’ll discuss whether there are problems with our writing curriculum, whether the worksheets make sense or not, whether they are of appropriate level to students. We’ll discuss and complement one another's strengths.

In addition to internal PD opportunities, the principal also built teacher competence with an outside expert to advise and make recommendations on the writing programme, and encouraging teachers to share their practice with other schools. Not only did this academic exchange help teachers strengthen their knowledge about PA and enhance their professionalism, it also gave them insights into how to improve their PA practice further, and simultaneously built up their self-esteem. As the principal proudly shared: ‘not only has our English Department Head been invited by the Education Bureau to demonstrate how PA is put into practice, our teachers have done sharing with other schools for a number of years’.

Teachers thrived because of administrators’ leadership in constructing professional learning spaces to encourage meaningful support from one another, mitigating threats to the longevity of the programme. The investment demonstrated by teachers to help each other solve problems and respond to challenges indicates a high level of teacher agency within the innovation. Such PD opportunities were further reinforced by the principal's effort to extend teacher learning beyond the school walls (Fullan and Knight, 2011).

Futuring to Promote Sustainability Through Shared Vision and Goals

Another influence was a requisite synergy between the English teachers and school administration to cultivate a shared vision and goal about teaching, learning and assessment with the PA innovation. Through team learning, coaching, and transparency across pedagogy and assessment, Fullan and Knight (2011) note the transformative power of aligning stakeholders at all levels around a common goal to build a strong, cohesive foundation for change.

The English Department head believed that a clear vision would make her team resonate with the school direction and move towards a common goal. Before the practice of PA, she shared the rationale of implementing PA so that ‘teachers had a reason for practising PA and that they understood the meaning behind, like, how students could benefit from it. Only then would they believe such practice was valuable and worth the effort’. She also believed that the alignment around a common vision required consistent investment, especially as the vision became continually refined according to their specific context. To rally the team around a shared vision, she regularly reminded them of the rationale behind implementing PA in departmental meetings. Allison appreciated such reminders as colleagues might forget the purpose of PA and ‘lose the enthusiasm over time’.

Defining, conveying and reinforcing vision seemed to result in increased buy-in from the team; the teachers were more receptive to PA and believed in the approach. As Caitlyn said:

I’ve listened to some sharing and heard how teachers go about doing things. And I heartily approved of the approach and that's when I followed what everyone else was doing. So, it's necessary to let teachers know about PA. If they don’t approve of it, it won’t get filtered down to the classroom level.

Rallying teachers around a common vision was a gradual process. Allison initially had hesitations about adopting PA because PA works within a process approach, which results in fewer pieces of writing than a product-oriented approach. Yet with time, Allison saw the improvement students made and became increasingly receptive to the approach, recognizing that it was more about quality than quantity. Allison stated: ‘At the beginning, I did have hesitation. I was thinking the students were only required to do a few pieces of writing in a year …. But gradually, I find that this approach works well … the students write better.’

Common vision and goal for PA is also reinforced by successful learning outcomes. Teachers admitted that students’ assessment literacy was enhanced through PA in that students developed greater capacities to judge the quality of their writing and that of peers according to assessment criteria. Students could also monitor their progress towards goals they had laid out, act upon feedback received from teachers and peers, track progress using an error log, and set further goals to challenge themselves. Allison commented that ‘PA made the students feel they weren’t only a writer but also an assessor.’ Caitlyn mentioned that ‘students successfully monitored their progress using the error log’. Exercising a repertoire of strategies, students developed greater autonomy, became more self-regulated and assumed greater control of their own learning (see Mak and Wong, 2018).

As the PA innovation developed and expanded across multiple years, the vision and goals of the programme also matured. Initially, the administrators and teachers hoped to promote autonomy and self-reflection through PA. As indicated by Allison's comment, the vision evolved to include students’ identity development as writers and assessors in their L2. The students’ increased agency and enhanced ability to engage in the metacognitive process of learning and writing fuelled teachers’ commitment to sustaining the programme.

Supporting Sense-Making and Change Through Individual and Collective Learning Flows

Throughout the innovative processes, teachers’ learning encompassed the interaction between individuals at multiple levels including teachers, colleagues and administrators. Not only did the teachers participate in professional dialogues and collaborate with colleagues, they also engaged themselves in continuous reflection and learning both individually and collectively. Learning is thus a joint venture of exploration involving sense-making that drives the teachers to constantly review and fine-tune their pedagogical material and reflect upon their instructional practice. There was also a supportive and stimulating learning environment, where teachers were able to share their concerns arising from the innovation.

The high level of collaboration was instituted through teacher co-planning and co-teaching, which in turn fostered deepened levels of relatedness and collegiality in the learning process. In co-planning sessions, teachers pooled expertise and delivered instruction that catered to diverse student abilities. To successfully collaborate and co-teach, educators collectively considered the intimate relationship between planning, instruction and assessment, paying particular attention to the unique needs of language learners (e.g. linguistic adaptations, bridging and building content knowledge) (Honigsfeld and Dove, 2010). Rather than rehashing pedagogical material of previous years, teachers worked together to review, scrutinize and modify material for improvement. For instance, they would ‘think about how to make the material more engaging and interesting’ (English Department head). More concretely, Megan added:

The material developed for the new unit ‘My Dream School’ was too simple. The mind map, with just a circle with ‘My Dream School’ in the middle was far too vague and did little to help students develop their ideas. Knowing this problem, all of us sat together and discussed how we could add more details to the mind map to guide students’ thinking … We’d make materials harder or easier based on students’ abilities.

The teachers appreciated meaningful collaboration among team members. Although the English Department head found it ‘hard to persist’ if they ‘didn’t do this collectively as a team’, Caitlyn believed working in isolation ‘would have been a totally overwhelming task’. This collaboration was instrumental as English teachers also incorporated technological tools into their teaching, sharing best practices for brainstorming in the instructional scaffolding phase with Google Jamboard. Class observations showcased sustained improvements as teachers adopted and continuously innovated with technology. Such collaboration promoted relatedness among colleagues, leading to expanded trust and creating a cohesive community with strong team dynamics. Not only did the flourishing relationships within the team enable communication and learning to flow fluidly between team members, they also fostered solidarity and commitment. Allison shared how she loved her team and commented that the teachers were ‘willing to discuss unconditionally every year, say how to fine-tune the PowerPoint for teaching according to the ability of different students’ and that the whole team worked cohesively to share the workload and committed themselves wholeheartedly to student learning.

Individual and collective sense-making also drove the team to constantly reflect upon their refinement of their practice, which addressed both technical/pedagogical aspects of teaching (i.e. practice and beyond practice), and the internal dimensions of teacher identity (i.e. philosophy, principles, theory) that drive change (Farrell, 2015). Collectively, these varying dimensions of reflection built capacity for teachers to deepen their learning and increase competence from teachers to support PA longevity.

Violet believed it is necessary for every teacher to ‘stay alert all the time and stay reflective’. She reflected on her own practice, questioned her taken-for-granted assumptions regularly, and always found ways to realize her plan for improved practice. Similarly, Megan commented that critical self-reflection made lessons more fun and promoted active student participation, and Allison engaged in reflection-on-action (Schön, 1987) where she scrutinized her teaching approaches and sought advice from her colleagues on practices when she was ‘stuck at or unfamiliar with something’. Leaning on her colleagues for individual sense-making, she stated:

There was once when I felt that my approach didn’t work well and didn’t scaffold the students towards my expectations. I then sought advice from my colleagues on what kind of input I was supposed to give so that they (the students) could produce the written product … It's all through discussion that we become better.

To ensure that the team's professional growth in competence was on track and that they were pursuing the common goal, administrators kept track of each staff member's performance and work progress to identify and overcome difficulties and conflicts. Every semester, teachers turned in their students’ portfolios with all learning materials. The department head then carried out periodic checks to monitor teachers’ work and ensure they adhered to the established guidelines for effective instructional approaches for PA implementation. For instance, teachers were asked to write questions on student drafts to help them generate ideas and enrich the content of their writing. If teachers were found not complying with these guidelines, ‘the Department Head will meet with individual teachers and ask them why they aren’t following the guidelines … she’ll provide teachers with advice’ (Megan).

The English Department head added that book checking is necessary to ‘ensure that the quality is there’ and that colleagues understood ‘how to put the guidelines into practice’. Research on the relationship between instructional leadership and student outcomes in English-as-a-second-language (ESL)/EFL contexts indicate that these monitoring practices are effective in schools with an established culture of trust among teachers (Kwan, 2016). With mechanisms for monitoring instructional practice, teachers were more likely to see the value in the PA innovation, which, in turn, supported the sustainability of this practice.

Through the supportive environment which promoted openness to discussion, exchanges of ideas and collective sense-making, the team was equipped with enhanced assessment literacy with expertise in developing students’ capacities to understand, interpret and use feedback to improve their writing. Violet remarked on having a more comprehensive picture of PA and a greater capacity to develop material to enhance learning over time. She reflected:

When PA was first created, we only had a faint idea about how to go about PA or we hadn’t in fact mastered the skills of when to do what, and how best to do it. Now, it seems we have a much clearer picture. Some teachers even make suggestions about the writing material to help students become more aware of the goals.

Allison believed her skills in PA were honed through practising PA and collaborating with colleagues as she could ‘observe more experienced teachers and listen to the sharing from other colleagues’. She expressed that she had mastered PA skills, and could even improvise her teaching according to students’ responses to teaching activities she had planned in lessons. Engaging in this classroom-based research practice supported teachers’ simultaneous development of skills, knowledge and dispositions in writing instruction, ultimately enhancing their self-efficacy as teachers (Mak et al., 2022). The competencies possessed by teachers thus ensure ongoing improvement of teaching while promoting student learning.

Overall, the level of learning influences the sustainability of PA innovation. Specifically, the transition from intuiting to interpreting was evident, where PA, initiated by the two teachers, was explicated and demonstrated to the team. Through mutual understanding of the philosophy and rationale behind PA, the team underwent integrating through pursuing a common vision and goal and implementing PA in a coordinated manner. Institutionalizing occurred when sustainable development of PA was in place. As the findings show, driving PA to the road of sustainability is a complex enterprise and demanded collective effort of actors at multiple levels.