Stimulating music educator worldview change through a pedagogy of provocation,critical thinking and peer-review

Why should the worldview of music teachers be challenged?

The introduction of technology into society and education has brought about radical changes in the past two decades (Benedict & O’Leary, 2019; Peters, 2017). Specifically, in the field of music, it has changed three key areas: the instruments and techniques that a number of music professionals use – from instrumentalists to DJs to producers to composers (Bell, 2013; Ruthmann & Mantie, 2017; Smith et al., 2017); the educational methodologies that are available for classroom music instruction (Green, 2002, 2008; Waldron, 2013; Waldron et al., 2020; Waldron, 2018); and how, how often, and in what forms students experience music outside of school (Hone, 2017; Lamont et al., 2003; Temmerman, 2005; Youth Music & Ipsos MORI, 2019). But due to the way school music graduates are trained as classroom music educators in university music departments, teachers’ experiences and their expectations largely derive from a traditional, Western art music (WAM) approach to music education (Bull, 2019; Bull & Scharff, 2017; Dwyer, 2016; Green, 1988; Philpott, 2010). Following this training, many music teachers have a worldview that treats the instruments, forms, and approaches of WAM as having priority over non-Western music and new technological/digital developments. As a result, these teachers do not include – or may explicitly exclude (Kallio, 2015, 2017a, 2017b) – recent developments in music from their classrooms, rather than taking a pluralist approach that encompasses both the old and the new (Humberstone, 2017; Webb & Seddon, 2012). The students these teachers see in their classrooms, on the other hand, have neither a WAM background nor a worldview that privileges it (Bull & Scharff, 2017; Green, 2008; Youth Music & Ipsos MORI, 2019), leaving teachers and students with quite different experiences and understandings of music. So although music is extremely meaningful to today’s music students, the music they value is typically not the music they meet in the classroom setting (Lamont & Maton, 2010; Youth Music & Ipsos MORI, 2019). And although sophisticated music theory underpins both the old and the new, the 21st-century music careers that are available to all young people interested in studying music may require competencies that have little or nothing to do with WAM instruments, forms, and practice, and much to do with culturally diverse musics, technology, digital music, digital instruments, improvisation and experimentation.

The University of Sydney saw an opportunity to address this situation through an internationally available Massive Open Online Course (MOOC), undemanding in its time-commitment, and giving course participants immediate access to a community of colleagues striving to come to terms with and decide how to respond to the latest music education research Humberstone, 2016. Through video with narrative commentary by the lead author of the study, course participants have the opportunity to virtually visit schools that take markedly different approaches to technology in music education (for example, Humberstone, 2018b, 2018d, 2018e), meet colleagues who have different approaches to whether or how to combine WAM and 21st-century developments (for example, Humberstone, 2018a, 2019b, 2020) and listen in on conversations between the lead researcher and musicians and educators who have given the topic of contemporary music education a great deal of thought, or even a lifetime of devotion (for example, Humberstone, 2019a, 2021a, 2021c). The aim of the interactions is to provoke critical thinking in participants by engaging with the material and each other to reflect on how they can make sense of the provocative juxtaposition of seemingly contradictory research and practice the course presents, and to what extent and in what specific ways they might modify their own classroom practice. This paper looks at whether and how the MOOC succeeded in its aim of challenging participants’ worldview through such critical thinking: in parallel research, we examined how the use of public blogging in the MOOC engendered their vulnerability to change Humberstone et al., 2020.

Music teachers’ worldviews in the literature

The cycle of traditional WAM education engendering an unquestioned continuation of the same curriculum, generation after generation, has been characterised as creating a Bourdieusian cycle of cultural reproduction (Bull, 2019; Dwyer, 2016; Green, 1997). It is well established in the literature that success in the music education system of so many countries is measured in exam grades on majority WAM instruments (Bull & Scharff, 2017); in school exams in Western-majority musics (Elpus, 2015); towards music teaching qualifications steeped in the WAM tradition qualifying teachers to teach often Eurocentric syllabi (e.g. Bull & Scharff, 2021; Cox & Stevens, 2010; Wright & Davies, 2010). School students successful in the school segment of the cycle can become qualified teachers, then transmit this model to the next generation, producing another cohort with similar values and worldviews about what is important in music education (Green, 2012; Philpott, 2010; Temmerman, 2005). The great majority of music teachers identify as performers (Hargreaves & Marshall, 2003) rather than as improvisers (Bernhard, 2013; Wright & Kanellopoulos, 2010) or composers (Randles & Sullivan, 2013; Strand, 2006). Having most likely studied a pre-digital instrument (Welch et al., 2010) and/or in a pre-Internet world, the teacher with a WAM background is also more likely to actively resist the adoption of new technologies (Bledsoe, 2017; Gall, 2013). Music educators, as a population, resist change (Peters, 2017; Philpott, 2010).

Breaking the cycle and shifting worldviews

The finding that music educators are resistant to change is further exacerbated by evidence of the difficulty in changing anyone’s worldview on aspects important to their identity, however logical the person may be (Slovic et al., 2002, 2007). Contrary to thinking in the first half of the 20th Century – when it was believed that early childhood represented the ‘impressionable years’– 21st century socialisation and psychological development models suggest that worldview does not become fixed until adolescence or even early adulthood (Vollebergh et al., 2001), although it is still then principally informed by the worldviews of the subject’s parents (Barni et al., 2011). Even more recently, Kahan (2012) has exposed the influence exerted on worldview by the opinions of other members of groups with which an individual identifies, favouring such views over evidence derived from outside the group (Cultural Cognition Theory). Self-report of open-mindedness to evidence-based learning does not correlate with worldview change in response to reasoned argument (Tan et al., 2016). In fact, invoking ‘facts’ in response to unsubstantiated statements by the self-identified ‘open-minded’ can actually strengthen their certainty in their unsupported beliefs (Nyhan & Reifler, 2010).

So how can change take place? To make such a transformation, individuals need the opportunity to question their own beliefs and assumptions rather than being ‘told’ what is right or wrong (Lewandowsky et al., 2012). Such critical thinking could be a key process in transforming the music education practices of those stuck inside the WAM-education cycle; if it is to be broken, teachers may need to confront the reality that they are potential ‘blockers’ of creative, practice-based and technology-infused paedagogies. We fully define critical thinking with reference to the literature as part of our methodology, below: for the purposes of understanding the intervention procedures, it can be identified as a disposition to questioning one’s own beliefs (Ennis, 1996); relying on the best evidence for forming new beliefs (Paul, 1984); and seeing the process of critical thinking as a moral or ethical imperative (Bermingham, 2015).

Intervention procedures: Paedagogies of provocation

The MOOC incorporated a variety of paedagogies aimed at encouraging critical thinking (Humberstone, 2021b). Central to this approach was the determination to expose learners to ‘problems relating to themselves in the world and with the world’, so they would feel ‘increasingly challenged and obliged to respond to that challenge’ (Freire, 1970, p. 54) in each week’s work. These problems were purposefully designed as provocations promoting critical thinking (see, e.g. Boler, 1999; Bozalek et al., 2013; Mills, 1998; Postman, 2009; Trifonas, 2003). Each week we juxtaposed two examples of evidence-based literature that promoted differing solutions to a music education problem, but prevented simplistic, dichotomic thinking, such as learners facilely classifying one as ‘right’ and the other as ‘wrong’ (Humberstone et al., 2020).

For example, the MOOC presented research representing both of Craft’s (2012) categories of technology’s potential impact on children’s lives: either ‘childhood at risk’ or ‘childhood empowered’ (p. 176). Participants read literature showing the negative impact of screen-based media on children’s development (Brooks, 2015a, 2015b; Christakis, 2010; Christakis et al., 2004; Christakis, 2011; Zimmerman & Christakis, 2005), but also on its remarkable benefits (Hein, 2013; Mitra & Crawley, 2014). Both points of view were presented through literature on education broadly and music education specifically, as well as videos of diverse current practice filmed specially for the MOOC. The latter included footage filmed in a technology-free Rudolf Steiner (Waldorf) school (Humberstone, 2018d) with an extensive Orff-Keetman Schulwerk programme, followed by scenes from a technology-embedded programme in a school with purpose-built recording studios and stages (Humberstone, 2018e); and an interview with Ahmad Sarmast, the founder and director of the Afghanistan National Institute of Music in Kabul (Humberstone, 2020), in which instruction is undertaken online by teachers across Europe.

The value of musical pluralism and the importance of acknowledging the sophistication of the musical cultures in which children participate was discussed in reference to the literature cited above. Observations of and interviews at a Musical Futures programme in Melbourne (Humberstone, 2018c) was contrasted with both the Orff approach, as well as an interview with leading advocate of traditional (WAM) music education, Richard Gill (Humberstone, 2019a), of Australia (discussed further in Humberstone et al., 2020). A full overview of (1) the related MOOC video content, (2) provocations presented through the juxtaposition of the content and in the prompt for each week’s blog, and (3) the rubric for peer marking is available in the MOOC or online (Humberstone, 2021b).

Assessment through blogging, the importance of blogging publicly, and peer review

In keeping with recent research on best-practice in MOOCs, assessment for the course was designed as a series of required blog posts that could be characterised as public, regular, and scaffolded (Freeman & Brett, 2012; Zandi et al., 2014). The first blog post prompt was designed to allow the researchers to assess not only critical thinking on the prescribed topic, but also the participant’s command of technical ‘blogging’ skills. The rest of the blogs had fewer restrictions and could take the format of, for example, a song or videoed presentation (i.e. a vlog). Blogs – unlike typical discussion forums or assignment submissions in Coursera and similar Learning Management Systems – were required to be published externally and publicly to enhance ‘real life’ accountability: participants published as a teacher, or pre-service teacher, representing their learning and resulting philosophy to the world.

This approach arose from research identifying reflective public blogging as stimulus for both deeper thought and CT due to the responsibility inherent in such ‘exposure’ of their views (Woo & Wang, 2009), and the finding of the promotion of CT in asynchronous discussion is supported by further research (Marra et al., 2004; Perkins & Murphy, 2006). Public blogging has also been found to stimulate high levels of student engagement, which continued after the conclusion of the course in which it was required (Giarré & Jaccheri, 2008; Turvey & Hayler, 2017) and to encourage ‘building of professional knowledge and understanding’ (Turvey & Hayler, 2017, p. 43). We have investigated this literature in more depth in an earlier publication (Humberstone et al., 2020).

Participant learning was assessed through peer-marking, and to ameliorate inconsistencies in evaluations, marking rubrics were developed. These rubrics requested that peers evaluate not the content or conclusions drawn in each blog post, but the process of thinking that had taken place. This was designed to encourage critical thinking in response to the ‘provocations’ presented each week, whether or not CT was explicitly named in each rubric: in some cases CT was marked, and in others traits of CT were identified. In terms of accuracy and impact on outcomes (efficacy), a properly set up peer review has been shown to match instructor review for quality (Bachelet et al., 2015; Luo et al., 2014). Additionally, peer review has been found to contribute to opportunities for critical thinking and other higher-order thinking skills (Luo et al., 2014).

Population, dataset and analytical methods

At the time of identifying the population, 1,633 music teachers were ‘active learners’ (Coursera’s term) in the MOOC. Here we refer to this as the full dataset. However, as is common with MOOC enrolments (Yang et al., 2013), many participants had not completed very much (or even any) assessable work.

To answer RQ1, and locate participant reflections (public blogs) to analyse for evidence of CT, we adopted Clow’s (2013) funnel of participation model, in which a MOOC is defined as a unique educational experience in which the ‘drop out’ rate is expected to be unusually high because ‘there is less initial commitment, so the filtering happens at a later stage’ (p. 187). Clow’s funnel of participation was therefore applied to only include learners who engaged further into the course, which in this case included 249 participants who had submitted more than one assessment, including the last. Given an anonymous participation code by the Coursera system, we used true random generated numbers to extract a sample of 50 blogs of 10 participants. We refer to this subset of the full dataset as the funnelled population. Most participants’ blogs were submitted as written text, although a few used other media. Using Coursera’s participation codes, we were able to extract the peer review work of these same participants and the peer review of their work by other participants. While participant data was anonymised by Coursera, some educators shared biographical information about themselves voluntarily on their blogs, which allowed us to reveal a wide range of countries represented (Australia, Brazil, Canada, Denmark, Russia and the USA), and age groups (early career to in-retirement). Eight of the participants were male, one was female, and one did not reveal their gender. The blogs and peer feedback comments were coded and analysed for evidence of CT with basic content analysis (Weber, 1990), a process outlined in more detail below.

To answer RQs 2 and 3, we analysed the full dataset in two ways. First, we performed a statistical analysis of engagement through the course and peer-marking through the course, to identify patterns in learning and success. Peer assessment marks for all funnelled participants were analysed statistically, and compared with the full sample. Second, we mined the text in the qualitative peer-feedback (which was given alongside peer-marking) and context-network maps (Kapiche, 2023) were generated from the peer-assessment comments to reveal broader patterns and topics of discussion.

Content analysis of blogs for evidence of critical thinking (RQ1)

In interrogating the first RQ, learners’ assessment submissions, and then their peer-assessment marks and comments in response were examined for evidence of CT. To analyse the submissions themselves at the highest level, basic content analysis (Weber, 1990) was performed to ‘apply deductively generated codes derived from prior theory and research’ (Drisko & Maschi, 2015, p. 86). Content analysis was also chosen because of its currency in the CT literature: Turvey and Hayler (2017) and Freeman and Brett (2012) used content analysis to analyse data from the blog posts of their education students, and there is already over two decades of research into CT in asynchronous online forums that adopt a range of content analysis methodologies relevant to each (Marra et al., 2004; Newman et al., 1995; Perkins & Murphy, 2006).

Deductive codes were developed by applying definitions of CT in the literature (cited below) with the paedagogies (provocations), tasks, and rubrics that were set in the MOOC, to create a series of categories specific to music education and CT. Seven codes were initially written by the lead researcher, and then, consistent with basic content analysis, reduced to five by the rest of the research team ‘to identify and clarify topics of interest’ (Drisko & Maschi, 2015, p. 25), to establish their reliability, and to avoid bias. Assessment submissions were classified as showing signs of CT if they met any of the following categories:

Each educator’s blog was read multiple times and coded for each of the above CT qualities. By coding each blog in the order they were written, patterns (increasing evidence of CT as time went on, e.g.) were easily identified. Codes were recorded as shown in Table 1, below, alongside additional researcher memos and quotations that justified each code. Where participants showed evidence of resisting CT (e.g. dismissing the use of technology in music classes despite the evidence presented), such expressions were coded as negative categories, consistent with methods adopted in the literature discussed (Wang & Woo, 2010; Woo & Wang, 2009).

OPEN IN VIEWER

Table 1.

Content analysis coding of CT categories.

Pseudonym	Asmt. 1	Asmt. 2	Asmt. 3	Asmt. 4	Asmt. 5
Shannon	B	−C	D		−B, D
Steve	E		B, E
Ivy	A	−C, D	A, B, C	B	A
Kelly	E	B, −C, E	A, E	B, C	A, B, C, D, E
Aaron	A	−C, −D	B, −C	−A, B	−C
Paul	A, B, D, E	A, C, D	B	B, C	A, B, C, D, E
John	E	A, B, C	A, C, E	B	A, B, C, D, E
Darren	A, B	A, B, D	B		D
Annaliese	A			D	A, B
Richard	A	A, D		−B, C

The use of the minus symbol identifies instances where the writing was considered contrary to the category of CT that it precedes. ³

Statistical and context-network analysis of quantitative and qualitative assessment data

RQs 2 and 3 were investigated through analysis of a combination of both the funnelled and full dataset of written feedback on one another’s blogs, and statistical analysis of quantitative data from Coursera on engagement and peer-marking. The latter data was especially useful to identify which assessment tasks prompted the most CT (RQ 3) over the full cohort of over 1,600 participants.

Kapiche, text-analysis software, was used to create visual connections between emerging topics in all of the written peer feedback. Context-network maps (Kapiche, 2023) show ‘the expanded topic with all of the terms included’ (para. 15) , and the thickness of lines between topics show the strength of the connection. This was particularly useful for identifying the relationship between CT exhibited and identified in participant work and feedback (RQ 2).

As established above, detailed descriptors were included for each rubric criterion to make the peer marking process easier and more consistent: an example of one rubric can be seen in Humberstone, Zhao, & Liu (2020), page 603. The rubric removes irrelevant marking components (written style, grammar, spelling and so on), and simply asks peer reviewers to quantify as many CT elements as possible (e.g. ‘many’ vs. ‘several’ vs. ‘at least two’). ²

Content analysis of participants’ blogs

While the 10 selected participants cannot be claimed to be representative of the 249 in the funnelled population, their 50 cumulative blog posts provide nearly 32,000 words of detailed data: honest and thoughtful responses to the provocations. Table 1 presents the coding of participants’ blogs. At first glance, it provides an overview of whether and for which participants’ CT might have increased or decreased over the duration of the course. It is also simple to compare assessment columns to summarise whether some tasks (provocations) were more successful at eliciting CT than others.

In Humberstone, Zhao, & Liu (2020), we investigated the establishment of a Community of Practice (Wenger, 1998) in the MOOC through both the process of public blogging but also anonymous peer feedback within the Coursera system. We found that ‘the experience of engaging publicly via social media engendered a vulnerability that may have made those new to the field and experienced professionals alike more open to change’ (p. 591), although this did not apply to all participants. For example, in that initial analysis of this same data, we noted that some participants’ blogs were coded to show greater numbers of CT categories than others. Some teachers, such as Shannon and Annaliese, hunkered down in their already-formed worldviews, or pushed back against progressive ideas presented in the videos, one writing against the presented informal learning pedagogy (Green, 2008): ‘Why should we deprive our students of the understanding behind the written language of music? . . . Let’s not get caught up with current fads’.

In contrast, Table 1 clearly shows that some participants engaged more with the challenge to think critically as the course progressed. John and Kelly brought strong and well-expressed worldviews of music education to the course in the first week, where Kelly wrote ‘[t]he best approach, in my view, is to neither stand stubbornly still nor run blindly forward, but simply to advance cautiously, striving to glean the positives from various approaches while avoiding their respective pitfalls’. These strongly held and articulated opinions, based mostly on their teaching experience, did not, however, limit either participant’s approach to challenging their own worldview, nor to adopting new ideas from the literature and exploring them further. Both chose to do further reading on informal learning, with John writing ‘I found Lucy Green’s approach to informal learning worth investigating and pursuing’, and Kelly reflecting ‘I love the idea of taking a less systematic approach and allowing students to learn “haphazardly”, naturally, like pop musicians do, recognising that students are human beings and not the automatons that traditional education assumes they are’.

In the final assessment, the ‘manifesto’, both participants sought to extend their own future practice by applying what they had learned in the literature during the course. Kelly wrote:

Moving forward, I’d like to incorporate more student choice into my elementary music classes. For example, when working with ukuleles, in addition to the differentiated whole-group instruction that I currently do, I’d also like to relax the reins a little more and give the students more free practice time to work on their own or in small groups on songs of their choice.

And John wrote:

Yet the challenge from Lucy Green and Sugata Mitra that children can teach themselves, accomplishing high standards of engagement and competence, has made me re-think my approach to what comprises an effective learning environment and embrace the idea of informal learning.

Transformation was not just evident in those whose blogs were coded for the most evidence in CT, however. Ivy, who hadn’t taught music since 1979, exhibited a notable shift in thinking in Assignment 3. Having stated early on that she believed a balance needed to be struck in the adoption of technology, Ivy then wrote:

As an older ‘traditional classroom’ teacher, I find it sad that the role of teacher is being replaced by a facilitator, at the same time, it is exciting because our students will have access to greater resources and expertise than the traditional classroom offers them. So in answer to the module three outcome question, ‘Is a music teacher still required?’, I sadly believe that the day will soon be here where the answer to that question is ‘no’.

Considering the progression of each participant through the course, as shown in Table 1, it is possible to argue that there is a trend towards increased CT over the course, but this must be qualified by recognising that a disposition to CT (category A) is the strongest indicator for success – consistent with the literature (Ennis, 1996; Paul, 1984).

Just as Wang & Woo (2010) found that ‘[t]he nature of topics affects students’ critical thinking’ (p. 548), Table 1 also shows that some assignments were more successful in nurturing CT than others. These were provocations 2, 3 and 5 (detailed in Table 1). Since the structure of each provocation was similar (presenting two opposing sides of an argument that were both equally well grounded in the literature), and each rubric directed the participant towards aspects of CT, it is difficult to suggest why these provocations were more successful in eliciting CT. However, during the reading of the blogs, certain topics from the video material did pop up more frequently than others, and these were collected in the researchers’ notes: it may simply be that some topics simply stimulated more thinking than others.

The interview with eminent Australian WAM educator Richard Gill in week 2, included to provide a ‘counterpoint’ to the research of Hein, Green, and the first interview with producer Francis Xavier seemed to create an identifiable ‘safe place’ for 7 of the 10 participants. Reflections included:

I agree with Richard Gill that while it’s important to meet the students where they are at first, it would be ‘stupid’ to have students engage only with pop music just as it is ‘stupid’ to have them engage only with classical music. (Kelly)

In the video titled ‘Counterpoint’, Richard Gill expresses that it would be downright stupid to only teach children popular music and not expose them to the depth and intricacies of so many other genres. (Annaliese)

While this may account for the success of provocation two, several other themes stimulated most participants in week four, where in this case prompting questions were provided. Seven engaged with a question that asked whether they had been persuaded that a DJ possessed sophisticated musical skills, with widely differing responses from ‘Absolutely not. As much as Dr. Humberstone seems to want us to believe that DJs are high-functioning musicians that have a wide breadth of knowledge in a variety of musical disciplines, the simple fact is that the software takes care of nearly all of those skills in a highly automated way’, to ‘Their level of comprehension blows me away because they’re operating on two levels. Not only do they understand the musical aspects of what makes sounds work together (pitch, harmony, metre, timbre, modulation, transposition, etc.) but they have also mastered the technological aspects of the hardware and software that allows them to apply that musical knowledge’. Another question about whether learning really has gone open (Price, 2013) prompted similarly diverse and strongly-articulated responses. These findings provide nuanced answers to RQs 1 and 3.

Quantitative peer assessment of CT

As the literature around MOOC ‘drop-outs’ suggests (Yang et al., 2013), there were fewer submissions per assignment as the course went on, even in the funnelled population. Figure 1 shows the peer-assessment score distributions of the full population, and that the drop in submissions did not affect the trend of peer-review marks, which were high. Assignments 2 and 4 attracted a high number of full-marks in peer review. Provocation two centred around the consideration of teaching the music of children’s own musical cultures versus the teaching of traditional music skills, literacy and Western art music, a key area of discussion throughout the MOOC and of this study. Provocation four was unusual in that it also included a series of short prompting-questions to help participants structure their thinking around the musical sophistication of the DJ, open learning, and ‘the BLs’ (Project Based Learning, Enquiry Based Learning and so on). This suggests that asking the most challenging questions, and/or scaffolding of student responses helped participants achieve best in peer-assessment, but it is just as likely that weakness in rubric design – less detail or fewer components – made it easier to gain full marks. For instance, in Assignment 2, only one component of the rubric (30%) identifies CT qualities, and was written as a Yes/No rubric type, rather than a scale (for example, more marks for more readings considered).

OPEN IN VIEWER

Figure 1.

Peer-assessment score distributions in the five assessment tasks.

Figure 2 shows a comparative distribution, but for the funnelled cohort. Again, the trend of marks is high, and highest in Assignments 2 and 4; but even higher on average than the full cohort, suggesting that those who ‘stuck the course’ (in the funnelled cohort) were also likely those most successful learners.

OPEN IN VIEWER

Figure 2.

The funnelled cohort’s score distributions in the five peer-assessment tasks.

Next, we were interested to investigate whether high achievement (in terms of peer-assessment marks) in the early tasks predicted success in latter tasks, as had been observed in part in the above content analysis, which speaks specifically to RQ3. As Figure 3 shows, success (qualified as a score of 100%) in the first assignment does not forecast success in successive assignments: further, the possibility of a participant who gained high marks in the first assignment gaining high scores in the second, fourth, or fifth assignments is not significantly greater than average and low achievers (p > .05). However, identifying high scores in the first assignment does have more ‘power’ in forecasting the third assignment scores (p < .0001). This may simply be because the course is well scaffolded: the first assignment is a broad comparison of a school with high music technology use and one with none at all, and therefore only scaffolds participant learning towards assignments 2 and 3. More research is required to uncover this connection. However, this idea is further supported by correlating participants’ scores in assignment 3 with scores in assignments 4 and 5 (Figure 4): Those who achieved higher scores in the third assignment did significantly better than others in the final two assessments (p = .0005 and .03, respectively).

OPEN IN VIEWER

Figure 3.

Participants’ first assignment scores mapped to subsequent assignment scores.

OPEN IN VIEWER

Figure 4.

The correlation between participants’ assignment 3 score and subsequent assessment scores.

Peer-assessment specifically on CT rubric components

In Table 1, peer-assessable rubric components that assessed CT in participants’ blogs were italicised. These data are the closest coding in the quantitative components to the content analysis coding for CT. The funnelled cohort’s peer-assessment data was next filtered to show only the marks given for the CT components, standardised as percentages to account for differences in weighting: this is shown in Figure 5. In Humberstone et al., 2020 we showed that the average CT score for each assignment was between 82% and 90%.

OPEN IN VIEWER

Figure 5.

CT component peer-review marks in the funnelled cohort.

Overall, these data show that the funnelled population achieved high marks specifically for the CT components. Naturally, this begs the question: did those whose participation in the course was highest therefore develop better CT skills, or did the disposition to CT skills lead to the participant being more likely to stay to the end? We will investigate this question in response to the overall data in the Conclusion, below.

Analysing written peer-assessment

Next, analysing learners’ written review of their peers’ work enabled us to both further understand how peers rated each others’ ability for critical thought, and to consider the more clearly definable RQ 2, about peer-assessment.

The combination of words in Figure 6 shows the tension produced by the discussion of the traditional with student engagement and the models provided by the Liveschool exemplar videos that offered an alternative curriculum (or perspective). In their comments on one another’s work, participants were obviously stimulated (interesting), although the map alone does not reveal whether view was being challenged, or changed. Such insight can, however, be gleaned by viewing quotes that most closely correlate with the map. For instance, in response to another’s writing, one participant identified the tension between their worldview and the video content, as well as their wish to maintain the status quo while introducing new ideas:

I try to blend popular music with classic vocal literature to give the kids a balance. Usually if you hook them with something they already know they like, then the more traditional pieces will touch them in some way as well. ⁴

OPEN IN VIEWER

Figure 6.

A context-network map of the topics ‘students’, ‘traditional’, ‘nice’, ‘videos’, ‘interesting’ and ‘view’.

During the content analysis of detailed blogs, it was noted that 9 of the 10 participants analysed advocated for a balance between new ideas presented and ‘traditional’ music education. The data presented in the network analysis confirms this trend. Most participants sympathised with one another about trying to find a ‘middle ground’ rather than making any dramatic shift around these topics. However, responses to more conservative approaches that rejected the provocations entirely did meet criticism within the cohort, showing that most understood the process that they were going through, and that there were not supposed to be easy or one-sided answers:

The writer interpreted the question very narrowly. . . Does it surprise you that students often see music theory as hard to learn? In fact, the writer should have tried to consider what music theory and skills do electronic and popular musicians need to know or to teach themselves. To improve, the writer could possibly take notes of the main points in the videos to stimulate thinking about supporting details.

Another observation that arises from this map is that participants concur with one another’s thinking, even when responding to a wide range of viewpoints, some of which reflect the expected worldview of the WAM-trained teacher, and others that embrace a more radical shake-up of traditional music education. It seems that around the question of the ‘classical teacher’ in the 21st century classroom (Figure 7), there is a lot of empathy in the field for all interpretations of the provocation stimuli.

OPEN IN VIEWER

Figure 7.

A context-network map of the topics ‘use’, ‘classical’ and ‘teacher’.

In total, five context-network maps were analysed, and it was summarised that based on the reviewers’ comments, learners developed their positions towards each phenomenon, and that they then tended to carry these positions and integrate them into their later peer-assessment responses. This does not necessarily mean that worldviews did not change, or that CT was not taking place, but it does suggest that they likely did not change dramatically mid-course: either a change happened early in the course and was ‘cemented’, or that participants ‘buckled down’ in their existing worldview as they progressed.

Weaknesses of the study

The content analysis analysed a lot of rich data in nearly 32,000 words and 50 blogs, but nonetheless the number of participants (n = 10) sampled to extract that data remains but a small subset of the funnelled 249 participants on which the other analyses (peer review marks and comments) were performed. The findings in the content analysis therefore may not be generalisable to the population. Further research is required to look at the full dataset mined from a greater number of blogs totalling about 850,000 words, or perhaps the whole population.

Combining the approaches

In the final phase of crystallisation, we read and suggested edits for each other’s interpretation of the data, and compared conclusions. Several themes appeared in both the content analysis and the context-network mapping of peer assessment. First, that the question of the importance of popular music in music education was still one that ideologically divided participants, despite the care taken to frame the discussion around the relevance of the musical cultures of children, into those who embraced these (not-so) new ideas, and those who defended the importance of traditional musical literacy and its ‘depth and intricacies’, or as Philpott (2010) put it, ‘‘high status musical knowledge’ (p. 86). This dichotomy was reflected in the division of how participants considered skills of non-WAM musicians, especially around the use of technology and electronic music genres. Many embraced the ideas, but only from the context of the current system:

I’m curious to hear more about how these foundational and generally universal building blocks of music - functional harmony, scales and chords etc. - are taught using Ableton Live. For teachers who have experienced traditional forms of music for most of their lives this is a massive shift in thinking!

While others, evidently persuaded to consider the alternatives to their current practice, actively self-censored, based on their perceived expectations of the community in which they practiced – what Kallio (2015) called the ‘school censorship frame’ (p. 3):

The hardest part is doing something different is questioned by parents who went through traditional bands or choirs, other music teachers who are more traditional, and administrators who have always seen it done the way it has always been done.

It is possible that the strongest connection between each participant and the success of the aims of the course (to engender CT and worldview change – all RQs) was not the provocations themselves, but the existing disposition for CT that each participant brought to the course. In the content analysis, even those successful students who were not coded for disposition (A – see Table 1) in the early weeks of the course, such as Kelly, exhibited that skill more than once later on. And being coded for CT disposition in the first week was generally (but not always) an indicator for someone who would be coded for all CT skills by the end of the course. It is also possible, as suggested by the comparison of peer-review grades in Figures 1 and 2, that those with a disposition to CT may be more likely to survive the ‘funnel’: if one had an entirely fixed worldview, one would probably be more challenged by the material in the first week, and therefore less motivated to remain.

Implications

The success of the course cannot be simply measured on the number of worldviews that were nudged in a new direction, or not. As established at the beginning of this article, worldview is established in adolescence or early childhood (Vollebergh et al., 2001), and in adulthood is more likely to be changed by the group to which they identify, than by new information, however empirical (Kahan, 2012). Nonetheless, and addressing research question 3 on the paedagogical design, we found that the MOOC itself did create a community – an online community – and within this community, through the act of public blogging and peer-review, robust conversation was had, as can be seen in the following three anonymous feedback statements (separate participants):

Your perspective about kids not always needing to learn about what they want is very interesting. I’d like to see what kind of district you work in. In my district, if kids don’t like the music program, then they won’t choose my classes . . . and then I won’t have a job!

I agree with you that in many forms of music it is important for musicians to be able to share their musical ideas and to discuss how these were formed and developed. At the same time, there are traditional forms of music that have not of themselves developed a body of theory or shared vocabulary; for example, the folk traditions of many parts of the world.

I don’t agree that the course material was arguing against the role of theory in popular modern music; rather the course is challenging us to differentiate our teaching of these important elements, which we learned through traditional education, for the 21st century learner (whose learning experience is substantially differed to ours).

In line with Wang and Woo (2010), the nature of reflective public blogging was a great stimulus for CT, because their thoughts were being shared with a broader audience. While, as expected (Giarré & Jaccheri, 2008), some participants did find this challenging, the addition of peer-review of blog posts created a community of practice (Wenger, 1998; further contextualised for this study in Humberstone et al., 2020) within that public forum (Zandi et al., 2014) which was, as has been shown, extremely stimulating for many participants, whichever side of the ideological fences they ‘sat’. There is initial evidence that because the provocations took place within this community, change was able to begin to take place (Kahan, 2012). These conclusions may just possibly reveal more approaches to the provocation of critical thought about new ideas to engender slow, but nonetheless radical reform for music education, whatever the country.

[M]y upbringing in music education led me to view teachers who used technology as those who just wanted an easy way out. Any such teacher was just having technology do their job for them. But through my later experience in the field, as along with some of the wonderful resources in this MOOC, I now know that the previous notion I had almost 10 years ago is VERY incorrect.