Cognitive Processing in Digital Audio Workstation Composing

Introduction

In an increasingly digitized society where the capabilities and accessibility of technology continually expand, so too does the focus on students’ digital learning. During the COVID-19 pandemic, teachers discovered a need for new resources. Digital audio workstations (DAWs) provided a solution and can continue to serve future pedagogical needs. DAWs offer the opportunity to engage students in more complex composing forms regardless of their capabilities to play physical instruments. In turn, this offers music teachers a broader range of instructional and pedagogical strategies to employ in classrooms. Many resources discuss the connections between cognition and DAWs. However, this literature, particularly resources that emphasize cognitive science, tends to be inaccessible to the average reader. The scientific terminology used in cognitive science can be difficult to comprehend without extensive examples. Detailing music-specific explanations of cognitive concepts may help educators bridge the gap between academic abstractions and practical classroom applications.

When teaching digital composition in middle and high schools, I observed my students gain more confidence and skill composing with musical elements. They understood how to use genre, structure, melody, harmony or accompaniment, texture, and instrumentation to create original music, yet many could not transfer this knowledge to DAWs such as GarageBand or Soundtrap. This article offers a closer examination of students’ cognitive processes to shed light on this disconnect by explaining how these processes might manifest in students’ learning, engagement, and work produced from two such programs: GarageBand and Soundtrap. The intent is to provide music educators with a practical and accessible resource to help guide electronic composing curriculum.

Cognitive Processing in Music Composition

Cognition is “the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses” (Oxford University Press, 2020). It is a way of processing knowledge where the known and unknown interact to produce new knowledge that may be applied to different contexts or processes. Each individual uniquely conceptualizes music based on their experiences, though groups may share common musical experiences. For instance, lyrics and melody are two of the most memorable aspects of songs, and when recalled from memory, the song will often be associated with the context and perceptions surrounding how it was first experienced (Sternberg & Sternberg, 2012b).

According to Sternberg (2003), as people grow older, their thoughts and behaviors interact in more sophisticated fashions. Information processing is more thorough, increasing the likelihood of accuracy in solving problems. Adults can comprehend complex relationships more successfully and develop flexibility in how they use strategies or information. When listeners have more experiences to draw from, their perception of a song’s meaning may change. The same can be said for knowledge students gain in a music classroom. Increased knowledge of a song’s musical construction or lyrics may change students’ perception of the song. This opens a gateway for students to transfer their knowledge and perceptions to composing and performing their own songs.

The act of composing and performing music involves embodied cognition where “cognitive processes are deeply rooted in the body’s interactions with the world” (Wilson, 2002, p. 625). “When we hear a sound, our brain searches for familiar patterns to use in ascribing mean-ing to that sound” (Salter, 2019, p. 769). Converting sound into language and form is what produces music. Experiences of sound often change, and that effects how the mind cognitively represents that sound quality.

This composing process requires metacognitive strategies “concerned with planning, monitoring, and evaluation of the results” (Bogunović, 2019, p. 97). Emmerson (1989) outlines this process as a simple model of composition involving (a) the action of creating and combining sounds, (b) testing combinations by listening and determining if they sound right together, and (c) accepting, rejecting, or modifying. It is important to remember that the last step involves evaluation. Evaluation is a cognitive process that is shaped by each individual’s experiences with music and is part of what makes composing artistically subjective.

Following this review of literature, there were six areas of cognitive science that seemed most relevant to students’ digital composition experiences: cognitive tools, audio-visual processing, decision making, hypermedia, and social cognition. These areas are outlined briefly in Figure 1, in addition to a seventh consideration, teacher expertise and guidance. This article identifies common struggles associated with each area, and best ways to support student learning in Soundtrap and GarageBand.

OPEN IN VIEWER

Figure 1.

Cognitive processes underlying student interactions with GarageBand and Soundtrap (DAWs).

DAWs as Cognitive Tools

Cognitive tools are software or technology tools designed to enhance thinking, problem solving, and learning of human beings (Jonassen & Reeves, 1996). GarageBand and Soundtrap are two examples of DAWs, electronic devices or software applications that are used for recording, editing, mixing, and producing audio files. Making music using a DAW involves human interaction with sound, the interface (visual aspect of the program enabling the user to communicate with the program), and any collaborators. DAWs also allow users to record or create sounds that are then sequenced into structured arrangements that formulate complete songs. When using GarageBand and Soundtrap, students are actively engaged in creating new representations of sound reflecting their individual “comprehension and conceptualization of information and ideas rather than absorbing predetermined presentations of knowledge” (Jonassen & Reeves, 1996, p. 697). DAWs are used as “design spaces that engage and manipulate” (Gee, 2003), allowing students to analyze structural relationships of musical content.

Both GarageBand and Soundtrap are similar in their design and basic function. They enable students to drag, drop, and organize 10- to 30-second segments of recorded audio edited to repeat seamlessly when played end to end, known as looping. Students are able to choose from hundreds of options, or they may choose to record their own tracks using instruments, voice, or on-screen instruments played by the computer keyboard. Figure 2 shows the visual interface of both programs; that is, the graphics on the screen representing the audio. Each track (horizontal row) represents one layer of audio and the colored strips are loops. Students can also add tracks to layer sounds on top of one another.

OPEN IN VIEWER

Figure 2.

GarageBand (left) and Soundtrap (right) interfaces.

Brooker and Sharrock (2016) offer a more in-depth description:

Having input musical phrases and instruments, users can edit them, cut them up, move them around, change the notes played, change the instrument sound, layer them over other instruments, and so on. Playback operates through the tracking bar, which moves from left to right playing recorded and programmed phrases as they occur, coordinating the produced sounds with the visual aspects of the sequencer. (p. 465)

These features (tracks, loops, editing, recording, instrument changes, sound layering, playback) can seem overwhelming to first-time users but become straightforward with minimal experience. The cognitive process begins with acquiring knowledge, which, in the case of GarageBand and Soundtrap, involves identifying the features and determining their functions—in other words, learning what to do before determining how to do it using these features. The two aspects of what and how represent the new knowledge. To complete the cognitive process, the user must connect the new knowledge with their existing musical understandings to apply them. The user’s musical background shapes why the DAW is being used and for what purpose. In my experience, if I can help students isolate the what, how, and why of their composing process, the quality of their work greatly improves.

DAWs are more effective cognitive tools if students have prior experience with both the program and the music content. This enables them to focus on critical thinking. In my classroom I created short, exploratory activities for students to actively learn the DAW’s features. I found that basing the activities on musical elements helped build connections between existing music knowledge and the technical how-to of the DAW. Consider a focus on harmonic accompaniment (accompaniment is sometimes easier for younger students to understand), as many loops are based on chord progressions that students can quickly determine whether they work together or not. Students can easily designate instrument roles such as bass lines and chords. Students may also add drum-beat loops or create their own; one of the first questions my students always ask me is “Can we use drums?”

When students have selected and organized their loops, challenge them to explain their choices. Often my own students struggle to articulate the why. It is not because they don’t understand the characteristics of a musical genre, chord structure, or the roles of instruments but that their focus is on understanding how to use the program, and not the why. When this happens, I have found that asking questions about the genre and instrumentation help them to realize their selections have very little focus. Explaining how to use the search functions to find similar loops, and giving them time to edit their work, results in immediate improvement.

Audio-Visual Processing

The human brain processes visual and auditory stimuli separately before it connects to existing information held in long-term memory (Mayer, 2001). Students first need to connect what they hear with the visual representation shown on the screen (Figure 2) to use GarageBand and Soundtrap to maximum effect. On top of this, students need to transfer and apply their prior knowledge of musical elements to create a cohesive musical work. This is a large amount of information for nonadult brains to process, and students are processing repeatedly every time they make changes to their work.

Human beings have a limited capacity of working memory to process information received from visual and auditory channels (Sternberg & Sternberg, 2012b). When using DAWs, students are listening to audio, while watching a screen of multicolored shapes that visually represent what they hear. Common problems begin to emerge when students confuse what looks good, and what sounds good. When creating melodies, many students use their available working memory on the visual and tend to produce either (a) a single layer work with one melody heard at a time, moving horizontally across the screen or (b) a multilayer work with multiple melodies heard at the same time moving vertically down the screen. The screen may look visually appealing, but the audio either lacks musical complexity or is overly complex and does not sound appealing to the ear. On the flip side, students who use their working memory on the audio find the software interface too complicated and struggle to achieve the desired product without extensive knowledge of advanced tools. According to Mayer (2001), this overload on working memory is often exacerbated when interacting with unfamiliar information. This is something to keep in mind when designing curriculum.

In practice, the above problems might be avoided by offering students a scaffolded introduction to Garageband or Soundtrap focusing on one element at a time: visual or auditory. For an audio-processing activity, Student 1 faces away from the computer, while Student 2 operates the DAW and plays various melodies. Student 1 should select their preferred melodies by listening only and then Student 2 drags them into place on the screen. For visual processing, reverse the activity by muting the computer audio, and both students select their favorites based on the melody’s title or its graphic representation on the screen. The objective is to have students focus on either audio or visual to reduce their cognitive load. This frees space in working memory to make connections between interface content and existing understanding of music concepts. When my students listen to their work without the DAW visual, they are often surprised by the obvious issues in their work. I have found that gathering students together for an informal share of work in progress has been illuminating for them and results in improved final products.

Decision-Making

Decision-making occurs regularly when composing and is a process that involves reasoning and judgment to select among choices or to evaluate opportunities. “The goal of reasoning is to draw conclusions, either deductively from principles or inductively from evidence” (Sternberg & Sternberg, 2012a, p. 489). At times, decisions must be made within time frames that may not allow for examination of all information, and the brain must reduce information to a manageable amount. Heuristics “allow decision makers to process information in a less effortful manner” by using “principles of effort-reduction and simplification,” (Shah & Oppenheimer, 2008, p. 207). However, the “mental shortcuts of heuristics and biases lighten the cognitive load of making decisions, but they also allow for a much greater chance of error” (Sternberg & Sternberg 2012a, p. 490).

There are also biases (Sternberg & Sternberg, 2012a) that underlie such decisions that I have observed in my own students. First, overconfidence can lead students to overestimate their skills, knowledge, and judgment. When feeling overconfident, students may produce work that is incomplete or that deviates from the requirements. Individuals may also show a reluctance to apply feedback, refine, or revise their work. Second, illusory correlation can manifest as students layering sound loops, instruments, rhythmic, melodic or harmonic material, which they think go together when they do not. Third, with hindsight, students can think back to previous experiences, their own or those of their peers, and make decisions to either ensure or avoid similar outcomes.

Decision-making is where some of the advantages of GarageBand and Soundtrap become drawbacks; there is too much choice. With large preloaded loop libraries (and infinitely more available to download), even students who want to make informed decisions quickly realize there is no time to examine all the options. It is easy for students to get distracted by too many options and take the quickest route by selecting the first option that does not sound bad. Instead, they should carefully consider other options and make an informed choice about what sounds best. Teachers might try building on the audio-visual processing activity mentioned above by limiting choices to a particular musical genre or instrument group when creating a melody and harmonic accompaniment. This teaches students to apply specific criteria to determine what is suitable before examining the remaining options. Fewer options allow students to explore more musical possibilities without being encumbered by all the options and it becomes less effort to apply critical thinking before making decisions.

Social Cognition and Groupthink

Music performances and compositions exist within social constructs, as both are reliant on human interaction. How humans learn from observing and interacting with one another is an area of cognitive science known as social cognition (Frith & Frith, 2012). The rise of technology led to the creation of DAWs, which has dramatically changed music production but still requires social and embodied cognitive processes (McPherson & Tahiroglu, 2020; Wessel & Wright, 2002).

However, there can be disadvantages to working in groups. One such phenomenon is called Groupthink, which refers to decision-making that is premature and suboptimal, generally the result of group members attempting to avoid conflict (Sternberg & Sternberg, 2012a). Close-mindedness, rationalization, squelching dissent to feel invulnerable, and choosing unanimously in all decisions are all characteristics of Groupthink (Sternberg & Sternberg, 2012a). Students often encounter the above issues in group work, and when working on the same screen, they may find these problems amplified. Collaborating is a skill in itself and looks different when using devices instead of instruments, then different again when learning in person versus online. When in classrooms, students are able to physically interact with anyone in the room without prior planning or reliance on a platform for communication. Collaboration with peers helps generate ideas and offers validation for decisions. However, Groupthink can result in hasty decisions to avoid conflict.

In practice, informal sharing of work and discussions during music class can help build collaboration skills. Students may walk around the room and listen to what others have created or share their screens and audio in Zoom breakout rooms. Other options might include teacher-moderated websites or blogs with links to student work and space for comments. Encourage consideration of alternative ideas and risk taking. GarageBand and Soundtrap also offer a variety of ways for students to collaborate by connecting additional instruments through cables such as keyboards, microphones, or semiacoustic/electric guitars. Using individual instruments can provide students with opportunities to develop musical material to contribute to a single, recorded project. Alternatively, other devices can be used as controllers, such as Logic Remote, which enables handheld devices to act as a MIDI (Musical Instrument Digital Interface) instrument. The interface has similar capabilities to the app version of GarageBand and enables users to use a keyboard and chord strips, which can be recorded directly into any project via a Bluetooth connection (see Figure 3).

OPEN IN VIEWER

Figure 3.

GarageBand external input options; guitar, microphone (left) and Logic Remote on iPad (right).

Soundtrap is a web-based program with the same capabilities to input MIDI instruments through its handheld device application for Apple and Android devices. Both the web-browser and application versions also offer an inbuilt collaboration function where students can share a single project and work on it concurrently, similar to Google Docs. There is a live chat function on the right that offers text and video options to enable collaborators to work in real-time. Students are able to tag-team their work, keep track of past versions, and leave messages for their peers as the chat function keeps logs. Students and teachers can message back and forth in real time, enabling feedback to be instantaneous without anyone needing to leave their seat, call out, or even be physically present in the same room. Additionally, if the teacher enrolls students through the Soundtrap for Education portal, they can assign students to classes and opt to receive notifications anytime a student makes edits. This is helpful to be a fly on the wall and keep track of student contributions (see Figure 4).

OPEN IN VIEWER

Figure 4.

Soundtrap collaboration features; inviting and sharing (left) and text or video chat (right).

Hypermedia

DAWs are examples of hypermedia where media such as audio, graphics, video, and text appear on screen but without structure and are left to the human being to organize (Nelson, 1980). The need for user-imposed organization is why decision-making is so critical for successful use of DAWs. In the case of GarageBand or Soundtrap, the interface (graphic, video, text) is designed for the user to access the content (audio) but they must organize it themselves by either dragging and dropping loops or recording their own audio. No two users are likely to organize the content in the same way. As a result, it is the user-imposed organization which is meaningful and not the software (Jonassen & Reeves, 1996).

What makes both GarageBand and Soundtrap so accessible to students is that the designs are similar to that of video games. Studies have shown that people find that “visualization” and being able to “see it firsthand” are exciting and fun and help to “see different connecting pieces,” which serves to elucidate concepts and “fortify mental models” (Tawfik et al., 2012). Students have “mental models” and solid understandings of music content; however, the problem seems to lie in a disconnect between the music-based concepts and the visualizations on-screen.

DAWs may be designed for users to organize sounds (graphic/text representations) in ways that are meaningful to them, but younger students can struggle without existing organization or guidelines. Graphic representations of music are not exclusive to computer-based compositions, and there are many examples across a variety of genres. John Cage wrote many pieces that were scored in pictures, words, and lines, several of which are available as YouTube videos. Graphic notation is often used for body percussion, which would be accessible for students to perform themselves. Then there is the reverse approach: existing music that has inspired visual organizing of sound, such as the Line Rider series on YouTube. In these videos, melody and harmony are represented by separate vertical lines that vary in shape and length. When there is one melody, only one line appears, when harmony enters, then two or more lines are shown, and so on. Watching multilayered visuals while listening to the music offers a glimpse into how another human being perceived and organized music using the same vertically aligned system as in DAWs.

In practice, a more interactive approach might incorporate performance skills where students interpret unknown graphic representations of music with their bodies or on instruments. You may even try having them create their own graphic scores. For a more concrete connection to GarageBand and Soundtrap, students can record themselves playing or singing at different volumes and rhythms. Open the recorded track and discuss how the image of the soundwaves change. Waves jump higher to show louder volumes and get vertically smaller when the audio is quieter. Lines get closer together for shorter rhythm durations and farther apart for longer rhythms or rests. Armed with this knowledge, how might they choose to organize their recordings so they go from loudest to softest, or longer rhythms to shorter.

Teacher Guidance

A common thread ties together each area of cognitive science explored in this article: individual experience. For students to create effective digital compositions, they must draw on their past experiences listening to music, understanding of musical elements, and ability to navigate the DAW interface and features. At this point, teacher guidance and ideas (Figure 5) come into play as adults are more adept at cognitive processing and have a wider range of experiences to draw on. It can be challenging, however, for teachers to offer meaningful assistance to all students within scheduled classes due to time constraints.

OPEN IN VIEWER

Figure 5.

Curriculum ideas to help support students working with GarageBand and Soundtrap (DAWs).

In practice, developing how-to resources for students to use independently enables teacher focus to remain on assisting students with process and content elements. Example resources might include Google Docs with listed instructions and animated graphics interchange formats (GIFs) or short, single concept video tutorials recorded with QuickTime or Screencastify. Consider using Google Classroom, Schoology, or building a website using Google Sites to organize all your materials and resources. Doing so ensures students have the benefit of teacher expertise accessible at any time. Having these resources readily available for students affords the teacher more time to interact with students to offer specific, creative feedback focused on their individual work.

Teachers might try referencing things students can either see (visual) or hear (audio) to help them understand. Pointing out the visual representations of sound and how the rhythms line up with one another is one such way. Focusing on visual representations also builds on the graphic notation activities in the hypermedia section, except this time, they are using them as evaluation tools. Another way I increased my productivity was to create a Google Form for help requests for each class where they type in their name and submit. Viewing the responses generates a list that ensures you see students in order, avoids fighting over inaccurate lists, and helps you track whose work you have seen. If you are really pushed for time, stay at your desk and have students come to you, as this can save much-needed time.

Conclusion

There are many cognitive processes that happen in students’ minds when they are working with DAWs such as GarageBand and Soundtrap. In considering DAWs as cognitive tools the challenges involved in audio and visual processing become clear, which explains why students struggle to connect their knowledge of musical elements to digital compositions. Their decision-making processes are affected by heuristics and biases that compound the problem, and there are additional considerations that arise when factoring in group work. Students are more successful when focused on one aspect of DAWs at a time to reduce their cognitive load. Introductory activities that separate what to do, how to do it, and the intent behind musical choices can help maximize the effectiveness of the DAW. Minimizing options and focusing on either audio or visual aspects of the DAW can help students produce higher quality work. By unearthing hidden processes and examining them in relation to technology-based composition tools, we can better understand and support student learning.