Teaching multimodality in the museum through embodied experiences

Integrating a museum visit into a higher education course

“[M]useums are perhaps the ultimate multimodal classroom” (Fitzgerald and Blunden, 2019: 194). Their appeal as learning sites lies in the informal yet engaging learning opportunities and resources they provide. In my teaching in higher education, I have integrated museum visits, approaching them as complex multimodal texts “which make their meaning through a combination of resources across various semiotic systems, which often includes language” (Ravelli, 2006: 151). Such an understanding of museums provided engaging learning opportunities in a module I teach, which focuses on multimodal literacy development informed by a social semiotic perspective (Halliday, 1978; Kress and Van Leeuwen, 2006). As with most of my courses, I integrate at least two exhibition visits into this module: one to a traditional literary museum, and another to a youth gallery. Based on my observations over several years of teaching in museums, leaving the classroom behind to visit exhibitions is beneficial for several reasons: apart from having the shared experience of entering these spaces, students experience multimodal theory in action and apply the freshly studied concepts in the museum through embodied experiences. From sitting behind tables where the physical body is treated as the carrier of the brain, we enter a stimulating space where the students can interact with each other and the environment not only through talk but also through gestures, touch, and movement. However, such learning is often downplayed in higher education.

This reflection illustrates the pedagogical potential of integrating museum visits into higher education courses for an embodied teaching of multimodality through the example of a visit to an exhibition in Deák17, a youth gallery in the centre of Budapest, Hungary, with students attending the ‘Content-based language development’ module. A unique aspect of this gallery is that it is set up in a vast space with a window wall overlooking the city centre near the university. The exhibition we visited was titled ‘Silent Books’ and presented a selection of over a hundred wordless picture books for all ages. It included original artwork on the walls as well as physical books placed on tables and the long windowsill. Visiting this exhibition had additional value as learning about picture books as multimodal texts was another goal of this module.

Pedagogic design for an embodied learning of multimodality

The module was designed to engage students in learning about multimodality through multimodal texts, including picture books, illustrated fiction, magazines, posters, digital narratives, and museum exhibitions. More precisely, we explored concepts, such as choice, context, semiotic modes, metafunctions, visual grammar (e.g., narrative structures, gaze and salience) (Kress and Van Leeuwen, 2006) with a special focus on image-text relations (Painter et al., 2013), and reading paths (Kress, 2003) in multimodal design and interaction. Prior to the museum visits, we worked on activities to develop the students’ viewing, image search, descriptive and analytical skills as part of their classroom work. For example, the students searched and analysed folktale illustrations for different target audiences and posters in the street based on pre-set criteria. The final assignments involved the creation of multimodal texts inspired by a museum exhibition visited during the course, such as exhibition reviews and digital multimodal compositions. In these compositions, the students could create a poster, a slideshow or an animated slideshow, accompanying it with short reflective notes.

Embodied learning of multimodality in the museum

To demonstrate possible ways of supporting embodied learning of multimodality in the museum, I describe four activities which I designed for the students visiting the picture book exhibition in the youth gallery: (i) two types of explorations (individual and in pairs/groups), (ii) responding to guiding questions, (iii) group discussions and (iv) map drawing.

The students explored the exhibition first individually, and then in pairs or groups. During the initial individual exploration, the students walked around the exhibition for 3 minutes to collect multisensory impressions. The aim of this activity was to raise awareness about how a certain space makes them feel. At this stage, students’ comments included “laboratory condition”, “peacefulness”, “being in a library”, “brightness”, “plain space”, “too much light”, and “feeling exposed”. Their experiences focused on the light, temperature, colours, noises, sounds, and spatial organisation. Next, I asked the students to walk in pairs or groups to explore the exhibition. Some of them decided to walk alone. The students also read some of the books individually or in pairs for a short time. I encouraged them to talk, take photos and notes, and notice their pathway choices and the signs, texts or space dividers that directed them. The students appreciated walking and talking with their peers as they commented that it encouraged them to notice more meanings by observing each other’s actions. I also reminded the students to explore whether the museum encouraged speech, touch and movement in the exhibition space. By doing so, I aimed to draw their attention to museums and galleries as ‘socially produced spaces’ (Christidou, 2020: 8) in which embodied interaction with each other and the exhibited artworks gains special significance.

Informed by research in spatial semiotics (Ravelli et al., 2008), touch (Jewitt, 2017), and language in the museum (Blunden, 2020; Ravelli, 2006), the students received guiding questions focusing on topics, such as first impressions (e.g. “How do you feel in this place?”), the exhibition space (e.g. “Is there a pathway you follow?”), visitor interaction/engagement (e.g. “Is touch encouraged?”), language use (e.g. “Find examples of interesting uses of language in the exhibition. Take notes or photos.”), and multimodal resources (e.g. “How are sounds and music used in the exhibition?”). By directing the students’ attention to meanings construed by space, room layout, light, sound, colours, labels, and touch, these guiding questions targeted the students’ meta-awareness of the gallery space as a complex multimodal environment.

During the mediated group discussion after the visit in the gallery, I asked questions, such as “What has stayed with you?”, “What is the intention of the exhibition?”, “How are meanings organised visually, spatially and verbally?”, and “Were you encouraged to interact and touch the books?”. As we sat down to talk, the students asked well-informed questions and voiced their opinions about the books and the exhibition confidently. I noticed that they integrated multimodal concepts, such as modes, ensembles, metafunctions (focusing on compositional meaning) and gaze into their discussions of both exhibition and picture book design. They also reflected on being confused by some aspects of the exhibition, such as the placement of some of the labels, the organisation of displayed original book illustrations on the walls, and the presence of some QR code stickers at random places redirecting the visitors to music playlists. However, they looked for the reasons for this confusion with the museum interpretation. For example, inspired by their learning about reading paths in multimodal texts (Kress, 2003), a vivid discussion unfolded about the positioning of the labels. Moreover, when exhibits were placed below eye level, the students wondered whether it was a conscious decision to engage younger and shorter visitors. They also acknowledged that they were allowed to talk freely during the visit. During the discussion, I noticed that the multimodal concepts studied in the classroom helped the students analyse and critically engage with the potentials and challenges of multimodal design and its impact on their embodied experiences. Such use of multimodal concepts contributed to the students’ awareness of their own embodied learning.

Informed by research which uses map-making as a meaning-making activity in the museum (e.g. Christidou, 2020; Diamantopoulou et al., 2012; Insulander and Selander, 2009), I asked the students to design a map of the exhibition to help them further engage with their embodied experiences in the space of the gallery. The students were asked to make a map which could also integrate their own experiences and memories triggered by the exhibition. This was used during our follow-up discussions. The students designed top-down views of the gallery, often adding personal comments as shown in Figure 1. This practice aimed to build awareness of both the spatial and the interpersonal meanings construed in museums. In other words, this map-making activity served the purpose of teaching multimodality by showing how the students’ selection of significant elements from the exhibition reflected their interpretation of its meaning-making resources. This way, the students’ representations of the exhibition space revealed how interpersonal meanings were connected to the spatial organisation of the exhibition. In short, both the follow-up discussion and the map-making activities invited the students to reflect on their experiences both verbally and visually with the help of the metalanguage of multimodality.OPEN IN VIEWER

The potentials of the embodied teaching of multimodality

These experiences have motivated me to examine what happens at the intersection of teaching multimodality and supporting embodied learning in the context of the museum. Based on my experiences, the students’ learning of multimodality is strengthened by engaging with their embodied experiences through activities which explicitly integrate multimodal concepts. By focusing on embodied aspects, students become aware of how they involve their own bodily presence and sensations in interactions in a socially produced space (Christidou, 2020) in which other people and artworks equally impact their engagement and learning (vom Lehn, 2006). Moreover, such pedagogic design positively influences the students’ attitudes towards museums. The students’ confident and critical reflections after the visit were in sharp contrast to their experiences of feeling “overwhelmed”, “intimidated’’, or “prohibited to talk or touch anything” as they reported prior to the visit. Engaging with the exhibition in an embodied way and using the metalanguage of multimodality to reflect on it allowed their own experiences to become both validated and meaningful. More importantly, it enabled the students to engage critically with the theoretical underpinning of this module.

The embodied approach to multimodality in the museum served as a powerful prompt for understanding meaning-making and communication through multimodal texts. The pedagogic design I created acknowledged the impact of different semiotic resources, such as objects, images and written texts on the visitors’ engagement (Insulander and Selander, 2009). It also highlighted the multimodal dimensions of museum spectatorship and raised awareness of how visitors ‘rely on others’ verbal and embodied conduct’ during visits (Christidou and Diamantopoulou, 2017: 27).

In my view, a pedagogic design which has a multimodal theoretical underpinning and uses the museum for an embodied learning of multimodality is apt for courses in higher education, such as the literacy education course I taught. The multimodal and multisensory features of the museum texts, the spatial design, the organisation of the exhibits, and the bodily presence of other people were the prompts for the students to use the multimodal social semiotic conceptual toolkit that the course provided. These considerations indicate that literacy teaching can benefit from integrating such museum-based embodied experiences into the learning design of higher education courses. As this reflection has demonstrated, students’ learning potential can be extended through an explicit focus on how they make and negotiate meanings and learn through embodied multimodal experiences in museums.