Book review: Charles Forceville,Visual and Multimodal Communication: Applying the Relevance Principle

Theoretical coherence has been elusive in multimodal studies. Charles Forceville’s monograph Visual and Multimodal Communication: Applying the Relevance Principle is one in this attempt. Forceville redefines the notion of code and extends the application of Sperber and Wilson’s (1995 [1986]) relevance theory (RT) beyond the classic face-to-face verbal communication between an intimate couple, to codes of various kinds in meditated mass-communication.

The monograph comprises 11 chapters, each featuring an introduction and a summary, meticulously presenting the author’s view in a cohesive and concise manner. Chapter 1 introduces the topic with a systematic review of previous studies on cognition and communication. It provides evidence to illustrate the cognitive and communicative dimensions of films and arts respectively. Moreover, Forceville exemplifies that the evolutionary goals to survive and procreate contribute to human beings’ cooperative behavior of communicating. Thus, this chapter sets the scene for RT from an evolutionary perspective.

Chapters 2–4 explicate the theoretical underpinning of this monograph. Chapter 2 presents a condensed version of RT, explaining how individuals process information to maximize cognitive effects with minimal mental effort. Specifically, distinctions are made between several pairs of terms, including the cognitive and communicative principles of relevance; communicative and informative intentions; effect (benefit) and effort (cost); encoding and decoding; explicatures and implicatures; descriptive and interpretative uses of utterances. According to RT, every act of ostensive-inferential communication comes with the presumption of its optimal relevance (e.g. a fast car approaching as a potential danger signal to a pedestrian). Such communication differs to some extent from ‘symptomatic’ communication whose information is communicated without its sender’s conscious awareness (e.g. a driver’s fast driving due to his or her mood). However, in extreme cases, whether certain information is regarded as ‘symptomatic’ communication or ostensive-inferential communication depends a lot on the knowledge of the individual who interprets the information. Therefore, the central tenet of RT is that relevance is always to an individual. For instance, a private code is an explicit communicative act for insiders but meaningless for outsiders.

In Chapter 3, Forceville moves on to adapt RT to accommodate visual communication. Visuals are used here in the broad sense to refer to any nonverbal information in static images such as pictures, layout features, colors, typography, and motion/emotion lines in comics. Visuals, like words, come with the promise that they are relevant to the audience. In this regard, the audience must get sufficient cognitive effects from the visuals to balance the amount of mental efforts invested. In addition, visuals selectively code realities in their representation to achieve optimal relevance. Therefore, the audience needs to enrich and assign references to the visuals in order to derive implicatures (i.e. implicit meanings), as well as visual explicatures (i.e. truth-evaluable propositions) in a certain context.

Chapter 4 proceeds to apply the expanded version of RT proposed in this book to mediated mass-communication, which may involve a synthesis of verbal and visual communication. Inspired by Yus’ (2011) study of applying RT to electronically mediated communication, Forceville gives examples of communication involving three or more participants, and concludes that mass-communication differs from the classic Mary-and-Peter example in that it is ‘either addressed to more than a single person, or is mediated, or both’ (p. 115). In this case, relevance remains relevance to an individual, as demonstrated by the various interpretations of a live-streamed presidential speech by a large audience. Moreover, technical, institutional, financial and ideological factors all have an influence on how relevance is achieved. For instance, documentaries on Dutch public service television are adapted to fit a sixty-minute time slot, potentially altering their content and length, thus creating distinct versions for different audiences.

The notion of genre is introduced in Chapter 5, which serves as the basis for case studies in subsequent chapters, together with the theoretical underpinning from preceding chapters. Given the complexity of mass-communication, genre serves as ‘the single most important pragmatic factor that helps steer and constrain the derivation of relevance in mass-communicative visuals and multimodal discourses’ (p. 248). The communicator and the addressee in mass-communication have genre knowledge as a fundamental element in their cognitive environment, and they may share a perception of the prototypical features of a specific genre. Attribution of the same visual to different genres will arouse distinct presumption of optimal relevance.

Chapters 6–9 center on case studies applying RT to various genres, such as logos, advertising, and comics, with each chapter featuring progressively complex modalities. The recurring question is how visual and multimodal discourses can be ‘presumably interpreted in more or less the same way by the envisaged audience’ (p. 131). Chapter 6 focuses on pictograms, traffic signs, and logos, which are labeled as coded visuals by Forceville. Contrary to Sperber and Wilson’s (1995 [1986]) treatment of all non-verbal communication as implicit, Forceville proposes coded visuals as closest to linguistic communication in that their meanings can be decoded from the signs per se. These coded visuals and their locations create ostensive stimuli to certain audiences, giving rise to explicatures and strong implicatures.

Chapter 7 discusses case studies that apply RT to advertising, which typically consists of a combination of visuals and minimal written text. Compared with coded visuals, advertisements vary according to both the size, place, color and orientation of elements in them, and these elements’ internal structure. Therefore, the interpretation of advertisements depends on inferences made from these dimensions. By offering case studies of such advertisements as those of Samsonite, KitKat, and Coca-Cola, Forceville explicates how informative intention is recognized and subdivides this process into two tasks. One is reference assignment on the basis of the brand logo; the other is inference making from other visual and written materials in the advertisement.

Chapters 8 and 9 are devoted to cartoons and comics respectively. The former are ‘stand-alone visuals/word-and-image texts’, whereas the latter are ‘sequences of visuals/word-and-image texts’ (p. 184). Through ample case studies, Forceville proposes that relevance in both cases is achieved by a combination of decoding and inferencing, thus making the relevance-derivation processes more complicated than those in the previous two chapters. Political and non-political cartoons are characterized by caricature and stereotypicality. These cartoons reveal the style of the artist and are drawn for specific papers and magazines. To stimulate optimal relevance, cartoonists prefer the loose use of visuals, which serves as a ‘shorthand’ way of communicating information, to secure the same cognitive effect with less mental effort: readers are prepared to invest extra mental effort to earn extra cognitive reward. In comparison, relevance in comics is achieved not only by decoding isolated panels, but also by assigning references, enriching content, and generating implicatures from panel sequences. Due to comics’ sequential nature and the addressees’ desire for narrative coherence, explicatures and implicatures will be adjusted from one panel to another.

In the various case studies, RT is employed to model communication by elucidating how a given discourse achieves relevance. Meanwhile, iconography, semiotics, narratology, stylistics, metaphor theory and blending theory serve as tools or methodologies which make possible the interpretations of discourses. Such notions as ‘gestalts’ and ‘comprehension’ from cognitive linguistic studies are also mentioned in the derivation of relevance. The envisaged audiences share much of the knowledge and many of the cognitive mechanisms mentioned above. Therefore, they interpret the visual and multimodal discourses in almost the same way.

Chapter 10 goes further to investigate ‘controversial communication’, in which the communicator misrepresents an event and/or the preferred attitude to that event. Contrary to the loose use of visuals, parts of visuals in controversial communication are deleted so as to turn an otherwise negligible symptom into an ostensive stimulus. Controversial or subversive use of visuals is presented in the form of communicative layering, which shares similarities with RT’s metarepresenting. For instance, the color of O.J. Simpson’s (a famous black football player accused of having killed his ex-wife) face is darkened on the cover of Time magazine to portray him more like a criminal. Thus, the communicator visually or multimodally ‘comments’ on what is being presented. The audience, in response, must cautiously derive relevance from these controversial visual and multimodal stimuli. Chapter 11 rounds off the monograph with some concluding remarks. Forceville revisits his claims grounded in RT and pinpoints areas within RT that require further refinement.

Overall, the monograph is an impressive undertaking which innovatively extends the classic RT to accommodate visual and multimodal communication, thus providing a systematic approach to all forms of communication. Albeit varying from each other in how relevance is achieved, communication of distinct modes, media, genres or cultures functions in similar ways that can be accounted for by RT. This renders the book a significant contribution to the growing body of visual and multimodal studies. In addition, the combination of RT and other culture-oriented angles in the book highlights the importance of evolution (i.e. human nature of seeking cognitive relevance) and culture (i.e. societal nurture for achieving communicative relevance) in communication. This synthesis aligns with Wilson’s (2010 [1975]) concept of ‘gene-culture coevolution’, thereby promoting the convergence of multidisciplinary research. However, due to the focus on the multimodality of discourses and the interpretation of such discourses, not enough attention is paid to the communicator. Some aspects of Forceville’s exploratory analysis remain to be extended to other multimodal discourses. Despite these, the monograph is theoretically insightful and practically applicable. With extensive examples, it will not only be of interest to relevance theorists, but also be highly accessible to scholars in visual and multimodal communication.