Engine for the imagination? Visual generative media and the issue of representation

Machine vision and memory mediatization

Proliferation of digital technologies has impacted, among many social and cultural practices, the ways in which individuals and communities remember and forget, providing new opportunities to formulate, reinforce, or challenge interpretations of the past (González-Aguilar and Makhortykh, 2022). Mediatized memory is the one which “flows across territorial and social boundaries” (O’Connor, 2022: 635) interwoven with arrangements of cultures, agents, and materialities of production technologies (Lohmeier and Pentzold, 2014). When applied to the context of conflicts and wars, collective memory has traditionally been impacted by conventional media, particular by news photography (Griffin, 2004; Parry, 2010). Human-produced war photography is appreciated as a continuing practice of cultural production within specific historical, cultural, and political circumstances (Parry, 2011), and, as Griffin (2010) argues, a full consideration of any image of war must include an analysis of these conditions. John Berger might also say that war imagery is implicated by how the subject has been seen by other people and that “the specific vision of the image-maker was also recognized as part of the record” (Berger, 2008: 10).

But what is seen in AI-generated war images, and what remains unseen? Generative models bring about changes in agentive nature of technology (Natale and Guzman, 2022) in that AI is increasingly seen as an entity operating in ways too complex for human comprehension, with limited opportunities for human actors to tap into system’s infrastructure. Due to algorithmic opacity of visual generative media, there exists a perpetual tension between the perceptual bias arising from inductive biases in a machine vision system that determine its capability to represent the world (Offert and Bell, 2020: 1333) and the assumed “ways of seeing” – a diverse range of perspectives and realities that the system lacks the capacity to represent. As a result, AI-generated images can reinforce dominant hegemonic narratives in representing collective memory (Makhortykh et al., 2023). Hegemonic AI can act as a “bio-necro-technopolitical machine that serves to maintain the capitalist, colonialist and patriarchal order of the world” (Ricaurte, 2022: 727).

Thus, machine vision differs from human vision in its capacity to mediate collective memory. Cultural meanings in AI images are established through a representation by a technical system which carries someone else’s values and ideologies. Such universality impacts diversity of visual representation, and as Crawford (2021: 98) points out, shapes the epistemic boundaries governing how AI operates and, in that sense, creates the limits of how it can “see” the world. As visual generative media begin percolating through into social structures, it is timely to consider a visual culture in which algorithms become integrated into human processes of memory mediatization. By examining the implications of AI images for collective memory of Russia’s war against Ukraine, this study analyzes representational capacities and visual themes of AI-generated war imagery.

Considering the text-to-image generator Midjourney (https://www.midjourney.com/) as a “proxy for a broader collective consciousness” (Thomas and Thomson, 2023), I examine what is seen and what remains unseen in AI images that represent the Russia-Ukraine war in response to verbal prompts guided by the following research questions: How does Midjourney envisage Russia’s invasion of Ukraine in February 2022? How diverse are the results to RQ1 related to the past event of Russia’s invasion of Ukraine compared to Ukraine’s counteroffensive in July 2023? How does Midjourney imagine Ukraine post-war? I introduce the concept of AI image generation as a socio-technical practice, discussed next, to address these questions.

AI image generation as a socio-technical practice

Central to the approach taken in this paper are the concepts of mythology, technology, and representation. Drawing on Boyd and Crawford (2012: 663), I define AI image generation as a socio-technical practice based on the oscillation of:

Mythology surrounding disembodied machine learning reflects a pervasive assumption that AI yields authenticity and epistemic objectivity. In reality, cultural meanings in AI-generated images are established in a regimented way through operationalized technical systems. These systems as well as their representational capabilities are driven by the interests of commercial actors rather than ordinary people using generative media. Technology involved in the process of co-production is a subclass of algorithms known as deep neural networks which operate in covert ways (Castelvecchi, 2016; Von Eschenbach, 2021). Representation, viewed from the perspective of visual social semiotics, is a discursive process in which a social actor seeks to make a visual representation of a social/political event and uses visual generative media to “translate” a verbal prompt into a visual output. The outcome is AI-generated images that represent the intended events or happenings from a machine vision perspective – the viewpoint that frequently diverges from that of a human actor.

Considerations of mythology, technology, and representation inform the analytical focus of this paper in two ways. Firstly, the technological aspects of visual generative media and the speed at which it develops present us with methodological complexities when studying these socio-technical phenomena. To account for these complexities, I propose to integrate insights from media and cultural studies, computer science, and visual social semiotics. Secondly, there is a dissonance between what is known about the behavior of the system used to generate images and what these images represent. The emergence of visual generative media challenges our understanding of representation – the first key moment in the circuit of culture (cf. Hall, 2013) – reconfiguring the ways in which we make sense of the world outside of our immediate reality. To address the issue of representation, I next outline how image generators work, and endeavor to provide an account of the technical processes in a manner accessible to digital media scholars and a broader readership of this journal.

AI image generation: Mechanisms and critical inquiries

Current AI-enabled image generators draw on text-to-image diffusion models (see e.g. Rombach et al., 2022). Diffusion models represent a subclass of deep neural networks, a type of AI modeled on the human brain. Like the human brain, deep neural networks operate in covert ways (Castelvecchi, 2016), remaining opaque and largely hidden from human comprehension (Von Eschenbach, 2021). As efficiency and performance of diffusion models grow, little remains known about how they learn from themselves and make decisions on the visual outputs when prompted by human agents. The capabilities of diffusion models, as well as specific parameters they identify, remain elusive even to data scientists, with recent calls for a pause in AI development to address growing concerns of algorithmic opacity (Narayan et al., 2023).

Visual generative media have reached remarkable quality outputs in an extremely short period of time since the conference paper Generating Images from Captions with Attention by researchers from the University of Toronto (see Mansimov et al., 2016) presented a model for generating images from captions. Unimpressive as they are by 2024 standards, first AI-generated images served as a testament to the ability of deep neural networks to generate novel scenes not seen during training. This development has set a precedent for technology companies and communities of independent, open-source developers to build text-to-image generators at an exponential rate.

While it may appear that a diffusion model identifies objects from training data through verbal prompts provided by a human agent, or promptor, and then combines the identified objects to form a new image, the generative process is inherently more complex (Figure 1). A novel scene emerges from the latent space of a deep learning model which recognizes complex categories from a network with multiple layers, with earlier layers creating a representation of the raw data and the later layers operating on this representation (Christian, 2021). Thus, AI image generation unfolds through manifest systems, or systems external to human mind (Ehn, 1988: 48). Precisely because such systems are beyond human comprehension, their operational nature may “deepen some humanitarian problematics and introduce new ones of its own” (McQuillan, 2018).

OPEN IN VIEWER

Figure 1.

How generative AI models learn and produce visual outputs: (a) training data, (b) deep learning, (c) latent space, (d) generation, and (e) output.

To exemplify, for an image generator to be able to respond to the prompt “War in Ukraine, August 2023” and produce the visual output E in Figure 1, it first needs to be trained on hundreds of millions of images, known as training data (shown as A in Figure 1). Training data is usually scraped from the web together with captions that often come from texts uploaded for accessibility purposes. An exemplar large-scale image recognition dataset is ImageNet (see Deng at al., 2009), with over fourteen million images, each labeled with one of over 20,000 categories (Standford Vision Lab, 2020), consolidating existing taxonomies of the WordNet hierarchy and publicly available images. This dataset, widely used for training generative AI models, is riddled with two issues – publicly available images, often of a highly personal and compromising nature, were harvested without consent, and image labels carry various personal biases of the workers of Amazon’s Mechanical Turk who manually coded the data (Crawford, 2021; Crawford and Paglen, 2019). As ImageNet-trained models can be transferred to other tasks, problematic taxonomies and biases present in the original dataset can inadvertently propagate in various applications.

The next step of image generation model training is deep learning (B in Figure 1), facilitated by the ability of deep neural networks to analyze and draw inferences from patterns in large datasets. This process involves layering of multiple variables through building associations between them, drawing on the training data (Bengio et al., 2013). Deep learning is informed by the visual pattern recognition technology developed in the late 1950s which was then extended to the analysis of non-visual data (Pasquinelli, 2019). As a multilayered model with many nonlinear relations, it employs mathematical metrics to distinguish the mathematical representation of pixels in the deep learning space from measurements across vast amounts of variables and parameters that are unrecognizable to a human agent (Van der Velden et al., 2022). In other words, human agents neither know which sets of associations an algorithm assigns to which variables nor have any oversight of whether the visual output E in Figure 1 is representative of the complex phenomenon such as an event of which the system is prompted to generate a representation.

Similar to deep learning, latent space (C in Figure 1) constitutes a black box in AI image generation due to algorithmic opacity stemming from “remediable incomprehensibility” (Pasquale, 2015: 7). Due to limited transparency of machine learning processes within latent space, there is a growing concern that opaque systems might harbor biases that go undetected (Crawford, 2021; Von Eschenbach, 2021). For example, Birhane et al. (2021) points to troublesome, explicit images and text pairs of rape, pornography, racist and ethnic slurs, and other problematic content with the LAION-400-Million Open Dataset, a large-scale image recognition subset of ImageNet, used as a benchmark for evaluating performance of various computer vision models. From a digital memory perspective, Makhortykh et al. (2023) problematizes the use of visual generative media for memorialization of mass atrocities and argues that AI images can generate narratives that support distortion and denialism.

Lastly, the process of generation starts after a human agent interacts with the model through a verbal prompt. It begins with visual noise (D in Figure 1) gradually turning into an image (E in Figure 1). Generation is the first and only point of direct contact with a text-to-image generator, with all the intricacies of training data, machine learning, and algorithmic behavior of deep neural networks hidden from the human view.

In sum, AI-enabled image generation is a vastly opaque process of image production through (a) machine identification of the metrics that separate images in the mathematical space of the training dataset, (b) machine-initiated introduction of the variables into a multidimensional space to improve its algorithmic performance and pattern recognition, and (c) subsequent generation of images from verbal prompts introduced by a human agent. Critical questions arise in relation to (i) training datasets and their problematic taxonomies, (ii) black box training and generation, and (iii) production, distribution, and cultural significance of visual outputs. The current study primarily focuses on the latter, seeking to advance our understanding of representation in AI-generated images and their potential implications for memory mediatization in the context of conflicts and wars.

Data and method

The dataset for this study includes a corpus of twelve images generated through the Midjourney model V5.2, released in July 2023. In September 2023, I prompted Midjourney (MJ) to generate images around three points of Russia’s war against Ukraine, anchored in the past, present, and future:

The prompts were deliberately designed to be open-ended, that is, they do not specify any desired visual content by introducing specific object or style descriptors (e.g. image subjects, “in the style of,” etc.). Rather, I was interested in MJ’s visions of two events – Russia’s invasion of Ukraine and Ukraine’s counteroffensive – and one entity (i.e. Ukraine post-war). To leave as much space for MJ’s vision as possible, I only utilized a text prompt functionality of MJ without specifying any technical parameters (Figure 2).

OPEN IN VIEWER

Figure 2.

Types and specificity of prompts for generating images with Midjourney (https://docs.midjourney.com/docs/prompts-2).

MJ image generation is hosted on Discord (https://discord.com/). The MJ bot (@midjourney) is added to a Discord server, allowing registered members to generate images from verbal descriptions through the command /imagine. In the MJ V5.2, all registered users could see prompts input by other users, user Discord tags (unique identifiers starting with @), and images that the MJ bot generates. MJ’s Discord server had over fifteen million registered members in September 2023 (a statistic visible to the server members), with monthly subscription plans ranging from $10 to $120 in April 2024. As the only image generator that provides a shared social space during the process of image generation, MJ outpaces other prominent image generators such as DALL·E and Stable Diffusion, reaching its peak in popularity in May 2023 after the release of the MJ model V5.1 (https://docs.midjourney.com/docs/models). However, with the release of DALL·E 3 in September 2023, which offers in-built use of ChatGPT as “a brainstorming partner and refiner of prompts” (OpenAI, 2023), the popularity of DALL·E is swiftly approaching that of Midjourney. Relevant to this study, Ukraine is the eighth country in the world with the most interest in MJ, according to Google Trends (Figure 3).

OPEN IN VIEWER

Figure 3.

Popularity index of Midjourney, DALL·E, and Stable Diffusion in September 2022–September 2023 (https://trends.google.com/trends): (a) (left) – worldwide; (b) (right) – in Ukraine. 100 = Peak.

A search function on MJ’s website (https://www.midjourney.com/) resembles Pinterest’s “more ideas” feature, enabling users to explore associated prompts and AI images generated by other people. Notably, when other users input more detailed prompts to generate images specifying, for example, artistic style, color, contrast, or mood, MJ’s visual outputs were very similar to my open-ended prompts. Figure 4 shows one such example by comparing the visual output to Prompt #1 “Day 1, Russia’s invasion of Ukraine, 24 February 2022” and the following (highly specific) prompt by a Discord user @polynikez:

A platoon of 21st century Ukrainian infantry, Oil Painting, War Photography, Gamma, Chroma, High Contrast, 3D, 8K, Angry, Powerful, Moody Lighting, Mist, Ray Tracing Reflections, SSAO, in a symbolic style, patrolling the ruined streets of Bakhmut

OPEN IN VIEWER

Figure 4.

Homogeneity of visual outputs to open-ended (left) and highly specific (right) verbal prompts.

It is, therefore, noteworthy that regardless of whether prompts are open-ended and generalized or highly specific, the resulting visual outputs often exhibit a striking degree of similarity.

Analysis of the representational discourses of war in the generated corpus of images is grounded in visual social semiotics. I view discourses as semantic constructions of specific aspects of reality that serve the interests of particular social contexts. As Foucault (1977) notes, discourses do not only involve a field of objects (or visual analytics in machine learning) but also “the definition of a legitimate perspective for the agent of knowledge” (p. 199) in a given context. Taking a perspective to the use of visual generative media as the construction of meaning “trains our attention on what meanings are made in AI-generated images, making them ripe for being read as “texts”’ (Thomas and Thomson, 2023).

To understand how AI-generated images position the viewer to relate to image subjects (or represented participants), I first annotated the dataset in relation to three elements of visual composition – gaze, size of the frame, and angle (horizontal and vertical) – drawing on an annotation schema adapted from Kress and Van Leeuwen (2021), as summarized in Figure 5.

OPEN IN VIEWER

Figure 5.

Coding frame for analysis of representation-viewer relationships in AI-generated images. Adapted from Kress and Van Leeuwen (2021: 142).

The coding frame presented in Figure 5 is a system network which maps visual choices in images that position the viewer to relate to image subjects in specific ways. The system network reads from left to right, representing several design choices in relation to gaze (contact), size of the frame (social distance), and angle (attitude).

I explore MJ’s algorithmic imaginary by examining “the site of the image itself” (see Rose, 2016: 32–34) – as a digital media researcher, but also as a promptor. As a promptor, I assume the role of a social actor engaging in the co-creation of visual outputs alongside MJ’s bot on Discord. As a researcher, I operate on the premise that AI-generated images are not straightforward extrapolations from existing generative AI technology, but are to be understood as the “contingent outcome of a series of social, political, and organizational factors” (Woolgar, 2005: 389).

Results: Midjourney bot’s imaginings of the Russia-Ukraine war

All images generated by the MJ bot in response to the three prompts introduced in the previous section featured people, 90.5% of whom were visually represented as male. Female subjects appeared only in images generated for Prompt #3 (“Ukraine post-war”). Findings are presented in Figure 6 which visualizes three clusters of code co-occurrences in the annotated corpus. The codes are located on the map according to their similarity, and the distances between codes represent how similar they are. The statistical data analysis was performed in MAXQDA Analytics Pro 2022 (https://www.maxqda.com/), a software for mixed-method research. A quantitative data analysis tool “Code co-occurrence” embedded in the software was used to uncover which codes occur in the same prompt group and how similar these codes are across the corpus.

OPEN IN VIEWER

Figure 6.

Code co-occurrence model representing three clusters of interactive meanings in the annotated corpus.

Cluster 1, shown in orange in Figure 6, indicates that most AI images featured male subjects presented as individuals through over-the-shoulder shots (vicarious offer) at an impersonal social distance (long shots), angled away from the viewer (detachment), and at an eye level (equal power). This set of visual design choices appeared in 42% of the corpus. In such images, individuals are positioned in the foreground, wearing military uniforms with no identifying insignia and marching through the battlefield filled with smoke, fire, and signs of destruction (e.g. Figure 7). However, these visual choices do not result in warm portraits of the featured individuals, and instead, individuals are homogenized as anonymized soldiers, due to oblique angles, over-the-shoulder shots, and impersonal social distance. Such images do not evoke sympathy or solidarity with the depicted persons, and instead, communicate somberness of a generalized soldier, whom the viewer is invited to follow into the vanishing point of fire and smoke as indices of destruction.

OPEN IN VIEWER

Figure 7.

A typical output that combines the most representative visual choices in the annotated corpus.

The most common visual choice from the system of contact was vicarious offer which, together with detachment and impersonal social distance, positions the viewer to metaphorically follow the path of image subject(s) traversing the sights of destruction alongside their comrades. Because of this selection, eight images for both Prompt #1 and Prompt #2 exhibit substantial similarity, differing primarily in their representation of group versus individual subjects. Figure 8 illustrates this further by showing how individuals are presented as a more cohesive group in contrast to Figure 7.

OPEN IN VIEWER

Figure 8.

Group representation of soldiers moving across the battlefield.

However, in Figure 8, group unity is not achieved by the means of visual cohesion but rather through a perceptual connection to the contextual factor (i.e. war). In this kind of images, people were depicted at an impersonal social distance and detached from the viewer, that is, they are represented through long shots and at oblique angles. Overall, these sets of visual choices result in constructions of battlefield scenes at an unspecified location where soldiers move through the terrain, with no enemy in sight.

Cluster 2, shown in green in Figure 6, was identified in relation to group depictions, with groups of soldiers represented for observation (direct offer) and from a low angle (participant power). These depictions result in a higher degree of social cohesion realized by the placement of image subjects closer together. Yet again, people in groups appear homogenized, at an impersonal social distance, with hardly discernible individual features (Figure 9).

OPEN IN VIEWER

Figure 9.

Group representation that combines visual choices of direct offer and participant power.

Cluster 3, shown in purple in Figure 6, reflects visual choices in AI images generated to Prompt #3 related to Ukraine’s future post-war. This subset is unique in featuring female subjects, appearing as individuals at a socio-consultative distance. They are mostly angled toward the viewer (indicating involvement), make eye contact with the viewer (suggesting demand), and are represented at a high angle (signifying viewer power). In Figure 10, two female subjects make a visual appeal through gaze, “demanding” viewer attention. Salient facial features and grim expressions differentiate these images from the depictions of male subjects. The distinction is particularly evident through the transition from a detached perspective to an engaged frontal angle, a shift from an impersonal to a socio-consultative social distance, and a change from a vicarious offer to a demand.

OPEN IN VIEWER

Figure 10.

Individual portraits of female subjects “demanding” viewer attention.

In sum, the analysis reveals homogeneity of visual scenes depicting Russia’s invasion of Ukraine, Ukraine’s counteroffensive, and bleak visions of Ukraine’s future. MJ’s bot appeared to favor representing male subjects as individuals, at oblique angles, through long shots, “over the shoulder” shots, and at level angles (Supplemental Appendix D). These choices result in a particular construction of war scenes emphasizing “the follow me” visual narrative and positioning the viewer as a detached voyeur. Interestingly, the future is female, yet the harsh realities of war are depicted through the lens of male soldiers, as traditionally perceived by society.

Discussion: Visual themes in AI-generated images

Two main visual themes were identified in the annotated corpus: (1) soldiers and fighters (but not injury or death) and (2) destruction and aftermath (but not reconstruction efforts). These include both what is and is not represented in images, in which “the absence of something functions as an active component of a percept” (Arnheim, 1969: 89). Much like the challenge of discerning the causes behind visual absences in Google Photos, which may result from opaque algorithmic recommendations or inherent platform constraints (Brantner et al., 2024: 107), the issue becomes even more complex when applied to AI-generated images due to the mechanisms of deep neural networks discussed earlier.

The most prominent visual theme was that of soldiers and fighters, showing the human side of war. Most often, these kinds of images portray male soldiers moving through the battlefield (Prompts #1–2) or in the moment of contemplation (Prompt #3). Due to the choices of a vicarious offer, detachment, and impersonal social distance, individuals are not portrayed as integrated into the viewer’s world. Instead, such imagery conveys the perspective of an observer positioned to share the soldier’s experiences as an onlooker (Figure 11). Simultaneously, images featuring soldiers do not depict injuries or fatalities, combat, or the enemy, obfuscating the harsh realities of combat.

OPEN IN VIEWER

Figure 11.

Positioning of the viewer as a passive onlooker.

The second visual theme was centered around destruction and the aftermath of fighting. This theme is characterized by depictions of explosions, fire, smoke, and rubble which provide a contextual backdrop for the portrayal of soldiers and convey the environmental impact of war. At the same time, these images do not represent displaced civilians or the loss of cultural heritage and artifacts as part of the broader context, thereby generalizing depiction of this war to that of any war.

According to MJ, the future of Ukraine is also marked by infrastructural destruction. The images generated for Prompt #3 do not provide a glimmer of hope amidst devastation, showing no reconstruction efforts. Instead, they showcase male subjects engaged in a passive observation of the ruins (e.g. Figure 11) or female subjects making a visual “demand” through gaze (Figure 10). Overall, all the images encompass destruction and portray individuals in a manner that neither emphasizes their agency nor entices the viewer to develop an emotional connection with those impacted by the war.

Conclusions and future research

AI-enabled image generators represent a novel technology that has the potential to be widely integrated into various social practices and mediate public perceptions of political events, conflicts, and wars. Yet, a more complete understanding of the future(s) of visual generative media and its broader cultural implications remains an elusive task because the cultural and communicative shifts engendered by technological advancements are inexorably linked to the passage of time. Interrogating cultural spaces where the past, present, and future converge beyond the dominant narratives of Silicon Valley is necessary precisely because it lies beneath the surface of our immediate perception.

As visual generative media grow in sophistication, their complexity and opacity pose ethical and practical challenges, particularly in terms of data sourcing and representational capacity. Among these challenges, this paper was mainly concerned with the issue of representation, providing insights into depictions, perspectives, and visual themes in the context of the Russia-Ukraine war. I now return to the notion of AI image generation as a socio-technical practice and discuss several implications and limitations of this study in relation to mythology, technology, and representation in visual generative media.

Supplemental Material