Abstract
Interventional digital media applications such as robotic surgery, remote-controlled vehicles or wearable tracking devices pose a challenge to media research methodologically as well as conceptually. How do we go about analyzing operational media, where human and non-human agencies intertwine in seemingly inscrutable ways? This article introduces the method of operational analysis to systematically observe and critically analyze such situated, interventional and multilayered entanglements. Against the background of ongoing efforts to develop operational models for understanding digital media, the method of operational analysis conceptually ascribes to media technologies a real efficacy by approaching them as adaptive mediators. As an operational middle-range approach, it allows to integrate theoretical discussions with considerations of the situatedness, directedness, and task-orientation of operational media. The article presents an analytical toolbox for observing and analyzing digital media operations while simultaneously testing it on a particular application in robotic radiosurgery.
Keywords
Introduction
For every decade that passes since the advent of digital media technologies, it becomes increasingly clear that a prominent feature of new media is their refusal to keep their distance: They encroach into every sphere of life. Rather than restricting themselves to symbolic or virtual spheres, today’s media technologies seem to be characterized by how they infiltrate and take effect in the physical environment (Hansen, 2015; Hayles, 2010). When addressing processes of digital mediation, media researchers are increasingly faced with shifting entanglements between physical and virtual layers of operations and interventions—with systems where human and non-human agencies intertwine and intra-act 1 in often seemingly inscrutable ways.
This also means that today’s media, such as digital imaging, software applications and semi-autonomous systems, are not adequately described in established theoretical terms. While the idea that digital media are layered is widespread in recent discourses, these layers tend to be treated in rather dichotomous and monocausal ways. Software and image theorists, for example, heuristically differentiate between the cultural and the computer layer (Manovich, 2001: 46, 289), or alternatively, between the technical and the symbolic layer (Krämer and Bredekamp, 2003: 13). Other theoretical considerations draw on the idea of a reality–virtuality continuum, a model first introduced by Paul Milgram et al. (1995), which continues to be based on the common but mistaken (as this article argues) assumption that mediation is inversely correlated with reality. To get a better conceptual and analytical grip on such entangled systems, which significantly involve and instruct living entities, we propose a methodology that relies on operational rather than representational models, and that ascribes to technologies and media a real efficacy 2 by approaching them as what we call adaptive mediators.
A striking example of human/non-human entanglements of the kind under scrutiny here, is found in the heavily instrumented practices of contemporary medicine, where the question of the real efficacy of mediation processes is pushed to the extreme: In present-day operating rooms, the infiltration of virtual layers and machinic agents have tangible, real-life effects where the health and lives of patients are at stake (Friedrich and Diner, 2019). Other striking examples are drone warfare, space and underwater missions, or other remote operations. Entanglements of physical and virtual layers, of human and non-human agencies, are also found in everyday applications such as web mapping services, global positioning system (GPS) navigation systems, wearable tracking devices, information sharing on mobile devices, and algorithmically driven social platforms. Media-induced interventional practices pose a challenge to media theory of how to properly conceptualize digital media applications (e.g. Fuller, 2003; Galloway, 2012; Lister et al., 2008; Manovich, 2001). Among the various attempts to rethink media so as to accommodate the transformative effects of digitization, a particularly promising line of inquiry is one that suggests that classical representational models be supplemented, and perhaps even replaced, by operational models, shifting the focus from media as vehicles of communication and channels of information, to media as providers of infrastructural conditions that co-shape our lived environment (e.g. Mitchell and Hansen, 2010; Starosielski and Parks, 2015; Hoel, 2018). Aligning with the ongoing efforts to develop operational and infrastructural frameworks for conceptualizing new media, this article addresses a related challenge, methodological in nature, pertaining to how to systematically observe digital media applications in actual use, and how to appropriately analyze them in order to bring out their operational and interventional aspects.
The proposed approach expands the investigative field of media studies beyond what is commonly referred to as the media: the main means of mass communication, including broadcasting, publishing, and the Internet, regarded collectively. Research on media operations often requires what we call an empirical encounter through which media scholars make themselves familiar with the media application under investigation, and just as importantly, with its context of use. It may require, for example, that media scholars sojourn with communities that employ the application in question, or that they become involved in media design and development.
The shift to media operations raises methodological challenges. Established media studies methods provide few pointers, for example, as to how researchers are to go about collecting data at the site of observation, or how they are to conduct themselves in the design setting. In many cases, therefore, media scholars who venture into the expanded field of media studies resort to methods from other disciplines, for example, for analyzing human–computer interaction or conducting fieldwork. However, while these methods are certainly helpful, they often lack media studies-specific perspectives on media operations and human/non-human entanglements. At least this is the experience of the authors of this article, both of whom have conducted research (individually as well as jointly) on digital media applications in actual use—more precisely, on the use of media technologies and image-guided applications in radiology, radiation therapy, and neurosurgery in Norwegian and German hospitals (Hoel, 2016; Hoel and Lindseth 2016; Friedrich, 2018 and 2021). Having reflected upon the methodological challenges of this previous research, we have come to realize that existing methods for studying media applications in actual use tend to be either too technical or not technical enough. A new method is needed that addresses the nitty-gritty workings of the technologies and software involved, yet in a way that—and this is important—does not reduce media operations to mere technical operations by losing track of their social situatedness. What is needed is a method that approaches mediation in operational terms, while at the same time acknowledging that operation is a more-than-technical concept. Combining qualitative methods with detailed media-theoretical inquiries allows to shed light on situated social and technical interactions as well as on multilayered technical agencies which together constitute media applications’ operational impact. To help articulate the terms operational and technical in the sense meant here, and to develop our idea of technologies and media as adaptive mediators, we draw inspiration from the philosophy of Gilbert Simondon, whose ideas about machines and their environments offer a promising new way to conceptualize human/non-human entanglements.
Against this background, the aim of this article is to propose a new method for observing and analyzing digital media operations. We call this method operational analysis. By proposing this method, we seek to integrate theoretical considerations about the operational and interventional aspects of digital media technologies, including considerations of their inner workings, while simultaneously factoring in their contexts of use. We do this by adopting an operational middle-range approach, that is, an approach that takes its point of departure from the task (operation, procedure) to be performed.
To this end, the article focuses on a particular media application, a robotic radiosurgical system called CyberKnife. The CyberKnife radiosurgery system is currently used in state-of-the-art hospitals and clinics across the world in the treatment of cancerous and non-cancerous tumors and other conditions where radiation surgery is required. It allows selective destruction of tumor tissues using ionizing radiation, in contrast to traditional surgery, where tumor tissues are excised with a blade (Yu et al., 2006). The interventions of the CyberKnife system are based on visual planning in combination with real-time imaging and control (Clearly and Peters, 2010). The implementation of robotic radiosurgical systems into common clinical practice is indicative of a broader development in contemporary medicine where digital technologies promise to increase precision, patient-specificity, and efficiency of diagnostics and treatment (Friedrich and Diner, 2019). In surgery, for example, the rationale for improving health care revolves around minimizing invasiveness, which means that innovation is geared toward new technologies that transform surgical practice by enhancing surgeon perception and navigation, including automation and robotization (Fuerst et al., 2021). 3 The CyberKnife system exemplifies the kind of media-induced and sometimes remote entanglement of human bodies and non-human entities characteristic of the digital media applications that we aim to critically analyze. In this article, image-guided robotic radiosurgery serves as a paradigm for what we refer to as a media operation: a technologically mediated action or procedure where symbolic and virtual resources are gathered to effect changes in the physical environment.
The method of operational analysis is still in its early stages, first steps being taken in this article, which develops the methodological framework including the key analytical tools. In the process of developing this method, the CyberKnife system has served as a first testing ground, our exploratory empirical findings having been fed back into the process of concept development. While radiosurgery may seem an unusual topic for media researchers, we have chosen this case for three reasons: First, because we want to call attention to how digitization calls for an expansion of the investigative field of media studies, due to how media technologies and media-induced practices to an ever-increasing extent travel across use contexts and societal domains; second, because image-guided robotic surgery is an obvious candidate for what in contemporary media theory is referred to as image operations or operational media, typically exemplified with highly interventional media applications (military, medical, or other) where life and death may be at stake (e.g. Eder and Klonk, 2016; Farocki, 2004); and third, because (as already hinted) it instantiates the complex intertwining of human and non-human agencies that is the focus of this article. Moreover, as we will come back to in the concluding section, the media-induced practices involved in robotic radiosurgery (imaging and data visualization, screen-based interaction, remote navigation) correspond to those involved in more mundane media use (e.g. smart phones, computer tablets, and virtual/augmented reality devices)—providing support for the transferability of the proposed method of analysis.
Our observations and analytical explorations of radiosurgical media operations have contributed to the specification and stabilization of the methodological framework to be presented and discussed in the main sections of this article. We start out, however, by presenting the conceptual background for our work.
Toward an operational conceptual framework for understanding digital media applications
As mentioned in the “Introduction” section, the motivation for developing a new methodology springs from the emergence of new media that constitute and rely on new kinds of human-technology entanglements, but also, crucially, from the accompanying shift of interest from representational to operational conceptual frameworks. We start from the observation that established frameworks and methods fall short of accounting for what is arguably the most characteristic aspects of digital media applications, namely, their performative and agential—or as we prefer to put it, operational and interventional—aspects. We are currently witnessing a profound shift in conceptual models of mediation, where approaches emphasizing performance and action seem to gain in momentum. This conceptual shift is spurred by digitization, which seems to call for more dynamic and relational approaches.
An early example of approaches emphasizing performance and action is speech act theory in linguistics and the philosophy of language, which studies how language is often used to perform actions (Austin, 1962; Searle, 1969). The influence of speech act theory reverberates far beyond the study of language, having inspired, for example, image theorists to introduce the notion of image acts (Bredekamp, 2018), and overall, to change the focus from what images are and mean to what they do. Another highly influential approach is actor-network theory (ANT), which is notable in this context for its equal treatment of human and non-human actors. This implies that, in their descriptions of sociotechnical networks, actor-network theorists undertake an ontological leveling, refusing to grant privilege to human actors by assigning equal amounts of agency to non-human actors (Latour, 2005). The idea that things do things is also a guiding assumption of the theory of technological mediation at the heart of contemporary philosophy of technology (Verbeek, 2005). The investigations of material and non-human agency conducted by these approaches include considerations of how humans and technologies co-constitute each other (Ihde and Malafouris, 2018) or co-evolve (Hansen, 2015; Stiegler, 1998). While the latter approaches emphasize that tool-making and tool-use are by no means exclusive to humans, seeking to develop a non-anthropocentric approach to material agency, the non-anthropocentric bent is even stronger in new materialist approaches, such as the theory of agential realism propounded by Karen Barad (2007). Barad’s starting point is that apparatuses are not primarily human, nor assemblages comprised by humans and non-humans. More than mere observing instruments, Barad (2007: 19, 140) regards apparatuses as boundary-drawing practices that enact agential cuts through which certain features become determinate, while others are excluded.
In media studies, too, there are indications of a shift toward models emphasizing performance and action. Also in this case, the driving ideas are not entirely new. Already Marshall McLuhan, with his much-quoted phrase “the medium is the message,” sought to remind us that our abiding preoccupations with the contents and uses of media may sometimes blind us to the true social impacts of media, which play out not so much at the level of opinions or concepts as at the level of altered sense ratios and patterns of perception (McLuhan, 1964: 9, 19). Another line of research focusing on the operational aspects of media was initiated by Friedrich A Kittler (1986) in the 1980s who inspired much of today’s work on media archeology and cultural techniques (Ernst, 2013; Siegert, 2015). At present, media operations are also investigated from aesthetic and visual studies perspectives (Eder and Klonk, 2016; Farocki, 2004; Paglen, 2014, 2016). In addition, approaches from the field of software studies have contributed to analyze the operative role of software as a multilayered technical process that impacts digital knowledge as much as socio-cultural contexts (Berry, 2011; Chun, 2011; Nake, 2008).
The methodological framework to be sketched in this article takes inspiration from these approaches. It is important to note, however, that the operational models and frameworks here alluded to are themselves in a state of becoming. In a comparative study of recent approaches focusing on media operations, one of the present authors has shown that the notion of operation continues to be under-theorized as a media-theoretical concept (Hoel, 2018: 27). We go about our task, therefore, by investigating specific media applications in their contexts of use and tying these findings back to theoretical considerations in a constant and multilayered iterative circle. Moreover, as we hope to demonstrate in what follows, Simondon’s philosophy of technology is a rich resource for developing the notion of operation as a media-theoretical concept, for many reasons—among others, because it avoids the tendency, widespread in the contemporary literature on media operations, to see machines and humans (or other living entities) as antagonistic opposites.
Technologies and media as adaptive mediators
As stated, the method to be developed here draws on conceptual models that conceive of media in terms of their operational and interventional aspects. The methodology is operational in that it focuses on media operations, but also, crucially, in that it approaches mediation as such as operational: More than vehicles of communication, media are approached in terms of what they do, their modes of operation.
When it comes to theorizing the operational, Simondon’s philosophy of technology is as promising as it is original. Challenging established mechanistic approaches to machines, it inquires instead into the mode of existence of technical objects. In his seminal work on technology (Simondon, 2017), a technical object is approached as a being that exists and develops much in the same way as a living being. This implies, for example, that a technical object operates by entering into a system with its surroundings. In Simondon’s view, a technical object has philosophical import due to the way that it intervenes as a mediator between human and world (Simondon, 2017: 183). Its philosophical significance resides, more precisely, in its power to institute a new system of reality (or a new phase in such a system), and hence, in its power to shift the relation of the human to the world. It does this by playing “the role of prosthesis, at once adaptive and restrictive” (Simondon, 2014: 12)—which amounts to the same as saying that the technical object acts as an “adaptive mediator” (Simondon, 2014: 142). To explain what he means, Simondon uses the example of a winch, a technical device that is used to move a heavy load. The load in question is so heavy that the human is not able to move it by the resources of her body alone. This means that the task—moving the heavy load—cannot be resolved without the intervention of some external object that serves as an adaptive mediator between the human and the load. Without the winch, the human and the load belong, so to speak, to two heterogeneous orders, incompatible with each other and with no communication between them. With the intervention of the adaptive mediator (in this case, the winch), the situation changes; the winch brings about a shift in the human-world system; it amplifies the system by inducing a change of level, which Simondon specifies as a transfer to a new order of magnitude, that is, to a new system of reality or a new phase in the human-world system, where the human and the load are rendered compatible. Thanks to the adaptive mediation performed by the winch, the human is able to move the load, and hence, to act as if she were much stronger than she is, as if the load belonged to an order of magnitude homogeneous to her own (Simondon, 2014: 142). Hence, in its role as adaptive mediator, the technical object institutes a new “middle order of magnitude,” which overcomes the initial absence of interactive communication between the disparate parts of the human-world system (Simondon, 1992: 301, 304). The interventional action of technical mediators builds an intermediate, structured world through which the relation of human to nature “takes on a status of stability, of consistency, making it a reality that has laws and an ordered permanence.” (Simondon, 2017: 251). In their roles as adaptive mediators, then, technical objects enact a new resolution of the human-world system, a material reconfiguration of the world that releases new potentials for perception and action.
It is important to note, here, that in Simondon’s view, adaptation is not a mere defensive reaction to a pre-existing environment. Instead, the adaptive mediator operates by conditioning the birth of a new environment, which Simondon refers to as an “associated environment.” 4 The modifications that mediate between the two environments call forth the creation of a third environment—a hybrid and mixed environment that is at once technical and natural. There are two more points to be made about this associated environment. First, even if the hybrid environment is called forth by the interventional action of a technical object, it is not, as Simondon insists, entirely fabricated. It is not fabricated because it “incorporates a part of the natural world” (Simondon, 2017: 49), which in turn intervenes as a condition for the technical object’s functioning. More precisely, the natural elements intervene as a condition of functioning by being incorporated in the system of causes and effects that reigns in the new environment. This leads us to the second point: by conditioning the birth of an associated environment, the technical object gives rise to a recurrent regime of “reciprocal causalities” (Simondon, 2017: 26–27, 59). This implies that the specificity of a technical object—so Simondon’s argument goes—can be ascertained only through an investigation of its recurrent operational regime, that is, of the technical and natural elements in their mutual reactions. Thus, as conceived by Simondon, the technical operation is always more-than-technical, and at the same time, more-than-human, since it involves a “convertibility of the human into the natural and of the natural into the human” (Simondon, 2017: 251).
In devising a method for observing and analyzing digital media operations, we build on Simondon’s idea of technical objects as adaptive mediators.
Initiating an operational analysis: engaging in empirical encounters
The aim of operational analysis is to bring out the operational and interventional aspects of digital media applications. Since these aspects take effect and play out most strikingly when the application is put into use, it becomes indispensably important to pay due attention to the situations in which media operations unfold. Engaging in an empirical encounter allows the operational analyst to map the use context, and hence, to identify the processes and humans aligned with the media technology under scrutiny. This helps the analyst to decide which operations and which human/non-human entanglements to focus on in the ensuing analysis. Existing empirical methods developed in other disciplines offer inspiring tools that we have drawn upon in developing the proposed method.
Studies from the fields of philosophy of technology, human–computer interaction and Science and Technology Studies (STS), for example, share an interest in systematically describing and analyzing the transformative effects of technologies, offering ideas on how to approach situated media operations. Approaches from the philosophy of technology, such as postphenomenology, provide a methodological framework for studying human-technology relations, including questions of embodiment (Ihde, 1990, 2002; Verbeek, 2005). Of equal interest are the more design-driven approaches to studying human–computer interaction, which include methods such as user-centered design (Bannon, 2000) and research through design (Stappers and Giaccardi, 2017). In STS, a common method is to follow the actors (Latour, 1987), be they human or non-human.
It also seems highly productive to approach human/non-human entanglements more broadly as socially situated actions (Suchman, 2007). Recent research on situated human–machine relations in professional contexts makes use of a wide spectrum of empirical methods, including ethnographic research methods such as informal interviews and participant observation (Alac, 2011; Vertesi, 2014). Other researchers have developed these empirical methods further by adapting ethnography into digital ethnography (Pink et al., 2016), or by engaging in a more hands-on fashion with media technologies and their use contexts to analyze their impacts both critically and productively (Ostherr, 2018; Parks, 2013). New methods have also been proposed to engage creatively with situated media technologies (Kember and Zylinska, 2012).
We seek to add to this growing list of new media studies approaches by devising a method for critically analyzing digital media operations conducted in practical, goal-directed use contexts. While this implies developing a method that factors in the site-specificity of media operations, the proposed method is distinguished by how it also pays close attention to the multilayered agencies at work within specific applications. Since the method of operational analysis stresses the situatedness and task-orientation of the media operations under investigation, an empirical encounter is often required at an early stage of the analysis and also at critical stages along the way as the process of analysis unfolds. However, since contexts and tasks are multiple, it is not possible to decide beforehand which empirical methods should be used by the operational analyst. In many cases, this decision itself requires initial informal meetings with users, preliminary observations of use contexts, or other measures to get acquainted with the media application of interest and its associated practices and discourses.
As preparatory measures to undertake operational analyses of robot-assisted radiosurgical media operations, we have conducted two short phases of participatory observation, including informal interviews and field notes. At the initial stage of the analysis, we visited the radiation suite to observe processes and practices relating to the robot-assisted delivery of radiation surgery, including the clinical discourse of medical practitioners involved in carrying out the procedure. Drawing on these initial observations, we decided to attend a second radiosurgical procedure, this time to undertake more systematic and targeted observations of the treatment planning and the subsequent radiation delivery. In addition to these clinical observations, we have consulted technical publications, websites and instruction videos to deepen our understanding of the radiosurgical system under investigation, including the technical discourse on its functionality and use.
A conceptual toolbox for analyzing digital media operations
Media operations involve entanglements that cut across established boundaries between the human, the technical, and the natural. To address the challenge of analyzing such entanglements, we have devised a set of specialized analytical tools. In developing the operational analysis toolbox, we took our point of departure from the idea that digital media operations consist of layered agencies, and that hybridity remains also within each layer. The idea, therefore, is not to disentangle, say, the technical from the natural or the human from the technical. Instead, agencies are conceived as hybrid and relational through and through.
The Simondonian idea of recurrent operational regimes suggests that the new cross-over environments instituted and sustained by adaptive mediators are highly structured spaces where heterogeneous elements and forces come to affect and reinforce each other in regulated and law-like manners. The method of operational analysis thus proceeds on the assumption that media operations consist of fine-grained layerings of agencies that play out in structurally ordered ways both temporally and spatially—making up what Simondon calls recurrent operational regimes. The method also assumes that media operations can be investigated on different operational scales (meso, micro, and macro), and hence, that media operations are stratified—meaning that, in many cases, different regimes of reciprocal causalities are operating simultaneously yet on different levels. The operational analysis, therefore, seeks to single out the critical operational layers for a certain situated task and to characterize the specific regimes of reciprocal causalities that prevail in each of these layers. In this way, it aims to bring out the new potentials for perception and action that are released at critical moments of the mediated procedure.
Operational analysis differs from existing approaches by starting from the action or procedure to be performed, referred to in the following as the task of interest. The process of analysis traverses the task trajectory, focusing on critical steps of its execution, which we call the operative moments of the media operation in question. The operational analysis starts out, in other words, by identifying the task to be performed, including the key moments of its execution trajectory. This means that in operational analyses, the task of interest serves as a heuristic principle of relevance, guiding the analyst in determining which factors to focus on.
While the notion of operative moment is inspired by Simondon’s account of machines and their environments, we push this idea further by allowing for media operations that are composed of a series of operative moments, that is, for media operations that are not executed in one go but instead consist of several sub-procedures that are carried out at different times and in different localities. This implies that some kind of coordinating infrastructure must be in place that ensures the comparability and transferability of information across operative moments. We refer to such coordinating infrastructures as alignment grids. In the case of digital media operations, alignment grids often take the form of a coordinate system—in our case, a Cartesian coordinate system. This means that the identification and circumscription of the task of interest and the key operative moments must also include a consideration of the alignment grid that sustains the integrity of the media operation under scrutiny, by allowing the characterization of separate operative moments as belonging to the same overarching task trajectory.
The boundaries and identity of a certain media operation are not a given but a product of what we call an analytical cut performed by the operational analyst. As the process of operational analysis unfolds, the analysis traverses the media operation to be analyzed along its temporal trajectory, enacting what we refer to as a horizontal analytical resolution of the media operation in question, identifying and circumscribing the key operational moments and the overarching alignment grid that binds these moments together. The analysis then proceeds to undertake punctual deep drillings at each of the identified operative moments, thus enacting vertical analytical resolutions on various operational scales—meso, micro, and macro—to bring out the multiple layers of operativity that play out simultaneously.
Even if the process of operational analysis sets out as an operational middle-range approach, starting from the meso-scale procedure to be performed, the analysis proceeds to investigate human or non-human entanglements on multiple scales by zooming in and out of various layers at micro and macro scales. Thus, what we are envisioning is a methodological panning or zooming device, referred to in the following as the analytical zoom lens, which enacts multiple resolutions of the field of investigation as it moves back and forth, zooming in and out.
In the following, we present the key analytical tools of operational analysis in more detail, relating them to the example of robot-assisted radiosurgery.
Task of interest
The method of operational analysis takes operations as its unit of analysis. The process of operational analysis starts out, therefore, by circumscribing what is to count as a media operation in the analysis to be conducted. It begins, in other words, by seeking to identify the task of interest that is enabled by digital media applications serving as adaptive mediators. While the delineation of the media operation is ultimately a product of an analytical cut enacted by the operational analyst, the analytical cutting act takes its cues from the practical use context—in our case, the clinical radiosurgical setting.
Our empirical encounters enabled us to map multiple clinical actors, processes, and practices that are involved in radiosurgical treatment. During the first encounter, we observed a treatment process of a patient with tumors in the brain, while informally interviewing the attending surgeon and technical staff. These observations and conversations helped us to identify the task of interest and circumscribe the radiosurgical media operation to be analyzed as follows: the delivery of radiosurgery for the purpose of treating tumors inside the patient’s brain, by optimizing the tumor radiation dose while at the same time avoiding damage to healthy tissues and risk structures such as the eyes.
Identifying, defining, and tracking the boundaries between pathological and normal tissues is one of the main challenges of radiosurgical treatment (Jaffray, 2012). Digital media technologies are key to tackle this challenge, enabling surgeons to accomplish tasks that, under other conditions, would be impossible. During the second encounter, we observed another radiosurgical case, this time more systematically, since we already knew more about what to expect. In this case, the task of interest was similar, except that the patient was diagnosed with tumors inside the lung. In conversation with medical and technical staff, we further exchanged about our preliminary idea of the task of interest and discussed the main stages and technologies of the treatment process. On this basis, we performed our initial analytical cuts, including identifying subtasks and key operative moments.
Operative moment
Having decided on the task of interest, the operational analyst proceeds to ask whether the media operation under scrutiny is carried out in one go, or whether, as is often the case, it is carried out through a series of subtasks, executed at different times and places. If the latter is the case, the overarching task needs to be divided into its relevant subtasks, each of which corresponds to a certain operative moment of the procedure. The identification of operative moments sets the stage for the subsequent analysis, where the operational analyst moves on to analyze each operative moment in terms of its operational layers, characterizing each of these layers in terms of their specific media-induced environments, and operational regimes of reciprocal causalities.
On the basis of the empirical encounters just described, we identified three operative moments that substantially contribute to accomplishing the overarching task of interest of treating tumors through the delivery of radiosurgery: the acquisition of planning data through computed tomography (CT) scanning, the generation and assessment of a treatment plan through the application of dedicated software, and finally, the execution of the treatment plan by means of the robotic system.
The first operative moment of CT scanning allows the operational analyst to trace how the patient’s body and the digital imaging system are rendered compatible. The compatibility of the two heterogeneous systems is a prerequisite for the ensuing execution of the mediated treatment process. The CT scanner acts as an adaptive mediator that mutually ties together the patient’s body, data acquisition processes and geometrical reference systems—an initial and enabling adaptation process that involves continual interventions and supervision of human operators (radiologists and technical staff). The new, hybrid associated environment releases new potentials for perception and action, as it allows to generate the imaging data that forms the basis of the subsequent planning stage.
The second operative moment of software-based treatment planning establishes yet another media-induced environment in which the imaging data obtained through the CT scanning is analyzed and subjected to various planning algorithms of the CyberKnife software to calculate the most effective radiation dose and the treatment delivery plan. While the generation of the treatment plan involves automated software procedures, the process is continually overseen and managed by human operators (surgeons and medical physicists), who assess the multiple treatment plans suggested by the software, and who decide on which plan to go forward with. The planning stage demonstrates another aspect of the human/non-human entanglement, showing how certain micro-scale media operations such as software algorithms are situated and used interventionally.
Finally, the third operative moment of image-guided radiation delivery marks the final step in our identified overarching task trajectory. It allows the operational analyst to observe how human and non-human agencies collaborate to ensure the alignment of physical and virtual layers during the radiation procedure. This includes the continuous control of the patient’s position on the CyberKnife treatment table, while the linear accelerator mounted on a robotic arm moves around the patient to deliver radiation in accordance with the treatment plan. During radiation delivery, a number of measures are in place to track the position of the patient, to guarantee that the tumor is precisely targeted and to prevent radiation from diverting into healthy tissues due to patient movement. During patient set-up, positioning lasers are used to align the patient’s body and the radiation system. This initial alignment also involves a collaboration between humans and machine as the medical staff needs to manually adjust the patient’s position on the table by way of the position lasers to make sure that the patient is within reach of the machine’s automatic alignment system. Next, during treatment delivery, x-ray images are taken at regular intervals to track patient position and tumor location in real time, allowing the system to adjust the positions of the linear accelerator and treatment table accordingly. The correct position of the target is obtained by overlaying the x-rays obtained on table with the previously obtained CT imaging data. However, since the two imaging formats are not directly comparable, the initial planning data have to be recalculated, being transformed into digitally reconstructed radiographs— in other words, into a format that allows comparison between the two sets of imaging data with regard to spatial alignment. The overlaid images are presented in the graphical user interface of the CyberKnife control software, to be visually inspected by the medical staff, in addition to being matched by the technical system. In undertaking this comparison, the human actors look for corresponding anatomical landmarks in the overlaid images, while the technical system identifies corresponding geometrical nodes in the respective data sets.
Alignment grid
While Simondon focuses on how the media-induced middle-order environment allows the heterogeneous parts of the systems to communicate with each other, he does not explain how the information obtained in one specific environment or regime is made to communicate with the information obtained in other environments or regimes that belong to the same series of sub-operations. 5 This, then, is where the idea of an alignment grid comes in. In the context of digital media applications, the cross-moment alignment grid typically involves some kind of geometrical system, such as in our case, a Cartesian coordinate system. In medical procedures such as radiosurgery, cross-moment compatibility is absolutely critical (Adler et al., 1999). Here, digital layers, for example, CT imaging data, are generated in the initial operative moment, forming the basis of the subtasks to follow (treatment planning and treatment delivery). The application of a (virtual) Cartesian coordinate system within the scanner technology ensures the alignment of human and non-human agencies both within and across the operative moments (Fitsch and Friedrich, 2018). For example, it is thanks to this coordinate system that the CT images obtained during the first operative moment can be transformed into digitally reconstructed radiographs comparable to the real-time x-rays obtained in the third operative moment; and it is only due to this coordinate system that the virtual and symbolic resources of the two sets of imaging data, in turn, can be brought to bear on the physical space of the patient body, and hence, together with the positioning lasers and the real-time x-ray images, enable the accurate positioning of the linear accelerator during treatment delivery.
In former years, alignment in radiosurgical procedures (for cranial pathologies) was ensured through the application of material measures such as stereotactic frames. These metal devices were attached to the patient’s head during preoperative imaging to determine the location of the surgical target in the resulting images, and subsequently, during the surgical procedure, to reassign the planning to the patient’s body (Friedrich, 2017). Contemporary radiosurgery systems differ from the classical systems by allowing frameless navigation, the coordinate reference system now forming an integral part of the digital media systems involved in the process. This implies that the Cartesian coordinate system serves as an adaptive mediator in its own right, aligning human and non-human agencies on multiple scales and levels. Within each operative moment, it allows compatibility between heterogeneous elements and forces by rendering them spatially compatible, which in turn, serves to align physical and virtual spaces (transspatial alignment). Moreover, within and across operative moments, it aligns different kinds of imaging and measurement data, again by rendering them spatially compatible—thus ensuring comparability and transferability across operative moments (transtemporal alignment). In this way, it ensures alignment between the various stages in the treatment process, that is, from diagnostics through treatment planning and execution to postoperative control. Hence, along with the CT scanner and the CyberKnife system, the alignment grid is a critical enabling tool when it comes to using symbolic and virtual resources in physical interventions—in our case, the targeted destruction of tumorous tissues in patient bodies.
Analytical resolutions
The idea that media operations are made up of layered agencies, geared toward performing a certain task and coordinated by an overarching alignment grid, implies that they can be analyzed systematically at different analytical resolutions. The concept of analytical resolution allows the operational analyst to examine human/non-human entanglements in more detail, by probing deeper into the layered agencies at work without losing track of the relational, differential, and reciprocal nature of the elements and forces involved in operational processes. To get a grasp of a certain media operation, the operational analyst needs to consider it along two axes: temporally along the task trajectory (horizontal analytical resolution), and in depth within each operative moment (vertical analytical resolution). The scope and scale of both of these analytical resolutions are defined by the media theoretical interests of the analysts as well as by the overarching task of interest and its key operative moments, which have been identified and circumscribed on the basis of one or several empirical encounters.
The horizontal resolution analyzes operative moments with regard to their temporal situatedness on the meso level and provides a more systematic overview of these operative moments across time and context, including information about the overarching alignment grid. In the case of radiosurgical treatment with the CyberKnife system, this entails to focus on how the three operative moments depend on each other in a transtemporal perspective. Since the operative moments mark consecutive steps in the clinical treatment regime of a patient, tracing their chronological sequence allows the operational analyst to detect workflows and changing responsibilities in the course of the media-based treatment. In a broader analytical perspective, the horizontal resolution also allows the operational analyst to zoom out to the macro level, for example, by considering the historical development of a certain task or subtask. A horizontal macro-scale approach to radiosurgery could include contrasting today’s software-based planning systems with historical methods for entangling human and non-human agencies by means of analog inscription devices like pens, rulers, and x-ray sheets, or alternatively, by means of the above-mentioned stereotactic frames (Friedrich, 2017).
The vertical resolution zooms in on a specific operative moment to investigate in more detail the deep layerings of heterogeneous agencies, starting from the meso level and zooming in and out to related critical operational levels for accomplishing the subtasks and overarching task. Again, also in this case, the various operational levels are not simply given, but need to be identified and circumscribed by the operational analyst. Zooming in and out amounts to making analytical cuts, which may serve to bring out the often black-boxed workings of parts of the system, such as software applications and digital imaging systems. 6 By undertaking punctual deep drillings of a certain operative moment, the operational analyst becomes able to scrutinize the role of codes, data, and algorithms with a view to how they prefigure the software’s operational range. In the case of CyberKnife, a particularly relevant vertical resolution would be to zoom in on the underlying data standard called Digital Imaging and Communications in Medicine (DICOM), which provides the technological basis for encoding, exchanging, and displaying medical imaging data within the digital infrastructures of hospitals and clinics. In the medical setting, DICOM serves as an adaptive mediator by virtue of being both a network protocol for data transmission and an imaging standard for handling a broad range of imaging and visualization modalities. In this respect, the DICOM standard simultaneously determines how patients are rendered machine-readable, and how human operators are able to perceive and handle these data across heterogeneous imaging devices and software solutions. Zooming in on DICOM allows the operational analyst to bring out an operational layer that formally, yet silently, encodes and processes medical imaging information, and in so doing, determines what becomes visible and therefore operational for medical staff. 7
Analytical zoom lens
As already stated, operational analysis is an operational middle-range approach that takes its point of departure from a certain action or task that is accomplished through technological mediators. The operational analyst, therefore, starts out by making an initial analytical cut that circumscribes the media operation to be analyzed, proceeding to make further cuts along the way, singling out the critical steps of the task and the operative layers to be investigated in more detail. This means that the operational analysis involves multiple vertical shifts between macro, meso, and micro levels of media operations, as well as frequent movements back and forth along the temporal or horizontal axis of the task trajectory.
To account for these analytical cuts, shifts, and movements, we are imagining, metaphorically put, an analytical zoom lens, that is, a dynamic and inquisitive magnifying lens that guides the operational analyst and allows her to move back and forth and to zoom in and out of operational layers in a systematic and shareable way. The metaphor of an adjustable and dynamic zoom lens reminds us that each and every analytical cut intervenes into the phenomena under investigation by delineating their boundaries. It reminds us, in other words, that analysis is a boundary-drawing practice (to speak with Barad) and that an analytical cut is always at the same time an ontological cut. 8 Moreover, the metaphor of an analytical zoom lens helps recognize that the phenomenon is seen from somewhere (to speak with Haraway), that the analytical cuts entail a methodological viewpoint—a line of sight provided by the analytical machinery that guides the ensuing investigations. 9 This, in turn, emphasizes the importance throughout the analysis to account for the bases on which the cuts are made and the analytical level at which we are at. Thus, by way of virtually moving the analytical zoom lens back and forth along the task trajectory, and in and out at each operative moment, we are made aware that, for each setting of the analytical zoom lens, there is a specific resolution of the analytical field, which allows a specific configuration of agencies to come into view. Thus, just as the operational analyst seeks to unearth how digital media applications set conditions for our lived environments, for what we can see, think, and do, the idea of an analytical zoom lens helps the analyst to become aware of the methodological conditions of the analysis performed, and to be explicit about how the settings of the analytical machinery co-determine the boundaries of, and between, the operational regimes under scrutiny.
In contemporary media theory, visual metaphors have come under criticism, being associated with ocularcentrism: the tendency in philosophy to privilege (rational and disembodied) vision. This tendency is especially pronounced in Cartesian epistemology, which inaugurated what Martin Jay has termed “the modern visualist paradigm” (Jay, 1993: 69–70). However, despite the use of visual metaphors, the method of operational analysis is not ocularcentrist, for two reasons: First, it draws on a conceptual framework that is decidedly non-Cartesian. As one of the authors has shown elsewhere (Hoel, 2020), the notion of adaptive mediators grows from a framework that assumes an ecological organism/environment model that rejects the Cartesian subject/object scheme, including its associated ideas of transparent representation and a God’s eye view. By using the visual metaphors the way we do, we seek to relieve them from their Cartesian heritage, reclaiming them for an operational–ecological framework by acknowledging the inventive and interventional aspects of zooming (what we refer to above as analytical or ontological cutting). Second, the method of operational analysis is designed to critically analyze media-induced operational and interventional practices that may or may not include visual media. This is why we prefer to talk about media operations rather than image operations. We return to this second point in our concluding discussion of the transferability of the proposed method.
Operational analysis: advantages, limitations, and transferability
The method of operational analysis allows to systematically observe and analyze digital media operations and their layered agencies, by accounting for the hybrid environments or recurrent operational regimes that media technologies induce at various stages of mediated processes. It is geared toward analyzing these hybrid environments, not by separating and disentangling agencies and sorting them into kinds (say, into its human, technical, and natural constituents, considered separately). Instead, it provides a method of analysis that is sensitive to the spacetime-critical aspects of media operations, including how media technologies operate through the establishment of middle-order associated environments where human and non-human elements and forces intra-act, forming precarious operational regimes of reciprocal causalities.
The most original aspect of the proposed method is the combination of a theoretical interest in operational media and adaptive mediators, with a pragmatic interest in situated and task-oriented actions accessed through empirical encounters. The resulting methodological framework offers an operational middle-range approach that opens new avenues for analyzing the entanglement of physical and virtual layers and the multilayered agencies at work in today’s digital media applications. The notions of operational media and adaptive mediators, including the closely related notions of layered agencies, media-induced environments, and recurrent operational regimes, allow the operational analyst to probe deeper into the workings of specific media technologies than is usually the case in existing methods for studying human-technology relations. At the same time, the proposed method does not get lost in technical considerations, keeping in mind that, in the context of operational analysis, the notion of operation is a more-than-technical concept, and that there are important socio-cultural aspects to media operations that need to be factored in at all stages of the analysis. For example, as indicated by our brief analysis of the CyberKnife system, the implementation of image-guided and remote-controlled robotic systems in surgical practice occasions a media-induced socio-cultural transformation of the surgical profession, challenging the practical knowledge of surgeons by requiring new sets of skills that are increasingly geared toward handling computer-interfaces. For all that, and contrary to a widespread narrative in the discourse on operational media, according to which humans in digitized society are facing an imminent replacement by machines (e.g. Paglen, 2016), the operational analysis of CyberKnife shows that the human element very much remains in the loop. The introduction of a robot-assisted system in surgery does not lead to a replacement of humans by machines. Instead, our analysis shows that, at all stages in the execution of the radiosurgical media operation, there is collaboration between human and non-human agencies.
The operational middle-range approach proposed in this article is distinguished from other qualitative methods in that it emphasizes the middle (understood at once as medium and milieu) in a new sense: an adaptive mediator is seen as an inventive force that intervenes into the human-world system by instituting a new middle-order environment where heterogeneous forces and elements are made compatible and able to communicate. By so doing, the adaptive mediator brings about a shift, a change of level, that amplifies the system by releasing new potentials for perception and action. This inventive and amplificatory process is what mediation is taken to mean in this article—a dynamic and operational process that Simondon refers to as transduction (Simondon, 2017). Thus, to approach digital media applications as adaptive mediators is to acknowledge their transductive roles. This implies that the method of operational analysis, too, includes transductive aspects, the analytical machinery serving as an adaptive mediator of sorts. The focus on transduction sets the proposed method apart from established qualitative approaches, which rely primarily on inductive (and to some extent deductive) methods. 10
The proposed method is also distinguished by the way that it—by assuming that digital media applications consist of layered agencies, and by devising a method for addressing these agencies at multiple levels—provides more specificity than comparable methods, such as ANT. It differs also, in that it assumes an operational–ecological notion of relationality in contrast to the poststructuralist notion of relationality that underpins ANT and related methods (Hoel, 2021), which again lends more specificity, since on the proposed approach, an associated environment is not a mere network or system but a recurrent regime of reciprocal causalities (a heterogeneous yet structured environment where elements and forces affect and reinforce each other in a law-like manner). Furthermore, to continue the comparison with ANT, by taking operations as its unit of analysis (starting from the task of interest), the method of operational analysis provides a heuristic principle that guides the analyst in what to focus on, something that is not always clear in ANT, as pointed out by John Law (2009: 148).
For all that, there are also obvious limitations to the proposed method. The emphasis on media operations, conceived as mediated interventions in the form of circumscribed tasks or specific goal-directed actions, provide focus for the analysis. This focus makes operational analysis an exceptionally targeted method of analysis, designed as it is to bring out the operational and interventional aspects of (digital) media operations. Thus, by delineating and identifying the task of interest, the operational analyst simultaneously determines the scope of the ensuing analysis—including a great many things that will not be taken into account.
The method of operational analysis, as presented here, raises questions about the transferability of the method in two directions. First, while we have developed the proposed method by using image-guided robotic radiosurgery as our first testing ground, we envision the method to have great prospects when it comes to observing and analyzing other digital media, including more familiar everyday media—to the extent that they rely on complex human/non-human entanglements and the constant alignment of physical and virtual layers of mediation. The proposed method fills a gap, by providing an analytical toolbox geared to bring out the operational and interventional aspects of complex digital media applications, thus providing a systematic take on analyzing the digital yet situated characteristics of these media. Moreover, the task-orientation, and the suggestion to use operations as units of analysis, provides a new and promising angle to study, say, the societal impacts of social media. Other obvious candidates for operational analysis would be digital media applications that involve augmented reality and navigation, for example, Google Maps; or, to point to an example within the domain of creative media, the practice of virtual film production. We also anticipate that the proposed method can make a significant difference when it comes to the critical study of various forms of machine agency, including artificial intelligence (AI)-driven media applications, ranging from facial recognition, through self-driving cars and automated decision-making, to deep fakes. Second, while we developed the method of operational analysis to solve a problem encountered in the analysis of digital media operations, we envision the proposed method to be applicable to non-digital media operations as well. As has been duly established in the history of media and science, instrumental uses of media technologies long predate the advent of digital media (e.g. Cartwright, 1995; Curtis, 2015; Gaycken, 2015). This implies that operational uses of media—including entanglements of human and non-human agencies and interlockings of physical and virtual layers of mediation—are nothing new. We have hopes, therefore, that the analytical toolbox presented here may also prove useful when it comes to bringing out the operational and interventional aspects of historical media applications—thus emphasizing operational continuities across non-digital and digital media.
Footnotes
Acknowledgements
The authors would like to thank the Charité CyberKnife Center Berlin for the opportunity to conduct fieldwork and interviews. We would also like to thank the members of the research group SENSING: The Knowledge of Sensitive Media at ZeM-Brandenburg Centre for Media Studies and the members of the Media Aesthetics Group in the Department of Media and Communication at the University of Oslo for their productive feedback on an earlier version of this article. We thank Carmen Westermeier for her help in editing the article. Last but not least, we want to express our gratitude to the three anonymous reviewers for their thoughtful comments and suggestions.
Authors’ Note
Kathrin Friedrich and A S Aurora Hoel are co-first authors of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Initial research was funded by the Image Knowledge Gestaltung: An Interdisciplinary Laboratory Cluster of Excellence at Humboldt University of Berlin (EXC 1027/1) with financial support from the German Research Foundation. Kathrin Friedrich received funding from the Volkswagen Foundation as a member of the research group SENSING: The Knowledge of Sensitive Media at ZeM-Brandenburg Centre for Media Studies. A S Aurora Hoel received funding from the European Commission as a Marie Skłodowska-Curie Actions Fellow (H2020-MSCA-IF-2014-661526).
