Cybernetic Sense

In his autobiography, Ex-Prodigy, Norbert Wiener wrote of an obsession with transforming the practices of observation in science. Invoking natural history, he recalled that as a child he had, ‘longed to be a naturalist as other boys longed to be policemen and locomotive engineers. I was only dimly aware of the way in which the age of the naturalist and explorer was running out, leaving the mere tasks of gleaning to the next generation’ (Wiener 1964, 63). Developing this theme, he would later write, ‘even in zoology and botany, it was diagrams of complicated structures and the problems of growth and organization which excited my interest fully as much as tales of adventure and discovery…’ (Wiener 1964, 63). Written in a reflective moment after World War II, Wiener's comments sought to mark the passing of one age to another — the end of ‘exploration’ and the emergence of another type of ‘organization’.

Continuing on the theme of methodology and perception, Wiener added that in observing too closely and documenting too ‘meticulously’, one is unable to deduce patterns, to produce in his words a ‘flow of ideas’. He wrote that, ‘if he [a student] decides to take notes at all, he has already destroyed much of his ability to grasp the argument in flight, and at the end of the course has nothing but a mass of illegible scribble….it is far better to give up the idea of taking notes and to organize in his mind the material as it comes to him from the speaker’ (Wiener 1964, 63). Ex-Prodigy's obsessive implication was this gap between thought and action, and not, as the autobiographical genre might lead us to expect, the need to document or account for past experiences. This subtle shift of emphasis away from concerns with documentary and personal experience opens a site to excavate the historical reformulation of relations between thinking and sensing.

Wiener's memoirs bridge between late nineteenth and early twentieth century ideals of knowledge, representation, and perception and post mid-twentieth century concepts of organization, cognition, and process. He expressed a desire to see previous traditions in natural history and scientific representation replaced by a discourse of active diagrams, processes, and complexity. To make the hand analogous, as fast as and in function similar to the mind, can be said to be one of the major imperatives for the future science of cybernetics.

If Wiener articulated a faith in the eclipse of older forms of science and observation by cybernetics, it was only substantiated on the archive of the past. Wiener had highly dissonant understandings of how to define the relationship between cybernetics and perception. On the one hand, he believed that cybernetics, and by extension computational technology, possessed a normative prosthetic capacity. Deeply attuned to ideas of human limits and disabilities (the trailing hand, the lapses of judgment and speed, for example), Wiener envisioned perception and memory in normative terms still deeply inflected by older modern ideas from psychology and pathology. On the other hand, Wiener's concerns with process and probability produced an autonomous form of sense no longer linked to limits. Wiener thus struggled between ideal concepts of how human bodies and minds worked, and his own theories that made humans but one part of a system of equivalencies with machines and other entities.

It is with uncanny prescience, then, that Wiener's autobiography so closely anticipated his own obituary. At Wiener's death, his major interlocutor, and the other central figure to propagating cybernetics as a methodology and a theory was the fellow MIT-based psychiatrist and neural net pioneer, Warren McCulloch, who would write,

Written in an undated, and perhaps never published, eulogy McCulloch attempted to summate the salient elements of his relationship to Wiener. In his account, McCulloch recruits Wiener's memory to substantiate a vision of the future described here as one where technology, probability (teleology), and communication would merge to rethink the ‘working of the brain’ and the senses (hearing, sound, Bell labs) through ‘general theories’ of statistical mechanics and filters.

For both men, the application of logical and computational methods to the study of psychology and physiology was central to their research, and the cornerstone of their intellectual relationship. If Wiener recalled seeking active diagrams and simulations rather than representations of a natural world, McCulloch's eulogy repeats this theme on another register, turning the interest in a generic ‘organization’ and diagrams to the reformulation of mind and body.

McCulloch, as a former psychiatrist, also possessed normative ideas of human physiology. He obsessed with pathologies as routes to explanation. He had often thought about psychotic and neurotic patients, the truly very nervous. He spoke of traumatized causalgics, paranoid soldiers, neurotic housewives, and delusional teenagers all in the name of demonstrating how circuits make the mind both mad, and organized, with the steadfast interest of rewiring patients in the interest of health (McCulloch 1965, 37).

But McCulloch also marvelled at the autonomous capacity and prodigious liveliness of the very elements of the nervous systems — the neurons — as somewhat wily and unpredictable creatures. He envisioned them as creatures of logic and probability seeming unrelated to normal functioning in people. He talked about neurons as ‘ghosts’, ‘little plants’, ² ‘telegraphic relays’, and ‘machines that think and want’ (McCulloch 1965, 38). The transformation of pathologies into circuits, which McCulloch pioneered, that could be moved from within people to the construction of other machines departed from any normative or absolute concept of how bodies or nets should work and instead focused on what types of machines, and circuits, could be built, modulated and enhanced. For McCulloch nervous nets were like Wiener's diagrams, capable of producing complexity and organization. The neurons themselves were a new form of performative representation; the literal embodiments of computational logic. The mind was comprised of material processes both philosophical and physiological at once, that could act like plants and telegraphs simultaneously.

This was a radically different concept of perception and mind, one no longer attached to human bodies, and concerned not with limits, but with capacities, circuits, and communication channels — in many different types of machines. As McCulloch never stopped reminding his adherents and reviewers, the question that guided him in all his research, whether on the logic of neural nets or the cognitive capacity of animal optic nerves, was a basic question, ‘what is a number, that a man may know it, and a man that he may know a number?’ Put otherwise, if Wiener was guided by ideas of truth and norm, of finding diagrams still linked to truths in nature, then McCulloch asked more philosophical questions about finding the logical building blocks that precede even ‘man’; and could produce a computational logic to describe, and perhaps make, thinking beings.

For Wiener, human bodies and subjects were still central, but for McCulloch ‘man’ was the result of logical processes. McCulloch engaged in his language in an ‘experimental epistemology’, whose goal was not to prove the scientific method sound, but to ask how epistemology (knowledge) itself, came into being. ‘There is no theory we may hold and no observation we can make that will retain so much as its old defective reference to the facts if the net be altered’, he announced with Walter Pitts in the famous article on the logic of the neural network McCulloch 1965, 37). The pair announced that it is the organization of the circuit that organizes knowledge, not some interminable external truth or higher consciousness. If what we know is the result of how we are wired, then it can also be changed if we change the organization of the net.

For McCulloch it was challenging the limits of knowledge, and proving failures in reason that interested him, whereas Wiener continually worried about scientific authority and authenticity. Both men harboured concepts of normal bodies and physiological functions, but counter to the narratives of reductive cybernetic logics they also created concepts of perception, embodiment, and mind no longer linked to single norms or even human bodies, but refigured as circuits and communication channels (Galison 1994; Hayles 1999; Golumbia 2009; Heims 1991; Edwards 1997).

In this paper, the relationship between the ideas of McCulloch and Wiener is examined to speculate on a mid-century shift in ideas of sense with implications for how we understand, define, and engineer human perception and cognition in the present. In their work, we can view a complicated and often contradictory movement between normative and modern conceptions of sense and cognition, and another attitude to sense perception as interactive, networked, and algorithmically governed. As I hope to show, it is precisely this relationship between the normative and the networked conception of perception that produces the infrastructure for our contemporary on-going obsessions with intensifying the proliferation of screens, media, and interfaces in our contemporary world.

‘Behaviour, Purpose, Teleology’

Any account of McCulloch and Wiener must start with mention of a famous paper ‘Behavior, Purpose, Teleology’ that was written during the war as a result of Wiener's work on servo-mechanisms and anti-aircraft defence. McCulloch regularly recalled having been impressed by the piece, and that it inspired his later work on mind and sense.

Although mathematical in approach, Wiener insisted on framing his work as centrally concerned with human sense and disability. He reminisced, for example, about learning at the age of eight about the ‘wonderful blind and deaf student — Helen Keller’, and continued to insist throughout his career on the centrality of sound engineering, telephony, and sense to the practice of computation (Wiener would in fact correspond with Keller and bring her to MIT). ³ Particularly valuable to Wiener was the possibility of using communicative mechanisms for medical and humane purposes, particularly, the ‘prosthesis and replacement of human functions which have been lost or weakened in certain unfortunate individuals’ (Mills 2011, 87). Technology, in this account would return a universal human to the world, making everyone equal.

Throughout his two-book autobiography, and in many of his writings, Wiener invokes his deviation from normative concepts of body, thought, and mind as the substrate for his intellect and his decisions to animate mechanisms. His own form of non-normalcy was, in his accounts, the very infrastructure for an alternative intellect that put Wiener at odds with a certain form of empiricism and tinkering. He was, he recalled, a boy with a small machine shop at his disposal, full of ‘magic lanterns’, ‘microscopes’, a ‘megaphone’ and other optical and sensory playthings, but he did not have the ‘mechanical skill’ to transform these instruments through his hands (Wiener 1964, 75). This obsession with sensory toys, but mechanical ineptitude guided Wiener's approach. He was never an experimentalist, preferring math and theory, and he was deeply interested in thinking about human perception in terms of computing, media, and communication machines.

It was in fact a consciousness of difference and race that initiated Wiener's desire to apply mathematical solutions to thinking about the behaviour of minds and machines. Wiener argued that his ‘knowledge that we [Jews] were being threatened with extermination…’ (Wiener 1967, 163) along with the spread of Nazi Antisemitism was a strong motive for his entry into the war effort in 1940, even before the official declaration of war by the United States.

Contemplating the speed of aerial warfare, seemingly driven to erase these segregations and differentiations between races and nations, Wiener, working at the Radiation Lab at MIT with electrical engineer Julian Bigelow, and the physiologist Arturo Rosenblueth, reformulated the problem of shooting down planes in the terms of communication — between an airplane pilot and the anti-aircraft gun. These researchers postulated that, under stress, airplane pilots would act repetitively, and therefore possessed patterned behaviours amenable to mathematical modelling and analysis (Bigelow 1943).

Wiener, Rosenblueth, and Bigelow's reformulation relied on reconsidering defence as a question of sense perception and measure. Wiener obsessed, in attempting to find the repetitive patterns of behaviour demonstrated by an aircraft pilot, about figuring out exactly how much data was needed to make an estimate, and when to take measures of the plane's location. In summary, considering that behaviours are continuous but measurement by the anti-aircraft gun(ner) is discrete how does one reconcile the two? A problem that anticipates a more difficult question coming to haunt computing between the discrete nature of logical and algorithmic representation where commands must be broken down definitively and answers must be absolutely ‘true’ or ‘false’, and the desire to represent and compute continuous phenomena.

Wiener and his colleagues reframed this tension between the discrete and continuous in terms of speed and accuracy (because this is a limited time activity happening at a particular velocity). Taking measurements on different subjects pretending to pilot a plane with a light beam through certain obstacles they came to realize that while different ‘pilots’ behaved differently, each individual pilot behaved similarly to themselves. Closing the system into this individuated model of self reference allowed the team to reconceive the object and method of measure, ‘The more we studied the problem [of shooting down planes] the more we became convinced that…the only question was: What did we mean by the best predictory? If errors of inaccuracy and errors of hypersensitivity always seemed to be in opposite directions, on what basis could we make a compromise between the two’ (Wiener 1956, 212). Their solution was to rely on the difference, the mean square of errors between these two forms of errors, and reduce one category to the other (sensitivity into inaccuracy) through a fairly simple statistical calculation of taking the square of the error of prediction at each time (the difference between the predicted and true value) and making the goal of their program to minimize this number.

Essentially they decided that what needed to be measured and represented was the ratio of errors inside a closed system, rather then comparing the actions of pilots, planes, or guns to absolute or ideal scenarios. The behaviour of systems could be assessed in a manner behavioural, personal, and without absolute baselines.

The result ended up being less about the actual movements and measures of anti-aircraft defence and more about predicting the future of a message arriving despite noise (artifactual data) in the background. Thus, Wiener's solution of treating the problem behaviourally and constraining definitions of error, effect, and action could also be applied to radar work.

As Wiener explained it, telephony had been produced as a result of constraints, the amplitude and frequency of a wave, which filters isolate within particular parameters related to human physiology. But radar, he argued, ‘appeals to the eye’. The eye, according to Wiener, who subscribed to a hierarchy of senses, along with his conceptions of able and disabled bodies, was more ‘sensitive’ and was capable of detecting phase errors as well as amplitude errors in input signals. To deal with radar required, for Wiener (and it is important to note his actual engagement with the engineering of any of these systems was extremely limited), not merely constraining and filtering the range of frequencies and amplitudes of signals that would be received by set parameters linked to human abilities, but comparing different signals to one another. For Wiener, the relationship between the eye and the ear, therefore, became a relationship between two forms of error to be dealt with through methods in integral calculus which Wiener had been working on already before the war with Eberhard Hopf (Wiener 1956, 212–214).

Wiener and his colleagues thus developed three critical elements in dealing with the problem of shooting down airplanes. The first element was to define the pilot and the plane behaviourally, as black boxes, that could be treated similarly through the logic of mathematics and statistics. Behaviourism also offered a form of measure in assessing only historical actions not future intentions or unarticulated desires.

It was, however, a modified statistical or probabilistic behaviourism grounded in ideas of gestalt psychology, that Wiener suggested. According to him, gestalt was also a probabilistic science. For him, the patterns, the prägnanz or ‘good figure’, of gestalt were models for communication. Wiener viewed gestalt forms as channels or architectures that described both the most reduced set of relations to organize experience, and were effects, that like his ideas of teleology, nonlinearity and physics, exceeded its discrete elements. Between the micro and macro scale in gestalt there is a disjuncture. Gestalt psychologists arguably, according to Wiener, also possessed concerns analogous to thermodynamics and statistical mechanics dealing with how different scales of a system act differently, so wholes cannot be directly postulated from the separate pieces. This idea of a prediction that is scaleable and nonlinear shaped Wiener's concept of how to use behavioural data in the search for these gestalt patterns. ⁴

The second aspect of the study was to focus not on discrete actions but on patterns of errors and corrections, on the interaction, or feedback, between the entities in the system. And the third element was to focus on prediction, or the future actions on the basis of past activity. These three elements are seemingly unimportant, and perhaps not even innovative when compared with what both telephony and psychophysics had already accomplished in previous decades, unless one contemplates the time of feedback in the model, and the fact that Wiener relocated older running concerns with statistics, noise, and probabilities into new-found problems of sensing, measurement, and communication. Holding together problems of logic with those of integral calculus, the precybernetic formulation of behaviour, purpose, and teleological action paved the way for recombining seemingly incommensurable logics of discrete decisions and continuous phenomena.

Already, therefore, within the early work on servo-mechanisms, similar methods were applied to different senses (vision and sound), and concepts of measure became self-referential and predictive rather then absolute and normative. Wiener, and his colleagues, invested as they were in prediction, ceased to be interested in the ontology of objects, or precise description of the world, and focused instead on extracting those patterns and forms that could produce effects like gestalt forms.

In seeking to create homogeny between persons, Wiener ended up popularizing a technique for compressing cognition with perception and automating both in a manner that separated from the specific vagaries of human difference, and became an autonomous process now affiliated with machines.

Cybernetics

In 1948, Wiener summarized concepts developed in the war in a widely read book dedicated to advancing a fantasized program for scientific and technical research defined by an invented term, ‘cybernetics’. The very definition of cybernetics already assumes a complex relationship to temporality and history; bridging the past with an obsessive interest in prediction, the future, and the virtual. Cybernetics is, in Wiener's words, an ‘emergent term’ derived from the Greek kubernetes, or ‘steersman’, the same Greek word from which we eventually also derived the word ‘governor’ (Wiener 1967). As the etymology of the word suggests, cybernetics is thus a science of control or prediction of future action. In further adjoining control with communication, it is an endeavour that hopes to tame these futures through the sending of messages.

For Warren McCulloch, who met Wiener through Arturo Rosenblueth in 1941–42, these concepts were instrumental to his reformulation of how to treat the nervous system. For a decade between the mid-1940s and early 1950s, the two men shared ideas, and students, in a curious barter of young men that included figures like logician Walter Pitts and neurophysiologist and psychiatrist Jerome Lettvin both of whom were mentored and offered laboratory space, money, and assistance through McCulloch and Wiener's affiliations at MIT and earlier at the University of Illinois–Urbana–Champaign, and through their vast network of acquaintances inside and outside the University. Trading men, materials, and ideas Wiener and McCulloch produced a space for investigation linking mind, perception, and communication.

The two men also participated in a conference on Cerebral Inhibition in 1942 at the behest of Frank Fremont-Smith later chief administrator of the Macy Foundation, bringing biologists and engineers together in an event that would lead to the infamous Macy Conferences. Both men would go on to participate in the conferences on Circular Causal and Feedback Mechanisms in Biological and Social Systems held in New York throughout the late 1940s and early 1950s (McCulloch would chair them) later known as the Macy Conferences on Cybernetics. McCulloch later came to MIT, following his young prodigy Walter Pitts, at Wiener's invitation, in 1952. McCulloch would remain at that institution for the rest of his life working, along with Lettvin and others, at the Research Laboratory for Electronics, an offshoot of the Radiation Lab applying work in communications, computation, and signal processing to problems ranging from sound recording to neurophysiology (Arbib 2000, McCulloch 1974).

The paper ‘Behaviour, Teleology, and Purpose’ was the key, according to McCulloch's memoirs, allowing him to connect the actions of animal nerves with mathematical logic and machines. McCulloch, and his collaborator the polymath Walter Pitts, in a famous paper introducing the concept of the neural net, ‘A Logical Calculus of the Ideas Immanent in Nervous Activity’ appearing in 1943, appropriated both the temporal feature of Wiener, Rosenbleuth and Bigelow's predictive approach and their behavioural equivalence between biological organisms and machines. In the paper, McCulloch and Pitts produced an account of mind as comprised of neurons that act like logic gates, whose impulse to fire or not, can be viewed as a true–false statement, and equated with Turing machines. McCulloch's later work was also greatly influenced by ideas of feedback which he correlated with reverberations in circuits that caused pathologies or normal functioning and could offer the teleological and nonlinear temporal direction of neuronal behaviour a dynamic capacity to change. ⁵

Materializing hearing

Produced out of an interest in isolating gestalt forms and transforming ideas of sense and measure in dealing with aircraft, it seems only logical that this reframed concept of humans and machines might be applied to the study of sense; assuming a world of informational density, perception came to be redefined as an analytic process of extracting patterns from data fields.

Although cyberneticians were invested in facilitating perfect, and rapid, communicative exchanges, the residue of these interactions was a vast cumulative space of data and information. In order to predict from past behaviour, some form of recording, storing and retrieving information was necessary. This same process could become the terminal failure point in perfect transmission of a communicative message; too much noise would interfere with the signal. Systems not capable of erasing the excesses of stored material or sufficiently ‘damping’ or time lagging their impulses would be prone to error with error always being a failure to effectively transmit a message (Wiener 1961, 122). Within closed systems, too much information could overwhelm the system's stability. In short, faced with an excess of information/stimuli the system may lose its capacity to manage and respond. A continually nagging ‘problem’ of overabundant information that is the preoccupying obsession of engineering in distributed and networked systems.

The selection of information became a pressing problem. Failure to adequately sort and sift through stimulus, and permit too much ‘noise’, according to Wiener, into a system, results in the loss of homeostasis, and excesses oscillation and instability. Cyberneticians used, for example, those models of servo-mechanisms responding too rapidly to a moving target, or a ‘functional disease’ such as mental disorders or blood clotting to make this point (Wiener 1976, 396–7). For Wiener, such losses of stability were possibilities located not only within the human subject, but operating at the level of large systems, particularly nations at the brink of atomic disaster (Heims 1980, 304–329). For cyberneticians, processes of perception were scaleable and mobile.

In cybernetics, avoiding transmission failure but still producing viable feedback systems mandated, therefore, not only storage, but a process of selection by which only that information necessary for response would be stored. As Wiener argued in 1951, both hearing and vision must be treated in terms of finding gestalt relations, not causal links between stimulus and effect. One must, he argued ‘treat scanning not so much as of a group as of a group of transformations of energy…in this way [a human or machine] can recognize a figure independently of its orientation’. ⁶ Abstraction could facilitate transmission. Wiener became obsessed by nonconscious acts of abstraction that permit negative feedback to commence without error. Perception and cognition were rendered part of the same channel whose function is to extract relationships between data points in advance of conscious representation of the experience.

Already, in 1949, Wiener had collaborated, for example, on a study of perception with electrical engineering professor and presidential science adviser, Jerome Wiesner, and a post-doc in Wiesner's lab, Leon Levine, in the interest of developing these ideas. ⁷ Merging his concerns with process and prediction with his earlier conception of pathology and abnormality, Wiener began to apply ideas of active perception to sensory prosthesis.

The ideas of the group regarding perception, like Wiener's conceptions of eyes and radar in the use of filters and communication in anti-aircraft servo-mechanism research, was not sense specific. In this early paper, Wiener, Levine, and Wiesner argued that (and particularly after the war considering the injuries) sensory prosthesis was a central concern for both medicine and computing. To be able to replace a sense or at least compensate for it appeared an important goal in improving human life, and in using technology to augment human perception.

Although the idea of substituting touch for the loss of vision or hearing had been in play for some time — with techniques like Braille — Wiener and his colleagues were convinced they had another approach. As Wiener later put it, ‘hearing is not only a sense of communication, but sense of communication that receives its chief use in establishing rapport with other individuals. It is also a sense corresponding to certain communicative activities on our part, namely those of speech’. Wiener argued that deafness is related to voice for that reason that, ‘for the purpose of sensory prosthesis, therefore, we must consider the entire speech process as a unit’. ⁸ Instead of approaching the senses as discrete or linked to particular forms of stimulus the group instead desired to focus on the idea of sense as a relationship between different forms of communication. As the group put it, ‘it is most important…that the patient should not separate the problem of active communications such as speech, from passive communication such as hearing’. ⁹ In their approach, linked to the production of the Vocoder (a machine for inputting and resynthesizing speech) at Bell Labs, they focused on the capacity to remediate the stimulus between mediums and senses. The group's innovation was to treat hearing like a channel with an architecture that related one form of input to another, and whose capacities could be modified and enhanced through the recoding and reorganization of stimulus patterns.

They envisioned and built (and how well the machine worked is questionable) ¹⁰ an apparatus with filters that would take sound waves and turn them not into direct representations of words or letters or even phonemes, but rather would seek the cadencing, rhythm, and relationships between sounds and offer the pattern that emerged from speech. The patient would feel as a vibration these patterns. This concept was based on previous work at Bell labs, where the Vocoder was also built on a principle that sounds audible to humans were only within certain ranges and could be identified and categorized and therefore translated into electrical signals and back into speech despite the limited amount of information passed through the channel.

In the hearing glove design, the patient would feel these patterns as a vibration by inserting their fingers into a panel with sockets. Wiener's equations for the filter applied statistical approaches to the electrical impulses to create outputs based on the comparison between the signal and the desired amount of noise the signal should contain. The output therefore that the deaf person would feel would be this differential between signals, and not just the signal itself. Language, therefore, could be converted to touch on the basis of seeking redundancies. What the individual thus would touch would be a ‘pattern of stimulation’ not a specific stimulus. The device was labelled a ‘hearing glove’. ¹¹

What did the group intend in arguing that pattern replaces discrete sensations? They hoped that instead of replicating other approaches that also transferred one sense onto the other to provide a prosthesis, such as the direct one-to-one correspondence of Braille or the letters on a screen, where a letter is equivalent to a letter, the deaf person might be allowed to sense the relationship, through touch, of what one sound is like compared with another, instead of simply learning each sound discretely. If the deaf individual could grasp the ‘gestalt’, in Wiener's parlance, of language they might be able to more fully communicate. Wiener wrote that what is important is that ‘the patient should not separate the problem of active communication, such as speech, from that of passive communication, such as hearing…’ implying that what the patient must do is maintain the relationship between different forms of communication — speaking and hearing — rather then discretely sensing one or the other. In order to do so, Wiener argued, the group had to ‘transfer part of the cortical function of the ordinary hearing mechanism to an electrical system outside the body’. ¹² Wiener implied that the stimulus offered through this apparatus would already be an analysis, a cognitive function, that sought patterns between data points, rather then offered discrete pieces of information. Touch, and in the future Wiener intimated by way of sight, the pitch, the structures and grammars of lingual communication, and perhaps most importantly the sentiments and affects of language would be revealed. Language could be made haptic and comprehensible through cadencing and pattern seeking. ¹³ More importantly, the idea of sense here was as a form of processing where signals had to be translated before ever reaching the brain.

The subject of such an apparatus is imagined as sensing interactions, perhaps even meaning and syntax, at the very point of stimulation. ¹⁴ Perception became equivalent to cognition, ‘cortical function’ in Wiener's words, and both rendered material as a channel that could be automated, extracted, reproduced, and remodelled and as part of a process by which data was being analyzed and manipulated from the moment of input. Language turned into affective pattern. The sensorium extended into the world and the boundary between sensation, transmission, and analysis reconfigured. It is necessary Wiener later wrote in discussing sensory prosthesis more broadly, ‘to equip him [a blind patient for example] not only with visual receptors but with an artificial visual cortex…’ which Wiener envisioned as being part of a single channel between active and passive sense, and between the cognitive processing and stimuli (Wiener 1961, 142).

Wiesner, Wiener, and Levine were thus focused on remediating the senses, not in resynthesizing voice. Their project differentiated from the Vocoder in attempting not to resimulate hearing, but to shift the function of hearing onto other senses, and to export, in Wiener's estimation, cognitive functions onto the external technology. They sought to train perception, not merely to allow deaf individuals to understand speech, but to speak differently; to ‘enable the deaf to appreciate the sounds of conversation through tactile sensations’. ¹⁵ Appreciation is an aesthetic category of sensory organization that exceeds mere function. The project wavered in conception between serving as a functional tool to standardize behaviour and facilitate efficient behaviour understood as basic comprehension of language and as a projective technique to produce sentiment and sensation around the experience of hearing and speaking.

This imagination of prosthetic technology predicated on information inundation and seeking patterns, posed both reductive and complex imaginaries of the sensorium. The group's work opened a frontier to think sense and computation together, making perception an active entity capable of finding relations and extracting patterns from the point of sensory input. For cyberneticians and related communication, cognitive, and neuroscientists the possibility was opened that processing was distributed and began at the point of stimulation, not merely at the arrival of the signal to the brain.

However, Wiener himself expressed contrary attitudes to this technology. He often viewed the hearing glove as a mimetic device, whose purpose was to mime ‘normal’ hearing and speaking. Positing a hierarchical relationship between senses — vision contains more information than touch — and assuming certain necessary functions resulting from the least amount of information necessary to produce language, he arguably possessed a reductive and normative vision of sense. Wiener's notions of language, reduced (or eliminated) meaning to the study of redundancy and statistical probabilities such as in the mathematical theory of communication, and can be understood as compatible with the industrialization of language as pioneered at Bell Labs. Language, or more appropriately hearing and speech, stripped to the terms of efficiency and mathematical representation was also sense assigned numeric values and monetized ¹⁶ (Mills 2011).

However, there is also a crucial corrective here. The group's interest in feedback between the senses and between speech and hearing complicated the idea of a basic signal being sent between two points. In theorizing his relationship to the concepts of the hearing glove, Wiener focused on the transferability of the very process of sense perception between senses and between different sites. This research was about producing computational processes to model human, and perhaps machine perception, and ultimately something more, ‘an appreciation’ of sound. In the course of this work, the aspiration to model sense perception shifted from prosthetic imaginaries that simulate ‘normal’ functioning to the possibility of an augmented, perhaps machinic, sensorium. If, Wiener later postulated, the right mathematical approach might be taken, ‘the machine might quite well be made to furnish us with the perception of sorts of gestalt which are not in our nervous system. This, to my mind is a much more intelligent and interesting use of the machine than the mere computation of numerical results in cases to bulky to handle’. ¹⁷ Assuming the capacity to isolate the perceptual cognitive process, Wiener fantasized a form of autonomous perception capable of producing new types of experience. Estranged from the engineering, or even from the direct contact with deaf persons, Wiener was obsessed with the virtues and potentials of materializing processes through mathematical logics. It was this material abstraction, literally phenomenal, that Wiener cared about, not its specific application to business interests as a commodity or medical concerns.

For Wiener, definitions of information competed between strict mathematical understandings, and more epistemological and phenomenal comprehensions of the same terms. In his world, the transmission of information was three-fold in contour (as it was for Shannon) between information as a signal-to-noise ratio, information as a statistical measure of the amount of entropy or uncertainty in a system, and information as a nonlinear and nondeterministic relationship between different scales of the system, the micro and the macro. For Wiener, on one hand, information was about reducing noise and extracting essential patterns, but, on the other hand, this noise reduction rapidly became a question of a broader issue of ‘choosing’ patterns, and by way of feedback of perhaps ‘choosing’ how one might wish to hear, or reformulating the relationships between different senses (Terranova 2004, 9).

Wiener's attitude to sense could thus be viewed as underpinning media-theorist Friedrich Kittler's argument in ‘Gramophone, Film, Typewriter’ about the interchangeability of senses in digital media, ‘sound and image, voice and text are reduced to surface effects, known to consumers as an interface’ (Kittler 1999, 1); where the body and sensory differentiation are merely aesthetic supplements offered to entertain humans with concepts of subjective differentiation.

However, Wiener's concerns coming from his own background still identified with a very different media system, and conception of human subjectivity, one that viewed pathology as a route to normality and capacity. Wiener's attitude to sense arguably still aligned itself to nineteenth and earlier twentieth century concerns with the limits and failings of human perception, and the production of media told based on theories of a mediating and fallible body. As Jonathan Crary has shown in his wide-ranging study of observation in the early-nineteenth century, physiological optics embodied in the technology of the stereoscope, ‘displaced models of vision that had been predicated on the self-presence of the world to an observer and on the instantaneity and a-temporal nature of perception’ (Crary 1999, 5). The human sensorium was found to be full of lapses, lags, and temporal delays, the human eye, ear, and touch was no longer equated with knowledge, and these incapacities became the very models for new media technologies and architectures such as optical toys and later photography, concert halls, and phonographs. ¹⁸ At the basis of this model were concepts of ideal or normal human capabilities and bodies that could be enhanced or replaced by machines. As Georges Canguilhem famously wrote, it is in the nineteenth century when the pathological ceased to be ‘qualitatively opposed [to the healthy]’ and instead became part of ‘forces joined in battle…to gain more knowledge for more effective action’ (Canguilhem 1989, 42). Wiener's own concerns with making all subjects equal through prosthesis can arguably be part of a force for more effective action, leading here, not to disciplinary technologies, but an overwhelming faith in the capacity of mediation to recuperate subjectivity. In Wiener's work his normative attitudes productively competed with his own computational understanding of sense as a material process that could be abstracted, recombined, and shifted between different registers and machines — animal or mechanical.

The autonomy of the eye

These concepts of sense and perception did not end with the production of prosthetic instruments for human beings. Wiener's concepts of perception and processing were highly influenced by his relationship to McCulloch, and reflected a broader shift in the emerging cognitive and neurosciences in approaches to mind and sense. Although Wiener's influence on McCulloch in developing neural nets is well documented, here I want to examine McCulloch's extension of his ideas about cognition into his treatment of sensory perception (Kay 2001; Hayles 1999, 59,66,80; Arbib 2000; Abraham 2002). Although McCulloch and Wiener stopped speaking in the early 1950s their relationship continued to inform cybernetic attitudes to sense. ¹⁹ The virtual and intellectual relationship inhered inside of cybernetic attitudes to sense perception.

Vision was another prominent example of this reworking of perception into a modellable and technical project. In vision, the eye would take up another function as a machine for abstracting the world — a black box. Vision, that sense that collects so much information, must, in the minds of cyberneticians, have some sort of dampening or straining process to facilitate the flow of information toward a more abstract state where what is stored is not everything seen. This process was imagined as a series of steps, in which each step of the process brings visual information ‘one step nearer to the form in which it is used and is preserved in memory’. Wiener posited that the most ‘plausible’ explanation of vision is as a process where outlines are emphasized. The eye, starting with the retina, must begin filtering the information, otherwise it loses its ability to transmit the stimulus onward, being bombarded as it is with constant stimulus. Wiener argued that alterations in ‘storage elements’ are necessary for transmission (Wiener 1961, 125,134). For Wiener, the eye constituted of processes working to filter and arrange information. Information entering the eye is at every moment both an index and a command for future action.

The experiments on frog vision done by McCulloch, working with the psychiatrist Jerome Lettvin, Humberto Maturana, and Walter Pitts, offers clear evidence of this new attitude to perception as a material process of abstraction. The foundation of Wiener's conception of vision, this was one of the more famous experiments in the annals of computer science and neurophysiology — ‘What the Frog's Eye Tells the Frog's Brain’. The title describing an eye speaking autonomously to the brain should already alert us to a transformation in the function and structure of sense. In this paper, McCulloch, Lettvin, Maturana and Pitts opened with a seemingly simple question — assuming a world of informatic overload how can we assume that all processing occurs in the brain? Their answer was revolutionary from the vantage point of history — it does not. Cognition, they argued, does not happen in a centralized location (the brain). They argued that the management of data emerged through the networked organization of the sensation–perception–cognition system.

Their initial logic was critical. They hypothesized that the optic nerve does not transmit every piece of data (light) it contacts. Such an assumption reconfigured their experimental practice. Rather then test discrete stimuli, the researchers exposed an optic nerve to variations in light. They created a test environment where a series of myleinated and unmyelinated fibres in the intact optic nerve were exposed to variations in light stimuli. Working on these moving edge detectors the team discovered a fibre that: ‘…responds best when a dark object, smaller than a receptive field enters that field, stops, and moves about intermittently thereafter. From the measurement of subsequent electronic impulse activity, they wished ‘to discover what common features are abstracted by whatever groups of fibres we could find in the optic nerve’. What they discovered was that when the eye was exposed to stimuli simulating a moving insect or an enemy (stimuli that moved or changed from light to dark) the electrical impulse given off changed before ever arriving at the brain; demonstrating that the eye, and it was an autonomous optic nerve on a dish, was capable of making decisions between such binaries as prey or enemy and non-prey and non-enemy (Lettvin, Maturana, and McCulloch 1959, 251).

They concluded that ‘the eye speaks to the brain in a language already highly organized and interpreted, instead of transmitting some more or less accurate copy of the distribution of light on the receptors’ (Lettvin, Maturana, and McCulloch 1959, 251). Their fellow colleague Michael Arbib summarized this finding as proof that the frog's eye could deal with universals like ‘prey’ and ‘enemy’ (Arbib 1964, 32–33). In summary, eyes were found to be Turing Machines. Perception, therefore, became the same as cognition; as autonomous entities — such as eyes — began the process of abstracting and processing information. This analysis opened the possibility that perception as an autonomous process could be technologically replicated; a conclusion further substantiated by the fact that this research continues to underpin much computer science work on vision. The emerging post-war neurosciences did not understand the image as a representation being transmitted and then translated upon arrival in the brain, but rather redefined vision as encompassing the entire relationship structuring the act of observation; a communication channel.

These neurosciences thus produced a flexible barrier between the realms of stimulus, the form of the data, the organs of reception, and the site of processing. Although such subjective perception had been found in nineteenth-century physiology and psychology in the work of individuals such as Herman Helmholtz, Jan Purkinje, Charles Wheatstone, and David Brewster on the physiology of hearing and optics, stroboscopic effects, and the afterimage, it was now no longer a problem for scientific objectivity and knowledge, but positively embraced for technological potential in neural nets (Doane 2002, 74–76; Crary 1990; Canales 2009). The very nerves, extracted from any particular body, are capable of processing and analyzing data. The act of processing information and the act of analyzing it became the same, and the possibility emerged that this decontextualized seeing process could be rebuilt in other locations.

This is not an insignificant experiment within the histories of visuality. The cybernetic model of perception desired a purely technical and autonomous eye. If one wished to see an insect then one builds a frog's eye, if one wants to see a missile silo, perhaps it is a different form. Vision circulated. There was no single norm for vision. The ideal of a singular or objective form of vision was replaced by a fantasy of effectiveness or affect serving particular functions. For us, what is critical is that it was a lack of concern for static ontology that facilitated the shift to conceiving sense perception as an interactive process and a material technology. This was an eye extended into the body and out into the world, a vision that was material and could now act on its own — flies eaten and air planes blown up, for example — a networked cognition beyond the brain and a new way to understand the differences between subjects and objects. There was no ontological stability in this cybernetic formulation of visuality, no stable enemies or preys.

The experiment on vision was part of a longer cybernetic effort to extract perception as a generic algorithmic process that could be technically reproduced, preferably for machines. The focus on perception as a matter of pattern-seeking made all the senses subservient to the same process. As McCulloch confidently stated in a letter supporting Jerome Lettvin for the Guggenheim Foundation, ‘What we seek to understand ultimately is what Kant called the transcendental unity of apperception. What we find is that such perception is computed in our receptors themselves and comes in some kind of pulse interval modulation, predigested for the brain. What these men have so far achieved is clearly the largest step yet made in this direction’. ²⁰ Cyberneticians, therefore, did not maintain clear boundaries between the senses. Nor were McCulloch and his team seeking a truth of the body, or a prosthetic capacity. Rather they sought to ask fundamental philosophical questions about universal processes with empirical research. They arguably discovered a nervous system open to the world, capable of processing already at the moment of stimulus.

If perception and mediation throughout the nineteenth and twentieth century was an on-going and vexing site of interest, speculation, investigation, and problematization across the social field from philosophy to human sciences, to cinema — cybernetics succeeded in suppressing these previous questions in favour of a modular and literally technical approach. One can say that we really rarely speak of perception any more except as a medical technology. Perception was temporally marked by past elements, it was a space between information reception, recollection, and reaction, but this was the opening for the possibility of processing, not a vexing problem of the historical record.

Conclusion

There was, however, a curious indexical and temporal nature to this ability to materialize perception. In order to focus on how eyes ‘speak’, and make speech, even semantics, audible to the brain demanded a lack of regard, or perhaps an automation of, recording and an assumption of an informatically dense world. The impossibility of ever accessing and processing sensory data was no longer in this experiment the question. Instead, the problem turned to how to manage and utilize data overflow? How to operationalize this vast and ultimately unknowable data field?

Here tensions emerged, both between McCulloch and Wiener, and for each of the men within their own work. These tensions between authority, authenticity, and operatability continue to trouble our contemporary ideas of self, sense, and computation.

For Wiener, despite the seeming feature that cybernetics posed an end to ontological stability of machines or animals, and produced a crisis for certain forms of observation and perception, he continued to maintain a faith in a positivistic science. For example, Wiener argued that ‘bluffing’, as in opponents lying to each other in a game, was not ‘natural’. Nature, he argued, is not ‘deliberate’ in defeating scientists or withholding information, and the scientist's vision should be ‘naïve’ to deal with her (Wiener 1956, 212). Science for Wiener must seek absolute truths, even if obfuscated, and nature possessed such features. For Wiener, technical mediation and manipulation still lay in the province of a ‘Manichean’ and deliberate evil as opposed to a natural and ‘Augustinian’ naïve evil of nature. Behind this discourse lay a faith in an absolute reality to still be unearthed, and an older belief in forms of objectivity inherited from earlier eras; a desire perhaps to return to tales of exploration rather then diagrams of complexity and organization. Accompanying this faith in a modern order of the world, Wiener also evinced a belief in a certain normative body, even as he, himself, eroded this concept.

McCulloch on the other hand seemed to love to dally with the mystical, non-natural, and false. McCulloch and Pitts, for example, end the piece on the calculus of the neural net with an astonishing statement concerning scientific claims, ‘Thus [this research proves] that our knowledge of the world, including ourselves, is incomplete as to space and indefinite as to time. This ignorance, implicit in all our brains, is the counterpart of the abstraction which renders our knowledge useful’ (McCulloch, 1965: 34). This ‘ignorance’ or subjective quality of all cognition was now the ‘abstraction’ that produces ‘use’. Subjective perception equated with technological potential without concern for mediation, and efficacy replacing the concept of an absolute reality as the measure of truth. McCulloch not only takes a non-Cartesian perspective, he resolutely declared any split between the mind and the body or reality and cognition undesirable and impossible. Instead of a route from normal to pathological, the work of McCulloch on nets, and then eyes, posited a world of channels capable of enhancement, reformulation, and mobility no longer attached to any normative concept of human sense and bodies.

However, in a strange turn, when McCulloch returned to the work on the frog's visual system soon after, the second study posits a different model. Instead of a cognitive–perceptual channel, the frog's optic nerves were found to be far more static and relaying merely data. One may ask why the sudden revision and return to older models of mind and body? (Maturana 1960).

What are we to make then of this turn to subjectivity as a site of technical possibility? And then the repudiation of this same discovery? Wiener, McCulloch and their many colleagues deliberately seized on older theories of sense, affect, and psychology to produce a new account of interactivity, autonomous processes, and cognitive perception. Balancing between emergent concepts of sense and older ideals of human bodies, the cybernetic reformulation of perception and cognition offers a genealogy of our contemporary discourses that waver between augmentation and simulation, and between reactionary imaginaries of biologically determined subjects and emergent ideals of infinitely modulatable bodies. Counter to most studies of cybernetics, the attitudes forwarded by McCulloch and Wiener were not about a divide between materiality and abstraction (Hayles 1999; Heims 1991; Galison 1994). For both men process was a material thing in the world that could transfer between representation and thing; between diagrams and actions. Both men produced accounts that focused on conceptions of pathology and normality in human bodies, while simultaneously envisioning forms of sense that could be enhanced, channelled and produced not only for humans but also for animals and machines; an attitude perhaps more aligned with Donna Haraway's call for a cyborg prosthesis that is ‘monstrous’ and about ‘connection’ to our tools and to other bodies, and denies Enlightened dreams of ‘wholeness’, alienation, sovereignty, and individualism (Haraway 1991, 180–181). The question, today, then, is no longer about the loss of embodiment, but rather about the arrangements and organization of these monstrous connections.

This cybernetic tension between envisioning sensory infrastructures as subjective and mimetically prosthetic or networked and autonomous continues to serve as the infrastructure for our contemporary drive to increase media penetration and ubiquity; forcing an endless effort to augment a human sensorium normatively understood as lagging behind an information network now lent lively autonomy, cognitive function, and infinite capacity to process data. This turn — towards affect, behaviour, and sensory intelligence — continues to inform, and confound, our contemporary media environments.