Is covert spatial orienting embodied or disembodied cognition? A historical review

Introduction

The phenomenology of attention was well-covered by James (1890) who told us that “everyone knows what attention is . . .” and the concept was assigned great importance for Psychology by Titchener (1908). Unfortunately, interest in attention was, for the first 1/2 of the 20th century, swept aside by the behaviourist objection to mental terms. Donald Hebb (1949) chastised the field about this untoward denial: “Everyone knows that attention and set exist so we had better get the skeleton out of the closet and see what can be done with it.” An early, and the most influential, scholar to follow this advice was Donald Broadbent (1958) who, building on Colin Cherry’s (1953) studies of “listening with one and with two ears” and the information processing revolution spawned by the ideas of Craik (1943) and Shannon (1948 ) and the arrival of the general purpose digital computer, put forward the first comprehensive theory of selective attention (Klein, 1996). Whereas Broadbent’s seminal theory was stimulated by research in the auditory modality, the attention of researchers shifted to the visual modality following seminal studies by Sperling (1960), Posner (1980; Posner et al., 1980), and Treisman and Gelade (1980) and the possibility of computer controlled visual displays.

Although we normally shift our gaze towards the objects and locations that contain or might contain the information that interests us, it has been well-established that gaze direction and the locus of attention can be dissociated. This fact was first demonstrated in a clever 19th-century experiment by Helmholtz (1896/1989) who observed that he could voluntarily selectively attend a region of visual space without directly gazing in its direction. Importantly, this ability is not confined to the artificial laboratory setting where it can be subject to scientific exploration. It is an important ability in combat, sports, and social interactions when one person may not want to betray the focus of their attention to other people.

The distinction between covert and overt orienting was one of two (see Figure 1a) that were highlighted by Michael Posner in his seminal paper, “Orienting of attention,” which was based on his 1978 Bartlett Lecture and later published in Quarterly Journal of Experimental Psychology (QJEP) (Posner, 1980; see, Posner, 2016, for an update). The other distinction concerns whether orienting is controlled bottom-up by the stimuli that impinge on our sensory systems, or top-down by our intentions. This distinction corresponds roughly to that between reflexive and voluntary control and was referred to as exogenous versus endogenous by Posner to reflect control originating outside versus inside the organism, respectively (Figure 1a and b).

OPEN IN VIEWER

Figure 1.

Visual orienting: (a) orienting can be overt as when we move our eyes or head, or covert as when we shift the processing resources without an overt shift (represented here as a metaphorical beam or spotlight). (b) When considered together with two possible modes of control, exogenous and endogenous (as illustrated by the iconograms in the top row), this yields a 2 × 2 matrix of possible visual orienting behaviours as described by Posner (1980). (c, d) Here, overt orienting is separated to distinguish between readiness to make an eye movement (programming) from actually making an eye movement (execution). (c) The oculomotor readiness hypothesis (OMRH) asserts that programming a saccade is the mechanism for attending covertly. The arrow on the left refers to the possibility that this might be true when activation of the oculomotor programme is controlled exogenously; the arrow on the right refers to the possibility that this might be true when activation of the oculomotor programme is controlled endogenously. (d) The arrows refer to the whether, when an eye movement is actually executed (left arrow is for exogenous control; right for endogenous), attention is allocated in advance to the location towards which the saccade is aimed.

In another 1980 paper, Posner et al. (1980) offered a spotlight metaphor for covert orienting, suggesting that “attention can be likened to a spotlight that enhances the efficiency of detection of events within its beam.” Posner and other early investigators of covert orienting (e.g., Jonides, 1981) imagined one “beam” that could be directed differently by endogenous and exogenous controllers. The first suggestion that different “beams” might be involved was provided by Briand and Klein (1987) who asked, “Is Posner’s beam the same as Treisman’s glue.” Their findings (later confirmed by Briand, 1998) that opportunities for illusory conjunctions interacted with exogenous orienting and were additive with endogenous orienting led them to conclude that the answer to their question was “yes” for exogenous and “no” for endogenous orienting. Klein (1994) later explored how endogenous and exogenous orienting interacted with non-spatial expectancies and found the opposite pattern: non-spatial expectancies interacted with endogenous orienting and were additive with exogenous orienting. Importantly, this conceptual double dissociation is about the effect of covert orienting on subsequent responses to targets, and not about how fast, automatic, or voluntary is the method of control. Together with a variety of other differences in the nature of covert orienting when under endogenous and exogenous control (see Klein, 2009; Klein & Shore, 2000, for reviews), it is reasonable, if not necessary, to assume that there are two “beams” of attention: one beam when control is exogenous and a different beam when control is endogenous.

The relation between overt and covert orienting when under endogenous control is at the core of this historical review. In particular, the focus will be on the role of the machinery for overt orienting in the implementation of endogenously controlled shifts of visuospatial attention.

Origins of the oculomotor readiness hypothesis

It was circa 1977 that I first proposed and tested the oculomotor readiness hypothesis (OMRH). The idea and experiments to test it were presented in a talk (entitled: “Does oculomotor readiness mediate cognitive control of visual attention?”) at a meeting of the International Association for the Study of Attention and Performance held on the Princeton campus of the Educational Testing Service in 1978. The written version (Klein, 1980) appeared later in the volume, Attention and Performance VIII, edited by Ray Nickerson.

The OMRH proposal was specifically about endogenous control which was referred to as “cognitive” (to distinguish it from “reflexive” control; see the arrow on the right in panel c of Figure 1). After presenting some behavioural and neuroscientific findings that made the OMRH at least a plausible idea, it was explicitly proposed that

Movements of the mind’s eye might depend on the saccadic eye movement system in the following specific manner: When attention to a particular location is desired, the observer prepares to make an eye movement to that location; the oculomotor readiness, via as yet unknown feedforward pathways, has the effect of enhancing processing in or from sensory pathways dealing with information from the target location. (Klein, 1980, p. 262)

At the time, I did not realise that the implicit roots of this proposal could be found in ideas to which I had been exposed as a student. As I put it in a presentation at a workshop on the topic of this article,

I realize, now, that my mind had already been biased toward a motor theory of attention by charismatic scholars to whose convincing writing I had been previously exposed. My immediate, if implicit, influences were: Hebb’s (1949) ideas about the role of action in perception (e.g., the triangular pattern of eye movements fixating the vertices of a triangle play an important role in perceiving a triangular shape) and the ideas in two important papers from 1967: Festinger et al.’s (1967) “Efference and the experience of conscious perception” which, following in Hebb’s footsteps, proposed a “motor theory of contour perception” and Liberman et al.’s (1967) “motor theory of speech perception.” I have since learned that in psychological science such ideas go back at least to Münsterberg and to Washburn’s “Movement and Mental Imagery” (1916) the subtitle of which advertises that a “motor theory of the complexer mental processes” will be presented. (Klein, 2016)

Building on this long tradition, the current climate of psychological science seems to be enchanted by the idea of embodied cognition:

Embodiment Thesis: Many features of cognition are embodied in that they are deeply dependent upon characteristics of the physical body of an agent, such that the agent’s beyond-the-brain body plays a significant causal role, or a physically constitutive role, in that agent’s cognitive processing. (Stanford Encyclopedia of Philosophy)

In this context, it is worth considering that the OMRH was essentially an embodied cognition proposal: covertly attending in visual space depends on our bodies; on our sensorimotor machinery for overtly attending.

Lakoff and Johnson’s book (2008), “Metaphors we live by,” is considered one of the launching documents for the contemporary interest in embodied cognitiona. ¹ By referring to covert shifts of attention, in his early contributions to the study of covert orienting, metaphorically as “movements of the mind’s eye,” it is reasonable to infer that Jonides (1976, 1980, 1981, 1983) too was implicitly adopting the notion embodied cognition.

Before proceeding to early tests of the OMRH, let us place it along the continuum of possible relations between eye movements and attention, from complete dependence to complete independence, that was provided in Posner’s Bartlett lecture (see Figure 2). Here, Posner outlined a continuum of logically possible relationships between overt and covert attentional mechanisms. The “efference” arrow represents Klein’s OMRH which was highlighted by Posner. The continuum from complete dependence (“identity”) to complete independence might, today, be seen as a continuum from embodied to disembodied cognition. The title of this article highlights this possibility, and Posner’s placement of “efference theory” closer to the “complete dependence” end of the continuum suggests a theory that is embodied.

OPEN IN VIEWER

Figure 2.

“Degree of dependence of eye movements and spatial attention.” The upper part of this figure has been redrawn from Posner (1980). The italicised labels at the bottom have been added to reflect the theme of this review.

Early tests of OMRH

Two distinct predictions follow from the OMRH: readiness or preparation to shift gaze should entail a covert shift of attention, and a covert shift of attention should entail preparation of a saccade. In two experiments, Klein (1980) tested each of these predictions. In one, participants were encouraged to prepare an eye movement to a particular location, and occasionally targets calling for a simple detection response were presented. Assuming participants had prepared the movement, the OMRH requires that attention would be allocated to the to-be-fixated location, and therefore, the targets calling for manual detection responses would be detected faster when presented there. They were not. In the other experiment, each trial began with a central cue informing the participant of the location where targets calling for a simple manual detection response were likely to be presented. Occasionally, a saccade was required instead. Providing evidence that attention had been successfully allocated in response to the central cue, manual detection responses showed the prototypical pattern of faster responses to targets at the cued than at the uncued location. If, as required by the OMRH, allocating attention entails preparing an eye movement, then the occasionally required eye movements in this experiment ought to have been faster when directed to the attended location. They were not. Consequently, Klein (1980) concluded that the OMRH, despite its appeal and plausibility, is incorrect and that covert shifts of attention under voluntary control are not dependent on the eye movement system.

Two early studies of the relation between attention and overt orienting are notable. One of these experiments was reported in the article based on Posner’s Bartlett lecture (Posner, 1980). In a “move” condition, the participant’s task was to generate a saccade towards a peripheral stimulus. Detection probes calling for a speeded manual response were presented at fixation or at the location of the to-be-fixated target. Importantly, these detection probes were much more likely to be presented at the location of the original fixation than at the location, originally in the periphery, that was the destination for the saccade. The results are presented in Figure 3. Well in advance of the eye movement (initiated at around 240 ms), performance at the location of the target (which was about to be fixated) was better than performance at fixation (even though from the point of view of probabilities, attention belonged at fixation). This finding (which is represented by the arrow at the left side of panel d of Figure 1) might be regarded as support for the OMRH, but there are at least two reasons it is not. First, the imperative signal was presented in the periphery at the location to be fixated. Thus, despite the task instruction, control was primarily exogenous and the OMRH proposed in Klein (1980) is about endogenous control. Second, the OMRH is about covert orienting, and in the condition whose findings are illustrated in Figure 3, eye movements were executed on every trial. Thus, even though the shift of attention took place before the saccade was initiated (during its programming), there is no way to know, based on these data alone, that the shift of attention would have taken place had the programmed saccade been cancelled prior to execution.

OPEN IN VIEWER

Figure 3.

Reaction time to detect a probe stimulus that was presented at various times after a peripheral target calling for a saccade. Detection targets were 80% likely to be presented at the location of the original fixation and 20% likely to be presented at the location of the saccade target. The upwards pointing arrow at about 240 ms represents the average initiation time of the saccade. The data points highlighted by the circle show that while the to-be-executed saccade was being programmed, probes presented at the to-be-fixated location were responded to faster than those presented at the currently fixated (and more likely to contain the probe) location. After the eyes arrive at their target, probes are responded to more rapidly at the original fixation even though it is no longer fixated (redrawn from Posner, 1980).

At ~500 ms post stimulus and after the eye movement, performance was better at the original fixation location which was now in the periphery. Posner (1980) inferred that

Attention moves rapidly prior to the eye movement and returns to the original fixation as the fovea settles in at the target. Even with the incentive of a high probability detection stimulus at fixation, subjects do not maintain attention while programming the eye movement. At first this finding may seem to contradict Klein’s rejection of efference theory. Certainly it suggests a non-trivial tie between overt and covert orienting that cannot easily be resisted by the subject. However, we had the distinct impression that we were able to return attention to fixation even as the eye was moving toward the target. (p. 17)

In a follow-up experiment with detection probes delivered while participants were preparing a second eye movement in the same direction as the first one, this impression was confirmed, and Posner concluded, “. . . programming of the eyes is in the direction opposite to the movement of attention. These results agree with Klein’s (1979) ² conclusion and seem fatal to an efference theory” (Posner, 1980, p. 18).

Six years later, Shepherd et al. (1986) conducted an ingenious study that combined endogenous covert and endogenous overt orienting. We will see how this study extended Posner’s demonstration that shifts of attention precede eye movements, from exogenously to endogenously controlled saccades. But there are other discoveries here for which this article is not well-enough recognised. The methods are illustrated in Figure 4a. Key features were the use of arrows that were either informative, uninformative, or counter-informative about the location of an upcoming probe calling for a manual response, and the inclusion of both a fixate and a saccade condition, with the latter being endogenously controlled (in the direction of the arrow).

OPEN IN VIEWER

Figure 4.

(a) Methods and (b) findings from Shepherd et al. (1986): (a) Each trial begins with a fixation followed by an arrow at fixation. In both the fixate and move conditions, the arrow indicated the probable location of a probe that required a speeded manual detection response. In different blocks of trials, the meaning of the cue for the location of the probe was manipulated: the arrow was either informative (80:20), uninformative (50:50), or counter-informative (20:80). On about 20% of trials, no probe was presented. In the fixate condition, responding to probes was the only task. In the move condition, there were two tasks: manual detection responses to the probes and the arrow also indicated towards which peripheral box the participant should make a saccade (reproduced from Shepherd et al., 1986). (b) The probe reaction time results from the two conditions plotted as cuing effects (see text for explanation) as a function of the CTOA between the cues and targets (X-axis) and meaning of the arrow for the location of the probes (legend).

Although their key results were presented in tabular form, here they are represented graphically (see Figure 4b) with an emphasis on difference scores (probe on the opposite minus same side as that indicated by the arrow). Plotted on the left are the cuing effects as a function of cue-target stimulus onset asynchrony (SOA) in the fixate condition:

In the move condition (Figure 4b, right panel), let us focus, initially, on the earlier SOAs when the probe was presented BEFORE the eye movement was initiated (the grey arrow indicates the mean saccadic reaction time (RT)). Here, all of the conditions show an upwards shift in cuing effects which represents a positive effect in the direction of the upcoming eye movement. This suggests that regardless how participants might have been using the arrow to orient attention endogenously a similar effect was contributed by processes associated with the about-to-be-executed eye movement. Importantly, these effects depend critically on the prepared saccade actually being executed, as was later demonstrated by Born et al. (2014) using a stop-signal paradigm.

Challenge and rebuttal

Seven years after Klein (1980) proposed and tested the OMRH, and on the shoulders of an effect they discovered using the Posner cuing paradigm, the meridian effect, Rizzolatti et al. (1987) proposed a “premotor” theory of attention. As does Klein’s OMRH, premotor theory asserts that “Attention is simply deployed to a given point in accordance to the parameters of the [oculomotor]motor program (p. 39).” In their discussion, Rizzolatti et al. correctly noted that “The paper by Klein (1980) has been particularly influential in minimizing the relationship between attention and ocular movement (p. 38).” Unfortunately, in my opinion, they neglected to mention Klein’s (1980) explicit proposal of such a relationship. Seeking to minimise the impact of Klein’s rejection of OMRH for their similar proposal, they generated a cogent criticism of one of his two experiments. Essentially, they suggest that when Klein encouraged his participants to prepare a saccade to a particular location perhaps the saccade was not prepared because “. . . the detection of a stimulus requires orienting of attention and therefore a specific oculomotor program, the best strategy was to wait until the stimulus appeared, and then prepare the motor program in the direction specified by the instructions (p. 39).” Because most trials required an eye movement and all the required eye movements were to a fixed location, Klein simply assumed the required saccade would be prepared. Had his experiment included a measure that could confirm or deny whether saccade preparation had been generated, the Rizzolatti et al. criticism could be rebutted or confirmed. Just such an experiment will be described later. As noted earlier, the OMRH entails two predictions, both of which were tested and disconfirmed by Klein. In the experiment that Rizzolatti et al. did not critique, covert orienting was encouraged and confirmed and yet the predicted benefit for saccades directed to the attended location failed to materialise.

A few years later (in work I presented at the 1992 meeting of Attention and Performance held in Erice, Italy, and published in 1994), the two predictions of the OMRH were retested using improved methods, including modifications stimulated by Rizzolatti et al.’s incomplete critique (Klein & Pontefract, 1994). Importantly, the cues used to either encourage participants to prepare an eye movement or to covertly shift attention were the auditory words “left” and “right” (or “ready” in the neutral conditions).

In one experiment, participants were preparing eye movements which were executed on most trials. Occasionally a visual probe, calling for a manual detection response, was presented at either the location towards which the eye movement was prepared or at the opposite location. The results are shown in Figure 5a. On eye movement trials (open symbols), we can see that the “prepared” eye movement was actually prepared (initiated more rapidly than the unprepared one). If OMRH were correct, then a similar pattern should be seen in manual detection times (filled symbols) because OMRH (like pre-motor theory) assumes that preparation is equivalent to attending. But preparing a directional eye movement had no effect on the time to detect and respond to the manual detection probes.

OPEN IN VIEWER

Figure 5.

Findings from Klein and Pontefract (1994). In two experiments, auditory-verbal cues signalled, for the primary task, the likely required saccadic eye movement (a) or likely the location of a visual target calling for a simple detection response (b). (a) When saccades were the primary task, visual probe stimuli calling for a simple detection response were occasionally presented at locations towards which eye movements were or were not prepared. (b) When detection of visual stimuli was the primary task, saccades were occasionally called for towards or away from the attended location (redrawn from Klein & Pontefract, 1994).

In another experiment, participants were encouraged to attend covertly by the auditory cues which signalled the likely location of the upcoming manual detection targets and occasionally probes (a different auditory voice saying “left” or “right”) were presented calling for a saccadic response. The results are shown in Figure 5b. As revealed by the significant cuing effect for the manual detection task, participants were allocating their visual attention according to the instruction and information conveyed by the cues (solid symbols). If the OMRH were correct, and attending entailed preparing an eye movement towards the attended location, then saccadic RT would follow this same pattern. But eye movement preparation (open symbols) was not observed in the attended direction. On the contrary, a small, but significant, inhibitory effect was observed: eye movements were initiated more slowly in the attended direction. Klein and Pontefract suggested that this small amount of “inhibition” was due to the natural tendency to want to look where one is attending and the participants’ inhibition of this unwanted tendency.

The OMRH is about the relation between overt and covert orienting. In Figure 1c and d, the overt orienting cells have been split according to whether oculomotor behaviour is just prepared or actually executed. When the oculomotor system is simply “ready to go” under endogenous control but no saccade is executed, it appears that the OMRH is wrong. Conversely, attending covertly does not seem to be accomplished by oculomotor preparation. These behavioural findings from my lab have been confirmed by others (e.g., Hunt & Kingstone, 2003). In light of the wide range of “channels” we can “filter by” (to adopt Broadbent’s terminology) including modality, pitch, and colour, none of which could be mediated by motor preparation, it should not be surprising that we can attend to spatial channels without relying on motor programmes. As suggested in a recent paper from the Klein lab (MacLean et al., 2015), it may even be the case that the OMRH is wrong for exogenous control (but see, Smith et al., 2012). The arrows in Figure 1d represent the possibility that when an eye movement is actually executed, the orienting of attention may precede the eye movement. Indeed, as described earlier in this article, such pre-saccadic shifts of attention were first reported by Posner (1980) and Shepherd et al. (1986), for exogenously and endogenously controlled eye movements, respectively. Particularly, if they were shown to be obligatory, such pre-saccadic shifts of attention might be called “embodied.” As examples of overt orienting, however, this phenomenon would not be pertinent for answering the question posed in the title of this article which is about covert orienting.

At the Attention and Performance meeting in Erice where Klein and Pontefract’s data were presented, Rizzolatti et al. (1994) gave a talk shortly before mine. In their presentation, a much more developed premotor theory was presented than what we saw in 1987. For present purposes, I will focus on one experiment and its interesting finding. At the beginning of a trial, participants were presented with a digit informing them which of the boxes horizontally arranged above fixation would contain an upcoming signal to execute a saccade. When this visual signal was detected, the task was to make a saccade upwards to a circle positioned between fixation and the row of boxes. Although eye movements landed accurately on this target, there was a significant tendency for their initial direction to be biased away from the attended location. Rizzolatti et al. interpreted this as a support for premotor theory. I interpret it in the same way as Klein and Pontefract interpreted slower RTs for saccades in the attended direction (Figure 5b). That is, to ensure that the instruction to generate a covert shift of attention is appropriately followed, unwanted eye movements towards an attended location are inhibited. We attend covertly to a location in space not using the oculomotor system but rather by suppressing it. Covert shifts of attention when controlled endogenously are therefore not embodied.

If not embodied, then what is covert spatial orienting?

But if it is not “embodied,” is endogenous control of covert orienting therefore “disembodied”? Using Glenberg et al. (2013) description of embodied cognition’s fundamental tenet: “. . . that thinking is not something that is divorced from the body; instead, thinking is an activity strongly influenced by the body and the brain interacting with the environment” (p. 573), I would favour “disembodied.” But, some of the reading I have done on this topic offers three types of embodied cognition (see: https://plato.stanford.edu/entries/embodied-cognition/):

Body as Constraint: an agent’s body functions to significantly constrain the nature and content of the representations processed by that agent’s cognitive system.

Body as Distributor: an agent’s body functions to distribute computational and representational load between neural and non-neural structures.

Body as Regulator: an agent’s body functions to regulate cognitive activity over space and time, ensuring that cognition and action are tightly coordinated.

The inhibitory effects upon eye movements when a location that is to be covertly attended might be seen as an token of the “constraint” type of “embodiment.”

Epilogue

One thing that is apparent from our experience as mind/brain scientists is that no single experiment is definitive. To defend an idea, supported by a limited set of observations, from alternative explanations, we require converging evidence. It has been ~25 years since Klein’s second set of disconfirmations of OMRH (Klein & Pontefract, 1994). A considerable amount of relevant, converging evidence using behavioural, neuropsychological, and neuroscientific methods has been generated since then. In one review, Smith and Schenk (2012) concluded from this evidence that in the domain of endogenous control (the domain of Klein’s OMRH), the evidence from these diverse methodologies weighs heavily against premotor theory and against OMRH (see also Casteau & Smith, 2019 and Hunt et al., 2019 for similar conclusions).

Let me end on a personal note: if the evidence generated by future scholars supports the OMRH or pre-motor theory, then I will have been an early proponent of an aesthetically appealing proposal. On the contrary, if OMRH continues to be disproven, then this will agree with my empirical rejections of it. Either way, I will have made an important contribution to our understanding of the relation between attention and eye movements.