Abstract
It has often been assumed that when we use vision to become aware of an object or event in our surroundings, this must be accompanied by a corresponding visual experience (i.e., seeing). The studies reported here show that this assumption is incorrect. When observers view a sequence of displays alternating between an image of a scene and the same image changed in some way, they often feel (or sense) the change even though they have no visual experience of it. The subjective difference between sensing and seeing is mirrored in several behavioral differences, suggesting that these are two distinct modes of conscious visual perception.
Over the past several years, much has been learned about the way that vision is related to consciousness. Although it was originally assumed that—at least for human observers—the use of visual input must always be accompanied by a conscious visual experience (or seeing) of the relevant objects or events, recent results have shown that this is not always the case. For example, visuomotor systems can be controlled by stimuli that are not consciously seen (Bridgeman, Hendry, & Stark, 1975), familiarity of unrecognized faces can influence skin conductance (Bauer, 1984), and forced-choice guessing of unseen stimuli can be better than chance (Fernandez-Duque & Thornton, 2000; Merikle & Daneman, 1998). Such results have provided support for the proposal that distinct neural systems carry out visual and motor processing in the complete absence of conscious awareness (Milner & Goodale, 1995).
However, the conviction remains that whenever we do use vision to become aware of objects or events, this must be accompanied by a corresponding visual experience, a “picture” of these objects or events involving sensory qualities such as color and shape. The experiments reported here show that this belief is incorrect. In particular, some observers can consciously feel (or sense) a change in their surroundings even though they have no visual experience of it. Results suggest that this is not a “weakened” or precursor form of seeing, but rather a distinct mode of conscious visual perception.
EXPERIMENT 1
Recent work has shown that observers cannot easily see large changes that occur during a visual disruption (Rensink, O'Regan, & Clark, 1997; Simons & Levin, 1997). The explanation for this change blindness is that focused attention is needed for a visual experience of a change; as long as attention is not sent to an item, it will not be seen to change (Rensink, 2000; Rensink et al., 1997).
Experiment 1 examined whether the sensing of a change can occur during such “blindness,” using a flicker paradigm of the type used in Rensink et al. (1997) and duplicating the conditions of that study as closely as possible. Observers were presented with an original image A and modified image A′ in the sequence A, A, A′, A′, A, A, …, with a brief gray field between successive images (Fig. 1). Images were photographs of real-world scenes. Each image was presented for 240 ms, and each gray field for 80 ms. Changes were made to an object or region in each image, with three types of change possible: presence (appearance or disappearance), color, or location.
Design of Experiment 1. Trials began with a 3-s gray field containing a white rectangle. This display was followed by a 1-s gray field, followed by a flicker sequence alternating between an original and a modified image. In the example shown here, original image A (statue with background wall) and modified image A′ (statue with wall removed) alternated in the sequence A, A, A′, A′, …, with medium gray fields between successive images. (In Experiment 2, the blanks between A and A′ were colored bright yellow.) Each image was presented for 240 ms, and each blank field for 80 ms. A trial ended after the observer pressed the response key twice or 30 s had elapsed, whichever came first.
Forty naive observers were tested. Observers viewed the display and were asked to press a response key twice. The first response was to be given when they sensed a change—that is, had a “feeling” that a change was occurring. The second response was to be given when they saw the change—that is, had a visual experience sufficient for a verbal description of the changing item or region and type of change. Each observer was tested on 48 trials: 42 containing a single change (each type equally represented) and 6 containing no changes (catch trials). Observers were told that a change would occur on all trials. Before starting the sequence of trials, observers were given 6 practice trials to familiarize themselves with the task.
Performance was characterized on the basis of duration, D, calculated by subtracting the time when the first response was made (t 1) from the time when the second response was made (t 2). An α trial was one for which D was less than 1 s (i.e., there was effectively no sensing); a β trial was one for which D was greater than or equal to 1 s (i.e., there was a significant duration of sensing). Note that the 1-s threshold was a relatively conservative criterion, allowing observers considerable time to experience sensing. It also gave observers a complete cycle to verify a briefly glimpsed change and respond to it as an instance of seeing; this allowed uncertainty of detection to be eliminated as a cause of sensing.
Given that sensing did not occur during an α trial, onset of conscious experience in these trials was taken to be the time of the first key press (t 1). (The time of the second key press could also have been used; the general pattern of results is not affected by this choice.) Average t 1 over all α trials is denoted t 1[α]. In β trials, t 1 and t 2 correspond to onset of sensing and onset of seeing, respectively. Averages over all β trials are denoted t 1[β] and t 2[β].
Each observer was placed into one of three classes on the basis of performance. Observers with a low proportion (<5%) of β trials were placed into the only-see group; for these observers, change was effectively perceived via seeing alone. The remaining observers were placed into the can-sense group if their false alarm rate (on catch trials) was less than 50%, and into the guess group otherwise.
Of the 40 observers, 19 were in the only-see group, 12 in the can-sense group, and 9 in the guess group. Reclassification based on randomly chosen subsets of the data indicated that this classification did not result from selection bias (i.e., picking observers from the tails of the distribution of responses), but reflects a true divide. 1
For can-sense observers, sensing was reported on 82 of the 504 trials containing a change; average response times are shown in Figure 2. Average duration of sensing 2 was 2.35 s, with 17 trials having a duration of 5 s or more. Frequency of response did not depend strongly on change type, χ 2(2, N=82)=3.75, p=.053; there were somewhat fewer sensing responses to color change than to presence and location change. The false alarm rate was 16.7%, a value not significantly different from the 15.8% rate of only-see observers, χ 2(1, N=216)=0.004, p>.95. Hit rate for sensing was 88.2%, corresponding to a d′ of at least 2.1. 3 In accord with this d′ value, the frequency of responses to trials containing changes was reliably different from the frequency of responses to catch trials, χ 2(1, N=165)=81.7, p<10−18. Average t 1 for (incorrect) responses on catch trials was 14.6 s, more than 9 s higher than t 1[β] (p < 10−5); thus, even when a false alarm occurred, it was usually made far later than a valid sensing response. Taken together, then, these results show that sensing is not an artifact due to guessing, but corresponds to the output of an informative process.
Response times for can-sense observers in Experiment 1 (basic condition). For all three types of change (presence, color, and location), average response times are given for onset of seeing given that sensing had not occurred (t 1[α]), onset of sensing given that seeing had not occurred (t 1[β]), and duration of sensing (D[β]).
But what kind of process is this? And why was it apparent for only 30% of observers? To answer these questions, it may be best to start with the simplest possible hypothesis: Sensing involves the same mechanisms as seeing, with the differences simply due to different thresholds for responding. Note that this account immediately runs into problems, for a general strategy hift corresponding to a change in threshold should cause an effect in all (or at least most) trials. To have effects occur only in a minority of trials would require that something interfere with the strategy of shifting between thresholds for sensing and seeing.
Problems only increase as this hypothesis is made more detailed. Consider first the case in which the threshold for sensing is higher than that for seeing, so that sensing is the result of a conservative strategy. Given that the same mechanisms are involved, the threshold of a seeing response would be reached before the threshold of a sensing response, implying that observers would have had to delay reports of seeing for several seconds, contrary to instructions (and to reports of subjective experience). Another problem is the pattern of false alarms: On the 72 catch trials, can-sense observers gave more false sensing reports (12 trials) than false seeing reports (1 trial), a reliable difference, χ 2(1, N=144)=8.45, p<.005, that would not have occurred if the threshold for sensing were the higher one.
Next, consider the case in which the threshold for sensing is lower than that for seeing; in this case, sensing would be a simple precursor of seeing. Assuming that the same mechanisms are involved, the onset of sensing should always precede the onset of seeing; this is the pattern found in individual trials, and it is consistent with reports of subjective experience. However, in this case, the average onset of sensing should never occur later than the average onset of seeing for any set of trials. But as Figure 2 shows, the average onset of sensing in β trials (t 1[β]) was reliably later than the average onset of seeing in α trials (t 1[α]) for changes in both presence (p<.005) and location (p<.0001). Moreover, the response time on α trials (t 1[α]) was not reliably different for can-sense observers and only-see observers (p>.2 for all three types of change). Taken together, these results show that the onset of sensing was correlated with relatively slow responses in β trials, contrary to the predictions of the low-threshold hypothesis.
A slightly more elaborate possibility is that sensing involves the same mechanisms as seeing, but at a lower signal strength. In effect, sensing would be a weakened form of seeing that might result, for example, from an incomplete engagement of some mechanism. This would account for why sensing has a relatively late onset. However, for changes in both presence and location, the onset of seeing relative to the onset of sensing in β trials (D[β]) was not reliably different from the onset of seeing in trials on which no sensing had occurred (both ps > .15). In other words, once an observer sensed a presence or location change, the subsequent seeing of it took about as long as if the trial had started over. This would be highly unlikely if sensing simply corresponded to a weakened form of seeing, for it should not take so much additional time to “strengthen” it to the point where seeing occurred. The pattern of results is especially problematic if sensing is taken to indicate that the relevant part of the stimulus has been found by the appropriate perceptual mechanisms, because finding the relevant part is believed to be the most time-consuming component of change perception (Rensink, 2000).
Finally, if sensing is a stage of perception that always precedes (or follows) seeing, there should exist a strong correlation between the onset of sensing in each β trial and the onset of seeing relative to sensing. However, no such correlation was found (average r<.05, p>.6, for all types of change). Taken together, then, these results suggest that sensing and seeing do not simply correspond to different thresholds or signal strengths, but instead are based on processes that involve different mechanisms.
EXPERIMENT 2
Given that sensing does not involve the same mechanisms as seeing, what might underlie it? One possible explanation of the sensing observed in Experiment 1 is that it involved a simple sensitivity to transient signals in the display. Although transients were always generated each time an image was presented, the existence of a change might have been signaled subtly in two ways. First, the changes in Experiment 1 occurred with every second display, leading to a difference in the temporal rhythm whenever a change was present. Second, the changes in the images may have created chrominance transients that were not completely swamped by the achromatic blank fields (Rensink, O'Regan, & Clark, 2000). Although such second-order transients would likely have been small, some observers may have been sensitive to them.
In Experiment 2, this hypothesis was investigated with another 40 naive observers. This second experiment was a variant of Experiment 1 in which the blanks between each original and altered image were colored bright yellow, and the blanks between identical images were kept medium gray. Thus, every alternation between an original and a modified image was accompanied by a “flash” that created large luminance and chrominance transients. If second-order transients were the basis of sensing in Experiment 1, sensing would be largely destroyed, in terms of both its sensitivity and its duration, because the luminance and chrominance swings of the color flash would swamp any second-order transients. Meanwhile, seeing would remain relatively unaffected (Rensink et al., 2000).
Of the 40 observers, 17 were in the only-see group, 14 were in the can-sense group, and 9 were in the guess group. As in Experiment 1, this partitioning was likely not due to selection bias, but corresponded to a true divide. Contrary to the predictions of the transient hypothesis, the color flash had little effect on the distribution of observers among the three groups, χ 2(2, N=80)=0.26, p>.85.
For can-sense observers, the color flash had little influence on the incidence of sensing: Sensing occurred on 91 of the 588 trials containing a change, a proportion not reliably different from that in Experiment 1, χ 2(1, N=1,092)=0.07, p>.7. The distribution of sensing responses among the different types of change was also not reliably different from that of Experiment 1, χ 2(2, N=173)=4.1, p>.1; as before, the frequency of response did not depend strongly on the type of change, χ 2(2, N=91)=0.81, p>.35. The false alarm rate was 17.8%, a level not reliably different from that of Experiment 1, χ 2(1, N=183)=0.0002, p>.95. Again, the similarity of results in the two experiments is not what would be expected were transient signals the basis of sensing.
The hit rate was 71.8%, a value reliably different from the hit rate in Experiment 1, χ 2(1, N=221)=7.6, p<.01, corresponding to a lowering of d′ from 2.1 to 1.5. But although d′ was lower, responses to trials containing changes remained reliably different from responses to catch trials, χ 2(1, N=211)=56.2, p<10−13. A similar reduction of sensitivity occurred for seeing, with d′ lowered from 4.0 to 3.2. But although the general trend of this pattern is consistent with the transient hypothesis, its magnitude is not: The magnitude is more consistent with the flash having a moderate interference on both seeing and sensing than with an obliteration of sensing alone.
The incidence of sensing responses to color change was unaffected by the large color transients of the flash; there was no significant effect on either hit rates, χ 2(1, N=56)=0.5, p>.8, or false alarm rates, χ 2(1, N=52)=0.02, p>.85. Given the large luminance and chrominance transients present in this condition, these results provide further evidence that sensing is not due to the simple pickup of transient signals.
Although the color flash had only a moderate effect on the incidence of sensing in can-sense observers, it had a considerable effect on response times (Fig. 3). Average duration of sensing, D[β], increased by 1.1 s to 3.45 s (p<.02); 33 trials had a D of 5 s or longer. Onset of seeing increased by roughly the same amount for all three types of change (1.2 s on average; all ps<.05), with no reliable differences found between D[β] and t 1[β] for any type of change (all ps>.15).
Response times for can-sense observers in Experiment 2 (yellow flash). For all three types of change (presence, color, and location), average response times are given for onset of seeing given that sensing had not occurred (t 1[α]), onset of sensing given that seeing had not occurred (t 1[β]), and duration of sensing (D[β]).
No reliable increase in onset of sensing was found for changes in either presence (p>.2) or location (p>.5). This result provides further evidence against the idea that sensing is a weakened form of seeing: Were this the case, the color flash would have affected sensing at least as much as seeing. Onset of sensing did increase for color changes (p<.0003), but this increase was several times greater than the corresponding increase for seeing (4.3 s vs. 1.3 s); this divergence in the reactions of seeing and sensing to the color flash provides additional support for the proposal that seeing and sensing involve different mechanisms. 4 This proposal is further reinforced by the lack of correlation again found between onset of sensing and onset of seeing (average r<.15, p > .2, for all types of change).
GENERAL DISCUSSION
The results presented here show that at least 30% of observers can reliably sense a continual change—that is, have a conscious awareness of it without an accompanying visual experience. Although sensing was demonstrated only on a minority of trials, it could last for several seconds and be fairly accurate. Clear behavioral differences corresponded to the subjective differences between sensing and seeing, supporting the view that distinct kinds of conscious experience are involved (cf. Kihlstrom, 1996; Merikle & Daneman, 1998).
Because this form of sensing involves a conscious experience without a corresponding sensory experience, it has some similarity to the feeling of familiarity (Mangan, 2001) or the feeling of knowing encountered in studies of metacognition (Hart, 1965; Reder & Ritter, 1992). However, the effect here pertains to the state of the world rather than the state of the observer. Therefore, it may be better described as a mode of conscious perception rather than a form of “metaperception.”
Given that this mode of perception involves a conscious (or mental) experience without an accompanying visual experience, it might be called mindsight—in analogy with blindsight, which describes a lack of both mental and visual experience (e.g., Stoerig & Cowey, 1997; Weiskrantz, Warrington, Sanders, & Marshall, 1974). Although a few blindsight patients can sense high-contrast visual transients without seeing them (Weiskrantz, Barbur, & Sahraie, 1995), mindsight appears to differ from blindsight in that (as shown in Experiment 2) it is not based on the simple pickup of transient signals.
What might underlie mindsight? The data suggest two constraints on any candidate process. First, given that sensing occurred on only a subset of trials, this process may operate concurrently with that underlying seeing (presumably focused visual attention—see, e.g., Rensink, 2000), with the experience of the observer in any particular trial depending on which of these two processes detects the change first. Second, the relatively late onset of sensing suggests that this process is likely to be relatively slow.
Note that these constraints can also explain several other results encountered here. To begin with, if the process underlying sensing is concurrent with (but slower than) the one underlying seeing, responses for seeing would occur relatively quickly, thereby preempting most sensing responses; this would explain why sensing occurred on only a minority of trials. In addition, given that seeing and sensing involve different processes, and that there are large individual differences in cognitive strategy (see, e.g., Schunn & Reder, 2001), there could be large individual differences in the way these two processes are coordinated, accounting for why mindsight is encountered in only some observers. An interesting possibility in this regard is that by adopting the appropriate perceptual strategy, observers who would not normally experience mindsight might come to experience it.
Several candidate processes are consistent with these constraints. One possibility is based on the representation of scene layout, which includes the locations of the more important objects in the scene, along with perhaps a few descriptors for each (see, e.g., Rensink, 2000). If the maintenance of layout information does not require visual attention (Tatler, 2002), detecting a change in layout would involve nonattentional processes. Given that the representation of layout is largely independent of the mechanisms that support visual experience (Rensink, 2000), this candidate process could help explain why mindsight involves only a nonsensory “feeling.” Another possibility is that attention (and therefore seeing) is involved with a relatively global level of object or scene structure (Navon, 1977), and that sensing takes place when a change occurs at a lower structural level.
Whatever the mechanism involved, it does not appear to directly transmit the relevant information when it detects a change, generating instead a nonspecific alert. (This independence would explain why sensing does not generally facilitate the subsequent seeing of change under the conditions examined here.) Such a lack of direct transmission might be considered to be a serious flaw, in that it can result in delays of several seconds until the observer sees the change (presumably by directing visual attention to it). However, under most circumstances, an alerting signal would increase vigilance, which would then prompt a search for motion signals; because motion signals are not swamped in most viewing conditions, this search would lead to a relatively quick drawing of attention to the change, with at most a fleeting experience of the alert. It is only under conditions in which the automatic drawing of attention is severely hampered that this strategy would fail, with the alert persisting for an extended period of time.
Given that the drawing of attention is sometimes hampered in real-life situations, mindsight could occasionally be experienced during normal viewing. This may account for the commonly held belief in a “sixth sense” in which information about the external world is experienced in a nonsensory way. Such a belief may have arisen because of the mistaken assumption that any awareness resulting from visual input must be accompanied by a corresponding visual experience; given this assumption, the absence of such an experience would imply the absence of visual input, and thus the involvement of a different sensory modality. But the results here show that although the subjective experience of mindsight differs from the sensory picture provided by “normal” vision, there is no need to assume a separate modality for it.
Similar effects may occur in other modalities. In audition, for example, an event might generate a feeling of something occurring without an accompanying auditory experience. Tactile and kinesthetic analogues may also exist. In any event, the results presented here point toward a new mode of perceptual processing, one that is likely to provide new perspectives on the way that we experience our world.
Footnotes
1Observers in Experiment 1 were reclassified first on the basis of a random half of the trials and then on the basis of the remaining half. If classification were due to independent random events, the classification of an observer as only-see on either set of trials should have probability P, and the classification of an observer as only-see on both sets should have probability P 2; other combinations should be distributed accordingly. Of the 40 observers, the numbers classified as only-see were as follows: 17 for both sets, 7 for the first set only, 2 for the second set only, and 14 for neither set. This consistency is not well accounted for in terms of independent events, χ 2(1, N=40)=10.9, p < .001. Of the remaining observers, the numbers in the guess group were as follows: 7 for both sets of trials, 1 for the first set only, 3 for the second set only, and 10 for neither set. This was also unlikely to be due to independent events, χ 2(1, N=21)=5.86, p < .02.A similar robustness of classification was found in Experiment 2. The partitioning into only-see and non-only-see was tested via random reclassification and found to be reliable, χ 2(1, N=40)=4.8, p<.03, as was the partitioning into guess and can-sense, χ 2(1, N=23)=5.8, p<.02.
2In the statistical analyses reported here, response times were logarithmically transformed to minimize the effects of skew and kurtosis (see, e.g., Kirk, 1995, p. 105); averages are therefore logarithmic means. This is a conservative approach, with the logarithmic transformation increasing statistical reliability at the expense of smaller effect sizes. Pair-wise comparisons were done via two-tailed t tests on the logarithmically transformed response times. Analyses of variance were likewise conducted on logarithmically transformed times. Comparisons of frequency were based on chi-square tests; tests involving two factors (2×2 tables) used Yates correction (see, e.g., Langley, 1970, pp. 285–291). This also is a conservative measure, with significance in the corrected case always indicating significance in the equivalent uncorrected case.
3Determination of d′ is complicated by the fact that if seeing occurs first, sensing will not occur later. To compensate for this, the analysis of sensing excluded α trials; responses on the remaining trials were taken as indicating presence (a sensing response) or absence (no response). This approach allows a conservative estimate of d′ to be made, provided that sensitivity of sensing does not decline strongly with time. To see this, note that sensing can either precede seeing or take place concurrently with seeing. If sensing precedes seeing, a correct seeing response (hit) must be counted as a correct sensing hit. Excluding α trials (which always contain seeing hits) lowers the proportion of hits on the remaining trials, and also lowers d′. If sensing operates concurrently with seeing, the seeing responses eliminate sensing responses slower than they are, and so reduce the average time of the sampled responses. If sensitivity is not a strong function of time, the sensitivity of the sample will be close to that of the parent population. And if sensitivity increases with time, the sample responses will be less sensitive than those of the parent population, and so yield a low estimate of d′. Having a lack of response indicate absence is also conservative, because some undecided responses might have been resolved had more time been available.
4This pattern also rules out the possibility that—contrary to subjective report—observers did not sense the change but only saw it, with the delay in the second response simply the result of a brief interruption unrelated to anything perceptual. In such a situation, the average time of the first response should still be the same for the interrupted trials (interpreted as onset of sensing) as for uninterrupted trials (interpreted as onset of seeing). The finding that onsets of sensing and seeing respond very differently to the presence of a color flash eliminates this possibility, as does the finding (Figs. 2 and 
) that the two kinds of onset differ significantly for almost all types of change.
Acknowledgements
Much of this work was done while the author was at Cambridge Basic Research, Nissan Research & Development, Inc., Cambridge, Massachusetts. Support was provided by Nissan Research & Development, Inc., and by the Natural Sciences and Engineering Research Council of Canada. I would like to thank Carolyn Harp and Kristen Macuga for their help in running the experiments. I would also like to thank Corinne Enright, Dan Simons, Ian Thornton, Patrick Wilken, Carol Yin, and an anonymous reviewer for their comments on earlier drafts of this article.