Abstract
In the visual quartet, alternating diagonal pairs of dots produce apparent motion horizontally or vertically, depending on proximity. Here, we studied a tactile quartet where vibrating tactors were attached to the thumbs and index fingers of both hands. Apparent motion was felt either within hands (from index finger to thumb) or between hands. Participants adjusted the distance between their hands to find the point where motion changed directions. Surprisingly, switchovers occurred when between-hand distances were as much as twice that of within-hand distances—a general bias that was also found for tactile judgments of static distances. This expansion of within-hand felt distances was again seen when lights were placed on the hands rather than vibrating tactors. Importantly, switchover points were similar when the hands were placed at different depths, indicating that representations governing tactile motion were in perceptual three-dimensional space, not retinal two-dimensional space. This was true whether the quartets were visual stimuli on the hands or were purely visual on a monitor, suggesting that proximity is generally determined in three-dimensional coordinates for motion perception. Finally, the similarity of visual and tactile results suggests a common computation for apparent motion, albeit with different built-in distance biases for separate modalities.
Introduction
The visual apparent motion quartet is a robust example of an ambiguous motion percept (Gengerelli, 1948; von Schiller, 1933). Gengerelli (1948) introduced a simple version in which two dots flashed at opposite corners of an imaginary square, alternating with two other dots that appeared at the opposite corners. This produces an impression of motion either left and right along the horizontal edges of the imaginary square, or else up and down along the vertical edges of the square. When the distances between the dots in this square array are roughly equal, the two motion directions may alternate in a random sequence.
Many factors can influence the dominance of the horizontal or vertical direction. These factors include priming (Pinkus & Pantle, 1997), visual inertia (where motion direction will tend to be preserved) (Anstis & Ramachandran, 1987; Ramachandran & Anstis, 1983b), cortical lateralization (Chaudhuri & Glaser, 2009), and other visual phenomena such as amodal completion (Shimojo & Nakayama, 1990). The proximity of the dot pairs, however, is thought to be the strongest contributor to the direction in which apparent motion is perceived, with motion more likely to be seen between closer dot pairs than more distant ones (Gengerelli, 1948; Ramachandran & Anstis, 1983a). Because of the dependence on proximity in the visual domain, the quartet stimulus is a useful probe of how distance is represented in different modalities and to identify if proximity-related grouping also drives the motion switch in those modalities.
In vision, there is also an interesting bias toward vertical motion even when the pairs are equally spaced in both directions. This may be due to cortical lateralization, but the source and extent of this bias is the focus of ongoing debate (Chaudhuri & Glaser, 2009; Genç, Bergmann, Singer, & Kohler, 2011; Strüber, Rach, Trautmann-Lengsfeld, Engel, & Herrmann, 2014). Again, testing the tactile quartet can examine whether this vertical bias is specific to vision or a common property of apparent motion computations.
The ambiguous motion in the visual quartet has been tested in the tactile modality. Carter, Konkle, Wang, Hayward, and Moore (2008) produced a tactile version of the quartet by stimulating the tip of a single index finger with a small array of moving pins and found typical results of the visual quartet (e.g., perceptual reversals, dependencies on previous trial judgments, proximity biases). They also found cross-modal interactions, where making an eye movement (with eyes closed) in the direction orthogonal to the perceived direction of motion increased the number of reported switches in the tactile modality. In another study, Liaci, Bach, Tebartz van Elst, Heinrich, and Kornmeier (2016) tested the effects of reference frames on the tactile quartet by stimulating locations on the arms in various configurations (e.g., crossing arms, holding arms in parallel vertically or horizontally). They found ambiguous motion with a slight (but significant) bias toward vertical motion, but with different results among participants, since some participants seemed to take into account the position of the arms along with the positions of the tactile stimulation. Finally, the numerous examples of cross-modal interactions on motion perception (Bensmaia, Killebrew, & Craig, 2006; Konkle, Wang, Hayward, & Moore, 2009; Pei et al., 2013) suggest that visual and tactile perception may share some underlying computations of motion.
In this article, we used the tactile apparent motion quartet to explore how tactile distances are represented within and between the hands. We then compared these results to versions with visual stimuli to look for evidence, if any, of a common representation for the resolution of ambiguous motion. In Experiment 1, we presented participants a tactile analogue of the visual quartet by stimulating two fingertips (i.e., the index fingers and thumbs of both hands) with vibrating tactors, which produced the apparent motion of two vibrations moving vertically (between the fingers of the same hand) or horizontally (between the hands). We determined the point at which motion would be perceived equally often as horizontal or vertical, that is, the point of subjective equality (PSE). A variety of hand configurations were tested to determine any biases in three-dimensional (3D) space. In Experiment 2, we compared the PSE between tactile and visual modalities by replacing the tactors with LEDs and obtained measurements for the visual quartet (on-hand visual condition), as well as the tactile quartet, using the same experimental setup. This would allow us to identify how proprioceptive feedback might influence this type of motion perception. We then tested a static, tactile judgment for within-hand versus between-hand spacing in the absence of motion (Experiment 3) to compare to the results with motion. Participants formed an imaginary square with the thumbs and index fingers of both hands to see if any biases that we found with motion judgments were based on baseline biases in the tactile domain. Finally, in Experiment 4, we tested a purely visual quartet using a computer monitor angled at different orientations to compare to the on-hand visual arrangement without the tactile component in the previous experiments.
We addressed the following questions: (a) Are within-hand and between-hand distances weighted equally in determining the switching point between the two motion percepts? Or is a within-hand distance overestimated due to extra resolution and precision of touch and motion within the hand, or alternatively underestimated because of proximity-based grouping effects on apparent spacing (Coren & Girgus, 1980)? (b) Are the tactile and visual domains affected equivalently by ambiguous motion? (c) If there is a proprioceptive bias favoring either between- or within-hand distances, does it carry over to visual stimuli placed on the hands? (d) Is proximity determined in 3D (real-world) space or 2D (retinal) space?
General Methods
Apparatus
The vibrating stimuli were generated by activating small electromagnetic tactors (Dancer Design, St. Helens, UK). These tactors are small enough to be attached to the fingers via tape or ring adhesives (tactors are 18 mm in diameter and 12 mm in height) and precise enough to vibrate up to 300 Hz. The tactor vibrations were controlled by a Dell Precision T1700 desktop computer (3.4 GHz, with Intel Core i7 processor, running Windows 7 Professional) using Python code that designated the duration of the vibrations and the interstimulus intervals (ISIs). The tactors were driven by a standard audio amplifier (Lepai Hi-fi stereo digital amplifier, model LP2020 A+), with two tactors connected to each output channel (i.e., four tactors in total).
For the visual stimuli in Experiment 2, the tactors were removed from the wires and replaced with LED lights (green, 5 mm 6 cd). These were taped to the fingers in a way that maintained visibility during the task. In Experiment 4, we presented the stimuli on an Apple iMac desktop computer (2.7 GHz quad-core Intel Core i5 processer; OSX 10.6.3) with a 27-in. display and NVIDIA GeForce GTX 660 M graphics card, 1,920 × 1,080 resolution (60 Hz refresh rate).
Participants
Study participants were recruited from the Université Paris Descartes and received compensation for their time (€10/h), unless otherwise noted. All participants signed an informed consent approved by the Université Paris Descartes Review Board, CERES, in accordance with French regulations and the Declaration of Helsinki.
Analyses
All analyses were conducted in MATLAB (2014b). Analyses of variance (ANOVAs) include participant ID as a random variable to control for between-subject variability and used Scheffé’s procedure to correct for multiple comparisons. Correlations (Experiment 2) examined the relationship between the two modalities (visual and tactile) with respect to the points of subject equality (PSE) ratios; the PSE ratios comprised of the between-hand distance to the measured within-hand distance (i.e., finger to thumb distance). It is important to note the hysteresis effect that could occur in this experiment. Previous research has shown that perceptual memory affects judgments of ambiguous motion or bistable stimuli, making it more likely to continue to be perceived as it currently is (Hock, Kelso, & Schöner, 1993; Poltoratski & Tong, 2014). This may be the result of a cooperative neural network for similarly tuned elements that maintain the excitatory state of neurons associated with one of the precepts (Williams, Phillips, & Sekuler, 1986). That is, the memory of a previous perceptual state can affect how shifts in the direction of ambiguous motion occur—presumably to maintain perceptual stability by resisting changes in the perception of ambiguous features. Therefore, we alternated measurements between inward and outward movements to minimize the bias from hysteresis that would occur if only one movement was repeated (e.g., only making outward movements). By averaging the two PSEs derived from the inward and outward motions, the bias in the result is minimized. Finally, when comparing the resulting ratios from the experimental conditions to an equal PSE, we used a baseline PSE ratio of 1.0 (i.e., no vertical or horizontal bias) as the constant value for comparison.
Experiment 1: Tactile quartet motion
This experiment aimed to determine the PSE (where the perception of horizontal and vertical motions occurred equally) while participants moved their hands away from and back toward each other. To examine any bias in 3D space, the participant moved their arms in three different configurations: (a) parallel to the torso horizontally, (b) across the torso vertically, and (c) to/away from the torso horizontally (see Figure 1). We examined whether motion within a hand would be perceptually favored so that the hands would have to be closer together than the index finger to thumb spacing before the motion would be seen between the hands. In this case, we assume that the motion within the hand would receive grouping effects as observed in the visual domain (e.g., Coren & Girgus, 1980), which would make spacing within the hand appear closer than spacing between the hands.
Experimental design for Experiment 1. Illustration of hand configuration and arm movement directions in Experiment 1: horizontal (fronto-parallel), vertical, and near-far. The black and white dots represent the locations of the vibrating tactors, which were activated in pairs for 100 ms with an ISI of 200 ms (in this illustration, the pairs with the same color vibrated simultaneously). Experiment 2 tested the horizontal and near-far conditions for both tactile and visual (on-hand) conditions. Experiment 3 used a nonmotion test of judged distance comparing within-hand to between-hand spacing. Experiment 4 tested visual stimuli on a computer screen in a way to simulate the horizontal and near-far conditions in the previous experiments.
In contrast, studies of distances judged by touch do not show an underestimation of distance within the hand, but rather an overestimation—an effect known as the Weber illusion and attributed to the overrepresentation of the hand on the somatosensory “homunculus” (e.g., Taylor-Clarke, Jacobsen, & Haggard, 2004). A similar effect is seen in vision where objects (or lines) in the periphery are judged to be smaller (or shorter) than a matched size object in the fovea (e.g., Baldwin, Burleigh, Pepperell, & Ruta, 2016). However, the tactile studies all compare the within-hand judgments to that for two touches further along the forearm or two touches, for example, on the back (de Vignemont, Majid, Jola, & Haggard, 2009). All of these spacings, being within a common region (hand, forearm, or back), would be equally grouped, negating any grouping factor. Most importantly, none of these studies have compared tactile judgments within a region to tactile judgments across limbs as we do. There are two that compared unimanual versus bimanual grasping (Smeets & Brenner, 2001; Tresilian & Stelmach, 1997) and found that grasping is more precise in the unimanual case. One study that did look at unimanual versus bimanual length estimates (Panday, Bergmann Tiest, & Kappers, 2014) reported only that the within-hand judgments were, unsurprisingly, less variable than the between-hand judgments. They did not compare the magnitude of the length estimate—our study is the first to do so.
Participants
Five participants volunteered for this experiment and signed the approved informed consent (three authors and two volunteers, one naïve and one female). No compensation was given for this experiment, which lasted under 30 minutes.
Procedure
Four vibrating tactors were arranged within a large pair of wool gloves to stimulate the tips of the index fingers and thumbs. Tactors were inserted into the thumb and index finger of each glove—four tactors in all. The participant wore the gloves and could manipulate the tactors through the fabric until they received comfortable vibrations on their fingertips. The two pairs of tactors turned on and off briefly in alternation—in the same pattern as the visual quartet stimulus—with a continuous cycle of a 100-ms 80 Hz vibration and a 200-ms ISI.
In a dimly lit room, the participant was blindfolded and asked to stretch their thumbs and fingers to a comfortable distance, forming a square with approximately 7.7 cm between each fingertip, on average. The hands were never allowed to touch each other (in this and all other experiments). The participant was told to indicate if they felt any motion between the vibrations, which could be felt as either apparent motion within each hand (running between the index finger and thumb) or between the hands (jumping across the gap between the parallel index fingers or thumbs). When they felt an apparent motion that travelled between the hands at this starting point, they then moved their hands further apart and told the experimenter when the motion percept changed to the other direction, to within the hands (taking approximately 2–15 seconds). Then, starting from a wider arm distance (hands roughly shoulder width apart), the participant confirmed that they felt the within-hand movement and then moved their hands back together again and told the experimenter when the motion switched to the other direction (the arm movement took 5–20 seconds). The distance at which the motion switched directions (the PSE) was measured by the experimenter with a ruler. The participant moved their arms in the three different configurations: fronto-parallel horizontal, fronto-parallel vertical, and near-far movement to/away from the torso horizontally. When performing the arm movements for the near-far condition, the angle of movement was not perfectly orthogonal (90°) to the torso—but was 60° to 75°, with the hand closer to the body being offset from straight ahead (the participant chose which hand to keep closer to the body and was kept consistent across all trials). Ten measurements were recorded for each orientation of hand movement, with five outward and five inward movements measured, in alternation, to reduce any hysteresis effect. See Figure 1 for an illustration of the arm configurations.
Results
Participants reported that the motion organization was not always clear, but they were able to complete the task. On average, the PSE (where within- and between-hand motion was reported with equal frequency) occurred when the distance between the vibrating tactors on the two hands was 1.44 times greater than that between the tactors on the thumb and index finger of each hand (Mean PSE = 11.14 cm; Mean finger distance = 7.7 cm; Mean PSE ratio = 1.44). The between-hand to within-hand PSE ratios were not significantly different among the three movement conditions, F(2,8) = 0.01, p = .987, but all PSE ratios were significantly greater than 1.0 (p values < .001; see Figure 2).
Experiment 1 results for the tactile quartet (n = 5). The bar graph indicates the average ratio of the between-hand to within-hand distances at which the tactile motion between and within the hands was reported equally often (the point of subjective equality or PSE); the dashed black line indicates the between-hand distance that would match the average finger to thumb distance (approximately 7.7 cm). Error bars are 95% confidence intervals across all subjects (i.e., not an average of within-subject values). All conditions had PSE ratios significantly greater than 1.0 (p values < .001). The first row depicts the hand movement directions, and the second row depicts the resulting between-hand to within-hand ratios (between-hand distance on the horizontal axis and within-hand distance of 1.0 on the vertical axis). PSE = point of subjective equality.
The hysteresis effect could be seen numerically in the PSEs for inward versus outward hand movements in this experiment: When making outward movements of the hands, the percept started with a between-hand motion and the PSE for the switchover occurred at 13.53 cm on average, with a between- to within-hand PSE ratio of 1.73. This was numerically larger than the PSE for the inward hand movement where the initial percept was of within-hand motion and the PSE for the switch to between-hand motion was 8.93 cm for a PSE ratio of 1.16. This difference, however, was not significant, F(1,4) = 3.99, p = .117, and there was no interaction with the arm movement condition, F(2,8) = 1.53, p = .222. This indicates that hysteresis in the judgments may be present but not significant. In any case, we alternated measurements between inward and outward movements and averaged the PSEs from the two directions to minimize the bias from hysteresis that would occur if only one direction was repeated.
Discussion
The results of this experiment were opposite to our initial grouping proposal that within-hand motion would be underestimated (in a way similar to effects of visual grouping on perceived spacing, e.g., see Coren & Girgus, 1980). The bias for our participants favored reporting between-hand motion suggesting that the within-hand spacing between index and thumb was overestimated (i.e., perceived as being longer than it physically was). Specifically, the switchover point between the two motions occurred when the between-hand separation was 44% larger than the within-hand distance (thumb to index finger).
Interestingly, the ratio when the hands were arranged in depth was quite similar to that measured when they were fronto-parallel with respect to the torso. If the proximity that governs the apparent motion is determined only by 2D retinal distances, then the PSE should require much larger between-hand spacing in the near-far condition than in the fronto-parallel condition. In the near-far condition, the 2D distance between the hands (e.g., on the retinal projection) is much reduced compared to their 3D spacing, making the stimuli on the two hands act as if they were much closer together. If processing is in 3D space, however, we should expect no difference between the conditions.
The similarity of the PSE ratio for the near-far and fronto-parallel conditions indicates that the motion computation was based on real-world 3D distances, not their 2D projections on the fronto-parallel plane. In addition, the absence of a significant difference between the horizontal and vertical orientations (see Figure 2, Condition 1 vs. Condition 2) suggests that there was little or no role played by the left versus right division in space relative to the body.
Experiment 2: Tactile and on-hand visual motion perception in different orientations
To directly compare the tactile results to the visual quartet, we repeated the tactile quartet and also added a visual version by using LED lights in the place of the tactors. This allowed us to collect both visual and tactile data (separately) using the same experimental design, and thus make within-subject comparisons on ambiguous motion perception. We examined whether or not we will still get a switching point with a large between-hand distance in the visual (on-hand) condition equivalent to the large spacing (1.46 PSE ratio) that we found in the purely tactile horizontal condition of Experiment 1. Since the LEDs are attached to the hands, it is possible that proprioceptive information could bias the results of the visual condition given that there are interactions across modalities (Bensmaia et al., 2006; Konkle et al., 2009; Pei et al., 2013). This experiment also examined whether ambiguous motion direction in the on-hand visual quartet is computed taking depth into account (in 3D space) as was the case for the tactile stimuli in Experiment 1, or if it is more dependent on retinal distance (in 2D space).
Participants
Twelve naïve participants were recruited for this study. All participants signed the approved consent form and were paid €10 cash for completing the experiment lasting 50 to 60 minutes.
Procedure
The same methods were used as in Experiment 1, except for the following modifications. Only two arm movement directions were tested: the horizontal movement in the fronto-parallel plane and the near-far condition in front of the torso. In addition to the tactile quartet, we tested an on-hand visual version by replacing the vibrating tactors with LEDs and removing the blindfold. The tactors or LEDs were taped to the fingers using double-sided ring adhesives, instead of being attached within a glove. Each participant was run on both tactile and on-hand visual conditions in separate blocks (counter-balanced order), with 20 measurements recorded from each condition (10 outward and 10 inward arm movements). The distance between the index finger and thumb at the start and end of each block was also recorded (for a more accurate measure of mean finger distance). Again, when performing the arm movements for the near-far condition, the angle of movement was not perfectly orthogonal (90°) to the torso—but was 60° to 75°, with the hand closer to the body being offset from straight ahead (the participant chose which hand to keep closer to the body and was kept consistent across all trials).
Results
Data from two participants were excluded due to incomplete data from one participant and extremely high measurements from another participant (i.e., outlier with values >2.5 standard deviation from overall mean in one condition); the reported results are for the remaining 10 participants.
An ANOVA of modality by movement orientation indicates no effect of modality, F(1,9) = 0.43, p = .526, but there was a significant effect of movement orientation, F(1,9) = 8.77, p = .016, with an interaction, F(1,9) = 5.46, p = .020. First, we report the difference between the visual and tactile stimuli by arm movement conditions. In the fronto-parallel movement condition, the tactile PSE occurred when the distance between hands was 1.64 times greater than that between the thumb and index finger of each hand (Mean PSE = 11.12 cm; Mean finger distance = 6.8 cm; Mean PSE ratio = 1.64); the visual PSE occurred when the distance between hands was 1.90 times greater than within hands (Mean PSE = 13.11 cm; Mean finger distance = 6.9 cm; Mean PSE ratio = 1.90). These PSE ratios were not significantly different from each other, F(1,9) = 2.47, p = .150.
In the near-far movement condition, the tactile PSE occurred when the distance between hands was 2.32 times greater than that between the thumb and index finger of each hand (Mean PSE = 14.97 cm; Mean finger distance = 6.6 cm; Mean PSE ratio = 2.32); the visual PSE occurred when the distance between hands was 2.24 times greater than within hands (Mean PSE = 12.29 cm; Mean finger distance = 5.7 cm; Mean PSE ratio = 2.24). These PSE ratios were not significantly different from each other, F(1,9) = 0.27, p = .617. In all four conditions, the PSE ratios were significantly greater than a ratio of 1.0 (p values < .001).
Comparing the results for the two movement orientations, the tactile near-far movement had a significantly higher PSE ratio than the fronto-parallel movement condition, F(1,9) = 12.19, p = .007. In the visual condition, the PSE for the near-far condition was numerically larger than the PSE for the horizontal condition, but this was not significant, F(1,9) = 3.59, p = .091. This difference in the strengths of the near-far versus fronto-parallel results is the source of the modality by movement orientation interaction reported earlier.
An overall ANOVA of the four quartet conditions by inward versus outward arm movement direction (not divided in 2 × 2 modality by orientation) confirms a significant effect of condition, F(3,27) = 5.41, p = .005, as well as inward versus outward arm movement direction, F(1,9) = 41.17, p < .001, with an interaction, F(3,27) = 46.14, p < .001. Again we found that hysteresis produced differences in the PSE for the outward and inward arm movements (which were significant here), with the tactile measures for the outward movement at 18.18 cm (PSE ratio = 2.75) and for the inward movement at 7.91 cm (PSE ratio = 1.21); in the visual condition, the measures for the outward movement was at 18.37 cm (PSE ratio = 2.96) and for the inward movement at 7.03 cm (PSE ratio = 1.18). We present the average of the outward and inward values (see Figure 3) and in the correlation analyses to reduce this hysteresis effect.
Experiment 2 results (n = 10). The bar graph indicates the average ratio of the between-hand to within-hand distances at which the motion between and within the hands was reported equally often (PSE), for each sensory condition; the dashed black line represents the between-hand setting that would match the average finger to thumb distance (approximately 6.5 cm). Error bars indicate 95% confidence intervals. All conditions had PSE ratios significantly greater than 1.0 (p values < .001). The first row depicts the hand movement directions, and the second row depicts the resulting between-hand to within-hand ratios. PSE = point of subjective equality.
Within the same arm orientation conditions (near-far vs. fronto-parallel), the tactile and visual PSE ratios were significantly correlated for the participants. The correlation between tactile and visual PSE ratios for the near-far condition (Pearson’s r = .865, p < .001; regression line intercept: b = 1.04) was slightly higher than the horizontal movement condition (Pearson’s r = .695, p = .026; regression line intercept: b = 0.90; see Figure 4).
Correlations between tactile and visual ratios in Experiment 2 (n = 10). Data for the horizontal arm movement are shown in the left figure and for the near-far arm movement are shown in the right figure. Each point represents the average PSE ratio of across-hands to finger-thumb distance for each participant; the tactile ratio is presented on the x-axes and the visual ratio on the y-axes. There was a significant correlation for both the horizontal movement (r = .695, p = .026; b = 0.90) and the near-far movement (r = .865, p < .001; b = 1.04).
Discussion
The strong effect of 3D separations, as opposed to the 2D visual projection to the fronto-parallel plane, was unexpected for the visual condition. In the near-far arm orientation, the dots across the hands are visually closer to each other on the horizontal axis in 2D coordinates (retinal coordinates for the visual condition) than in depth (3D coordinates). Since the angle in the near-far condition was about 60° to 75° from fronto-parallel, the 2D distances on the retina are compressed by a factor of 2 (for 60°) to a factor of 4 (for 75°) compared to the actual distances in 3D space. This would require that the PSEs for the near-far condition be 2 to 4 times larger than in the horizontal condition. In Experiment 1, the PSE for horizontal and near-far conditions were very similar favoring a 3D basis for the proximity that determined the tactile motion direction. In this experiment, however, the PSEs for the near-far conditions are larger than for the horizontal condition (ratio of 2.32/1.64 = 1.43 for the tactile case and 2.24/1.90 = 1.18 for the visual case). This increase of PSEs for the near-far conditions is in the direction consistent with a 2D resolution of proximity that drives the motion percept; however, both of these ratios are significantly smaller than 2.0 (p values < .01), which is the smallest ratio that would be consistent with a 2D (retinal) basis for motion. Nevertheless, the tactile PSE of 1.43 for near-far versus horizontal is higher than 1.0 (p = .01). This differs from the result of Experiment 1 (but not significantly, t(13) = −1.95, p = .073) where the near-far versus horizontal PSEs were virtually identical (ratio of 0.98). This result could suggest some contribution of 2D distance, for example, in egocentric, head-centered coordinates since the participants were blindfolded in the tactile condition. In contrast, the visual PSE ratio of 1.18 for the horizontal versus near-far conditions is not significantly different from 1.0 (p = .09). This result indicates that 3D spacing plays a strong role for the visual case when the visual stimuli are on the hands.
Since all of the between- versus within-hand PSE ratios were significantly larger than 1.0, it seems that the space between the hands was again treated as closer together than the space within the same hand, whether the stimulation was tactile or visual. In the visual conditions, this suggests that the proprioception that localizes the flashes of light to specific locations on the hands may have contributed to the motion computations, and that the proprioceptive input influenced both the visual and tactile localization to a similar extent. Even though the visual condition did not include vibrations on the hands, feedback from the hand movements may have affected the motion judgments.
In Experiment 3, we will examine whether or not the expansion of vertical space within the hand is a general property of proprioception that can be seen even in the absence of motion. In Experiment 4, we will examine whether or not proprioception is critical for determining proximity in 3D space by using visual stimuli not placed on the hands.
Experiment 3: Tactile static quartet distance judgments
The quartet motion experiment showed that between-hand distances were judged to be closer than within-hand judgments. Here, we asked whether this bias was a general property of tactile distance judgments or specific instead to just motion judgments. To do so, we asked participants to imagine they were holding a square tile with the thumbs and index fingers of the two hands on the four corners of the tile.
Participants
Five naïve participants (all female) volunteered for this experiment and signed the approved informed consent. No compensation was given for this experiment, which lasted under 30 minutes.
Procedure
Participants were asked to hold their hands up in front of them with their eyes closed, and to position the tips of their thumbs and index fingers to form a square, as if they were holding an imaginary vertical square tile. They were asked to reposition their hands in rapid succession as if holding several tiles of different sizes with the order and magnitude under the participant’s control. They made 12 to 14 different poses. Their hands were photographed each time for later analysis. Small pieces of tape were stuck to the ulnar edge of each hand. On the left hand, a piece of masking tape bore the name of the participant. On the right hand, a short length of centimeter tape permitted later calibrations of the interdigit distances from the photographs.
Results
The mean distance between index and thumbs within the hands across all the poses (5.38 cm) was matched to a distance 1.49 ± 0.08 times larger than between the hands (8.15 cm). This is significantly greater than 1.0 (p < .005) and similar to the ratio found for the quartet motion settings for a different group of participants in Experiment 1 with their hands in a similar fronto-parallel horizontal orientation (1.46 PSE ratio). This result indicates that the bias in judging space within and between the hands is a general property of proprioception and not a special case for motion judgments. This unimanual versus bimanual bias for length judgments has not been reported previously. Panday et al. (2014) did report that within-hand judgments were less variable than between-hand judgments, but they did not compare the magnitudes of the unimanual versus bimanual estimates.
Experiment 4: Visual motion perception in different orientations
This experiment tested whether the motion computations for visual stimuli are in 3D space by presenting the visual quartet stimuli on a computer screen at different orientations in front of the participant to mimic the range of angles when holding the arms in the near-far condition of the previous experiments (the computer screen being fronto-parallel at 0° and near-far at 60°–75°). This manipulation of visual angle aimed to clarify whether the motion direction in the visual quartet is computed in 3D or 2D (retinal) space when there is no involvement of proprioceptive feedback from the hands.
Participants
Six naïve participants were recruited for this study (three females). All participants signed the approved informed consent and were paid €10 for completing the experiment lasting 50 to 60 minutes.
Procedure
In this experiment, the visual stimuli were presented on an Apple desktop computer with a 27-in. monitor (1,920 × 1,080 resolution at 60 Hz) and controlled by MATLAB with Psychophysics Toolbox (Brainard, 1997). Participants were seated in front of the monitor in a dimly lit room at approximately 57 cm from the screen (no chin rest was used). The quartet dots were yellow, ∼1° visual angle, and presented on a black background for 100 ms with a 200-ms ISI (same durations as in the previous experiments). The quartet began with the dots closer to each other horizontally to induce horizontal apparent motion (at 75% of the vertical distance of approximately 5.3 cm). Once the horizontal motion was seen, the participant pressed the space bar and the two dots making the vertical edges of the stimulus began to move away from each other slowly (16.67 pixels per second or 0.52° per second in the fronto-parallel case). The participant pressed the keyboard space bar to indicate when the apparent motion switched to the vertical movement. Once this indication was made, the dots making the vertical edges appeared near the edges of the screen (300 pixels from the edges), which induced vertical apparent motion. Then, the dots began to move inward slowly and the participant indicated when the apparent motion switched to horizontal motion during this inward movement.
The display was designed to have similar visual depth cues as the version with the lights placed on the fingers in Experiment 2 (e.g., length of movement, placement of visual stimuli in depth). The baseline orientation condition presented the stimuli directly in front of the participant, similar to the horizontal motion in the previous experiments—this is the 0° orientation condition. The monitor was then angled at 60° and 75° by rotating the screen so that the right edge was nearer than the left edge, while keeping the base at the same location, in order to simulate the near-far conditions in the previous experiments (when performing the near-far arm movements, the angle was not perfectly orthogonal to the torso at 90°—instead there was an angle of 60° to 75°, with the hand closer to the body being offset from straight ahead). Twenty measurements were recorded from each of the three orientation conditions (10 outward and 10 inward movements), which were presented in counterbalanced blocks.
Results
Outlier trials greater than 2.5 standard deviations from the mean PSE were removed from these analyses (which removed only 11 trials out of the 360 total). On average for all three conditions, the visual PSE occurred when the horizontal distance was 1.22 times greater than the vertical distance (Mean PSE = 6.50 cm; Vertical distance = 5.3 cm; Mean PSE ratio = 1.22). In the baseline horizontal movement condition of 0°, the visual PSE occurred when the horizontal distance was 1.11 times greater than the vertical distance (Mean PSE = 5.95 cm; Vertical distance = 5.3 cm; Mean PSE ratio = 1.11). At 60°, the visual PSE occurred when the horizontal distance on the screen was 1.25 times greater than the vertical (Mean PSE = 6.67 cm; Vertical distance = 5.3 cm; Mean PSE ratio = 1.25). At 75°, the visual PSE occurred when the horizontal distance on the screen was 1.29 times greater than the vertical (Mean PSE = 6.87 cm; Vertical distance = 5.3 cm; Mean PSE ratio = 1.29). All PSE ratios were significantly greater than 1.0 (all p values < .001; see Figure 5).
Experiment 4 results (n = 6). The bar graph indicates the average ratio of the horizontal to vertical distances at which the vertical and horizontal motion was reported equally often (PSE), for each screen angle condition; the dashed black line represents the constant vertical distance (approximately 5.3 cm). Error bars indicate 95% confidence intervals. All conditions had PSE ratios significantly greater than 1.0 (p values < .001). The first row depicts the orientation of the monitor, and the second row depicts the resulting horizontal to vertical ratios. PSE = point of subjective equality.
An ANOVA comparing the PSE values indicated no significant effect of inward versus outward movement direction, F(1,5) = 0.13, p = .735, but there is a significant effect of screen angle, F(2,10) = 7.88, p = .009, and an interaction, F(2,10) = 4.72, p = .009. Testing for significance between the conditions (corrected for multiple comparisons using Scheffé’s procedure) indicates that the PSE in the baseline 0° condition is significantly smaller than both the 60° and 75° conditions (p values < .001), but there is no difference between 60° and 75°. Surprisingly, although the PSEs were larger with the monitor angled away from the fronto-parallel plane, the increase of about 10% is far smaller than required to compensate for the compression of the horizontal distances on the retina. An increase by a factor of 2 (to a PSE of 2.22) would match the geometry of the 60° angle, and an increase of almost 4 (to a PSE of 4.28) would match the geometry of the 75° angle. The PSE increases for the angled screens are well below these values—both PSEs were significantly below 2.0 (p values < .001).
Although there was a nonsignificant overall trend of hysteresis, there was an interaction with screen angle. A multiple comparison test indicated that the outward and inward movements were significantly different from each other only in the 60° condition (p < .001).
Discussion
These results suggest that the apparent motion computation operates in 3D coordinates even for purely visual stimuli—a property of apparent motion computation that has not previously been reported. The similar PSEs in real-world coordinates for the near-far versus horizontal conditions seen for the tactile stimuli in Experiment 1, and to a lesser extent for both tactile and visual stimuli in Experiment 2, may result from this 3D apparent motion computation. This implies that the 3D nature of the visual computation seen in Experiment 2, where the visual stimuli were on the hands, was unrelated to purely proprioceptive factors.
Our finding of a slight bias favoring horizontal motion in the purely visual quartet here is the opposite of the vertical bias reported in previous experiments (e.g., Chaudhuri & Glaser, 2009; Gengerelli, 1948). While this bias was small (the horizontal distances being 1.11 times greater than vertical on average), it was significant. Like the Liaci et al. (2016) results, there may be an influence of context in shaping perception when it leans toward being ambiguous. For the visual stimuli, the participants were free to inspect the display, and viewing away from the horizontal midline of the quartet may have weakened the vertical bias (as seen in Gengerelli, 1948). Also, the horizontal bias may have been influenced by the wide screen format of the monitor (16:9), where horizontal cues may have been more present than vertical cues (e.g., the bottom and top visible edges of the monitor being closer to the stimuli than the vertical ones, other horizontal cues in the room such as the table top).
To rule out the effect of order on the results, we conducted an ANOVA with the condition appearing first as a random variable. We found a significant effect of screen angle, F(2,4) = 10.71, p = .024, and order condition, F(2,4) = 12.42, p = .019, but no interaction, F(2,4) = 1.07, p = .369. This suggests that while there is an effect for which condition is shown first, it does not matter which screen angle it is (balancing out in the overall results).
Finally, another factor to consider in the proximity for the dots on the screen is the change in the magnitude of the vertical offset with distance when the screen is tilted away from fronto-parallel. The near vertical spacing will be expanded and the farther vertical spacing compressed. For the screen distance and stimulus positions of our display, however, the compression for the farther vertical spacing is about 11% for the 60° tilt and 14% for the 75° tilt. The nearer spacing is similarly expanded. Even if the compressed farther spacing were completely dominant in driving the switchover of the ambiguous motion, it would only offset a small fraction of the two- to four-fold change in PSE that would be expected if the motion was computed in 2D coordinates. Clearly, the visual quartet motion is determined in 3D coordinates, taking size-distance scaling into account, when the screen is visible under binocular viewing. This indicates that even though the vertical separation between the stimulus dots are expanded (for near dots) and compressed (for far dots) on the retina, size distance constancy ensures that the perceptual distances are mostly unaffected. If we instead made the retinal distances constant, the percept would be that the vertical separation of the farther dot pair was larger.
Conclusions
In this study, we examined how ambiguous motion is resolved by comparing the apparent motion quartet in the tactile and visual modalities. We should first note that the tactile motion task was difficult; two participants in Experiment 2 were unable to make stable or any judgments. For those retained in the statistical analyses (26 out of 28 overall), there were large variations in individual measurements. With only a total of 26 participants across our four experiments, we are cautious about interpreting the results. Nevertheless, the results revealed a number of clear patterns.
First, our tactile results indicate that the within-hand and between-hand distances are not weighted equally when determining the switching point (PSE) between the two motion percepts. Assuming that proximity is a main determinant of the perceptual switches, our results suggest that the space within the hands (thumb to index finger) is treated as farther apart than the space between the hands, resulting in motion switchovers when the hands are further apart than the thumb to index finger distance. This bias was confirmed in a test of static positioning of the hands and fingers in Experiment 3. When participants were asked to make a square shape with the index fingers and thumbs of the two hands (as if grasping a square tile), the between-hand distance was again larger than the within-hand distance. Ours is the first study to test and report this bias in unimanual versus bimanual distance judgments, with or without motion.
There may be an explanation for the bias in the layout and resolution of the somatosensory system, but it is not compelling. To begin with, judgments of distance between two touches on the skin are overestimated for regions that have expanded representation on the somatosensory cortex, such as on the fingertips versus on the forearm (de Vignemont et al., 2009). This is called Weber’s illusion but does not help us in comparing within-hand distance judgments versus between the two hands. There is no straightforward metric for comparing the cortical representation of space within the hand versus across the hands, as the two hands are represented in separate hemifields. A related argument can be made for tactile resolution where the threshold for discriminating a double from a single contact (two-point threshold) is best on the fingertips than on other areas of the body (e.g., the back, Weinstein, 1968). The finer receptive fields on the fingertips and on the hand compared to those on the back are thought to underlie such differences in two-point thresholds. Again, we could argue that the finer resolution on the hand might underlie the reported overestimation of hand distances relative to distances on the forearm or on the back. But there is no equivalent measure of two-point threshold across the hands and no receptive fields that bridge the two hands that would lead to confusion about which hand was being touched. Nevertheless, there is evidence that unimanual length estimates (thumb to index finger) are more precise than bimanual (index finger to index finger). There is a similar asymmetry in the precision of movement: There is greater precision in movements within the hand (i.e., between the index finger and thumb) than between the hands (Smeets & Brenner, 2001; Tresilian & Stelmach, 1997). These biases in precision may create a greater perceived distance covered by the within-hand movement. However, this argument is only a correlation between biases and not an explanation.
Interestingly, the overestimation of within-hand distances was also seen when the visual stimuli replaced the tactile stimuli on the hands. This suggests that the computation of visual locations is not purely visual but also takes into account some proprioceptive input from hand positions and the factor of being within versus between the hands. Yet, as seen in Experiment 4, proprioception is not the only factor that affects these apparent motion representations but also includes information about the 3D spacing.
Overall, the pattern of visual PSEs was quite similar to that for tactile PSEs. Most interestingly, the PSEs for the tactile and on-hand visual conditions were highly correlated across individuals (Experiment 2, Figure 4). These results suggest that apparent motion for tactile and visual modalities may share a common computation, or that their separate computations are calibrated to be similar across exposures to dynamic visual and tactile events.
The similarity of the quartet effect in both the visual and tactile modalities is in line with results from previous studies comparing the detection of motion in the two modalities (e.g., Pack & Bensmaia, 2015). In an apparent motion study, Harrar and Harris (2007) found similar effects in visual and tactile apparent motion when examining Ternus effects, where perception shifts between group motion and element motion, but with different timing for shifts in the perceived ambiguous motion in the two modalities. The tactile stimulus (which were pin stimulations on the fingertips) took longer for the shift in ambiguous motion than the visual stimulus. Also, it was unaffected by the previous presentation of the stimulus or by the simultaneous presentation of the visual stimulus that was also located on the fingertips—this suggests that the two modalities may have independent grouping mechanisms related to the Ternus effect. Other studies, however, did find some interactions between the two modalities. Konkle et al. (2009) found that the motion after-effect can transfer between modalities through repeated exposure to a visual motion. Craig (2006) reported a substantial reduction in the accuracy of tactile apparent motion direction judgments when visual motion was presented at the same time as, but in a direction opposite to, tactile motion. This reduction in performance is the “congruency effect,” where synchronous visual and tactile stimuli with incongruent motion directions will reduce the ability to detect the correct direction of the tactile motion. It also seems that the integration of local motion information that supports perception of motion direction is computed similarly in both visual and tactile modalities (Pei, Hsiao, & Bensmaia, 2008).
Finally, in regards to 2D versus 3D computation of motion, Experiment 1 found no PSE differences among the three different arm configurations, suggesting that proximity is resolved in 3D space for tactile apparent motion. While it might seem reasonable that touch would be represented in spatiotopic (3D) rather than retinotopic (2D) coordinates, the same is not assumed for visual stimuli. Nevertheless, the near-far visual conditions of Experiment 2 revealed a clear influence of 3D spacing: The resulting PSEs were much closer than would be expected for the 2D visual projection of the stimulus locations. Experiment 4 showed that this 3D basis for the motion computation was not a consequence of the involvement of the hands but was evident even in this purely visual task. Rather than an expected 200% to 400% increase in PSEs with the change in screen angle that would be required for proximity in 2D coordinates, the change was only 10%. This outcome suggests that the apparent motion is computed in 3D coordinates whether the stimulus is purely visual, or visual on the hands, or purely tactile. The monitor was presented in a dimly lit room and viewed binocularly, so there were ample cues to the 3D orientation of the screen—if the room had been in total darkness, and the viewing monocular, the results would have necessarily favored the computation of proximity in 2D coordinates. Our study is the first to report that proximity in apparent motion is computed in 3D real-world coordinates rather than 2D retinal coordinates.
To summarize, we found that tactile motion showed an expansion of the spacing within the hand compared to that between the hands. This is consistent with the greater precision of movement and length judgment within the hand as opposed to across hands (Panday et al., 2014; Smeets & Brenner, 2001; Tresilian & Stelmach, 1997). It is opposite to what would be expected if stimuli on the same hand were grouped and therefore seen as being closer together (Coren & Girgus, 1980). Proprioception from the hands did affect judgments of visual motion (Experiment 2), but this is not the primary explanation for the 3D computation of proximity seen in Experiments 1 and 2. The final, purely visual experiment (Experiment 4) demonstrated that the proximity driving the apparent motion was based on 3D distances, as with the tactile stimuli in the previous experiments. These similarities across modalities suggest that the computations for apparent motion share a common set of mechanisms in both visual and tactile domains.
Footnotes
Acknowledgements
The authors thank Arielle Veenemans for assistance with data collection and Tatjana Seizova-Cajic for providing feedback on the design of the first experiment and for comments on this manuscript. The authors would also like to thank Kaykee Kantorowski, Abby Machado, Juno Wu, Elaine Yan, and Phoebe Yan for assistance in collecting and analyzing the data of Experiment 3. The authors also thank the two anonymous reviewers who provided valuable feedback to improve this manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by European Research Council under the European Union’s Seventh Framework Programme (FP7/2007–2013)/ERC grant agreement n°AG324070 to P. C.; Department of Psychological and Brain Sciences, Dartmouth College grant to P. C.; UCSD Department of Psychology grant to S. A.