Training Haptic Stiffness Discrimination

Abstract

Objective:

In this study, we explored the time course of haptic stiffness discrimination learning and how it was affected by two experimental factors, the addition of visual information and/or knowledge of results (KR) during training.

Background:

Stiffness perception may integrate both haptic and visual modalities. However, in many tasks, the visual field is typically occluded, forcing stiffness perception to be dependent exclusively on haptic information. No studies to date addressed the time course of haptic stiffness perceptual learning.

Method:

Using a virtual environment (VE) haptic interface and a two-alternative forced-choice discrimination task, the haptic stiffness discrimination ability of 48 participants was tested across 2 days. Each day included two haptic test blocks separated by a training block. Additional visual information and/or KR were manipulated between participants during training blocks.

Results:

Practice repetitions alone induced significant improvement in haptic stiffness discrimination. Between days, accuracy was slightly improved, but decision time performance was deteriorated. The addition of visual information and/or KR had only temporary effects on decision time, without affecting the time course of haptic discrimination learning.

Conclusion:

Learning in haptic stiffness discrimination appears to evolve through at least two distinctive phases: A single training session resulted in both immediate and latent learning. This learning was not affected by the training manipulations inspected.

Application:

Training skills in VE in spaced sessions can be beneficial for tasks in which haptic perception is critical, such as surgery procedures, when the visual field is occluded. However, training protocols for such tasks should account for low impact of multisensory information and KR.

Keywords

haptic interfaces virtual environment human performance learning stiffness perception training

Introduction

Haptic perception and skills in general are commonly assumed to improve with practice. Certain populations, such as musicians, people with visual impairments, and surgeons, are associated with superior haptic sensitivity, presumably attributable to use-dependent or experience-dependent neuroplastic mechanisms (Dinse, Wilimzig, & Kalisch, 2008). Repeated exposure to sensory experiences results in enhanced performance in perceptual tasks and plastic reorganization of the adult brain (e.g., Karni & Sagi, 1991). However, perceptual learning is most extensively studied in the visual and auditory domains (Goldstone, 1998).

Researchers examining the time course of perceptual learning in these domains indicate that at least two different learning processes are involved in perceptual learning: fast, within-session learning, which takes place online (when stimuli are still present or immediately after); and slow, offline learning, or consolidation, which occurs between sessions (Atienza, Cantero, & Dominguez-Marin, 2002; Karni & Sagi, 1993; Sagi & Tanne, 1994). Consolidation periods of at least 8 hr may be needed during wake and sleep time, depending on the task demands and design (Karni, Tanne, Rubinstein, Askenasy, & Sagi, 1992). Despite its importance, only a few studies deal with the time course of haptic perceptual learning (for example, see Adams, Kerrigan, & Graf, 2010, for visual recalibration from haptic feedback; Lacey, Pappas, Kreps, Lee, & Sathian, 2009; and Norman, Clayton, Norman, & Crabtree, 2008, for haptic object recognition; and Wagman, Shockley, Riley, & Turvey, 2001, for fast perception of shape characteristics), and none of them focuses on the perception of stiffness.

Haptics include all aspects of information acquisition and object manipulation through touch by humans, machines, or their combination; and the environments can be real, virtual, or teleoperated (Srinivasan, Beauregard, & Brock, 1996). Mechanical, different sensory, motor, and cognitive subsystems work together to create haptic percepts and memory (Lederman & Klatzky, 2009). While one is touching an object, the haptic system obtains tactile and kinesthesis information (information about the displacement of the arm together with signals of applied force; Clark & Horch, 1986). In addition, information regarding the displacement or deformation of an object, for example, the finger positions over time, may also be attained from the visual system (Lederman & Klatzky, 2009). These inputs give rise to a percept of the object’s stiffness. When there is redundant information from visual and haptic modalities, as both arise from the same physical event, multisensory integration presumably occurs (Stein & Meredith, 1993).

Stiffness sensitivity is essential for many complex tasks, including medical procedures, such as surgery and teleoperation (Howell, Conatser, Williams, Burns, & Eland, 2008; Sherman, Cavusoglu, & Tendick, 2000). However, in many surgical procedures, the visual field is typically occluded (for example, as in maxillofacial surgery [MFS]); thus the surgeon has to rely solely on the information obtained from the haptic system.

Theorists have frequently regarded the haptic modality as inferior to vision in terms of perceptual accuracy (Hecht & Reiner, 2009; Kritikos & Brasch, 2008; Srinivasan et al., 1996; for the concept of visual dominance, see Posner, Nissen, & Klein, 1976). Moreover, recent research on acquisition of perceptual-motor skills showed that reliance on visual information occurs even if a nonvisual strategy is advantageous (Yechiam & Gopher, 2008). In that case, training with reduced visual information enhanced performance in tasks in which reliance on haptic information was more efficient.

Conversely, recent studies suggest that multisensory experiences may enhance unisensory processing and memory (Lehmann & Murray, 2005; Murray et al., 2004; Seitz, Kim, & Shams, 2006), showing, for example, that repeated images are better discriminated if initially presented as auditory–visual pairs rather than only visually. In addition, von Kriegstein and Giraud (2006) found that voice recognition was improved by audiovisual training, proposing that this effect is not unique for unisensory visual enhancement. Shams and Seitz (2008) concluded that multisensory exposure assists unisensory learning through an activation mechanism in the brain; multisensory learning involves alteration of connections between modalities, so that later presentation of unisensory stimuli activates a wider, multisensory network of brain regions. This proposed mechanism of multisensory facilitation of unisensory learning, if generalized, suggests that visual-haptic (VH) training may enhance later haptic performance.

An additional training manipulation that is often argued for enhancing performance is “knowledge of results” (KR; also known as performance feedback). The origins of this assertion comes from Thorndike’s law of effect (Thorndike, 1927), which implies that responses that produce a satisfying effect become more likely to occur again, and responses that produce a discomforting effect become less probable. According to the law of effect, KR is required to facilitate learning, since satisfying or discomforting effects cannot be a result of practice repetition alone. However, in many visual perceptual tasks, performance was found to improve even when training did not include external KR (e.g., Fahle, Edelman, & Poggio, 1995; Karni & Sagi, 1991; for a review of visual perceptual learning, see Sagi & Tanne, 1994), suggesting that explicit feedback is not necessary for perceptual learning. The involvement of internal feedback signals in perceptual learning is further supported by studies (Ball & Sekuler, 1987; Shiu & Pashler, 1992) that showed that practice without KR improved performance in easy tasks but not in difficult ones (in which internal feedback signals might be ambiguous).

Nevertheless, training with external KR is very common, and there are many experimental findings showing an enhancing effect of KR on performance (e.g., Herzog & Fahle, 1997; Seitz, Nanez, Holloway, Tsushima, & Watanabe, 2006; for a review, see Adams, 1987). Yet, as indicated by Salmoni, Schmidt, and Walter (1984), many KR experiments failed to separate the temporary, transient effects of KR manipulations from their long-term effects on learning. They also showed that when KR was withdrawn, performance deteriorated.

In a recent project (Bouchigny et al., 2012), we have developed a surgical training platform for MFS. This surgery relies on excellent sensory-motor perceptual abilities underlying highly controlled tool interactions with tissues of different stiffness, primarily the jawbones, with a major reliance on haptics versus visual information. These tool–tissue interactions are of a complex multisensory nature, combining tactile, kinesthetic, and visual input: Changes in bone structure should be rapidly evaluated to detect the transition between bone layers featuring different stiffness. Following a previous study that addressed the effects of presentation order and sensory modality in a single-session training (Korman, Teodorescu, Cohen, Reiner, & Gopher, 2012), in the current study, we investigated the course of learning in haptic stiffness perception and how it is affected by different sensory conditions and information feedback during training for a specific range of stiffness values (captured during cadavers head surgery, not reported here) that need to be discriminated during MFS. Specifically, we examined, first, whether practice improves mean stiffness discrimination ability across 2-day training and, second, how the addition of KR and/or congruent visual information during training affects later haptic performance and learning.

Method

Participants

Participants were 48 Technion students. They were summoned for two experimental sessions in two consecutive days. Each session lasted approximately 1 hr. Prior to the commencement of testing, all participants were provided with an information sheet, and a consent form was signed. Participants were right-handed (as assessed by the Edinburgh Handedness Inventory, Oldfield, 1971) and had no medical conditions that could impair fine motor and sensory perception. Medicine students and other professionals specifically trained to palpate objects were excluded from this study. All participants were paid for participation, and additional bonuses were granted to participants who received the highest 20% scores, to uphold high motivation through the experiment.

Setup

The experiment was conducted with the use of a virtual environment (VE) touch-enabled computer interface capable of providing users with visual and haptic stimuli corresponding to varied stiffness intensities (stiffness is measured in N/mm and is the opposite of softness, sometimes referred to as “compliance,” which is measured in mm/N). The apparatus included a computer, monitor, 3-D eyeglasses, mouse, and the PHANTOM Desktop haptic device. This device makes it possible for users to touch and manipulate virtual objects with a penlike stylus arm gripped and moved as in handwriting (see Figure 1).

Figure 1.

Experimental setup: The right hand holds the PHANTOM’s stylus and the left hand is placed on the mouse. Two targets are presented on the computer screen and are seen directly through 3-D glasses. Neither the right nor the left hand is in the participant’s visual field while performing the task.

Task and Stimuli

To investigate subjective stiffness perception, we used a two-alternative forced-choice discrimination task (Gescheider, 1997): In each trial, two targets were presented on the screen as two red, square plates. Participants were asked to probe the targets with the stylus and determine which target is softer (less stiff). Participants held the PHANTOM’s stylus with their right, dominant, hand (see Figure 1). The location of the stylus and the visual size of the targets were updated in real time on the computer screen. Participants were free to switch between the targets as often as they wished and to probe each target up to three accumulated seconds (thereafter the target disappeared). The left, nondominant, hand was placed on the mouse to issue the participant’s answer to each trial.

Each stimulus pair included one constant baseline stiffness value of 0.25 N/mm, and one comparison value out of 11 possibilities. Participants were unaware that the same standard stimulus was presented in each trial. The location of the standard and comparison targets (left or right) was randomized and counterbalanced across trials. The comparison stimuli were selected to be nonlinearly distributed above the standard stimuli: (1) 0.256; (2) 0.263; (3) 0.270; (4) 0.278; (5) 0.286; (6) 0.294; (7) 0.312; (8) 0.333; (9) 0.357; (10) 0.384; (11) 0.417 N/mm, resulting in 11 difficulty levels (accordingly numbered 1 to 11 from the most difficult to the easiest comparison). These values were chosen such that the intervals between the difficult comparison values (close to the standard value) will be smaller than intervals between the easy comparison values, in order to provide the needed resolution to address possible improvements (assuming that improvement in easier comparison pairs will be biased to ceiling effects).

The range of values was chosen in a preliminary field study (not reported here) to reflect the real stiffness values that should be discriminated by surgeons during MFS (Bouchigny et al., 2012). Since the standard comparison stimulus was always the softest, participants could develop a strategy of recognition of the standard stimulus instead of discrimination between the two stimuli in each trial. As participants were not informed about the underlying relationship of the comparisons and did not report any regularities in stimuli pairs, this strategy is less likely to emerge, but the current setting does not allow ruling out this possibility.

Two types of sensory stimuli were used in the study: With haptic stimuli, during target probing, information about the level of stiffness could be acquired by pressing on the targets (with the virtual stylus). Participants felt the haptic feedback as a resistance of the stylus to the force they applied. No visual information was available (the visual appearance of the targets was kept constant, and the stylus disappeared when contacting the targets, to exclude visual feedback from the stylus movement). With VH stimuli, during target probing, matched visual feedback accompanied the haptic feedback. The additional visual information involved changes in the size of the targets in congruency with the application of force and the level of stiffness: For a given applied force, the size of the target changed more slowly for stiffer stimulus as compared with softer stimulus.

The targets were programmed with Open Haptic and Open GL software, with the use of a static haptic model (i.e., manipulating the stiffness parameter of a virtual spring beneath the solid nondeformable target square and providing visual feedback as a change in the size of the target linearly proportional to the force applied). The force applied by participants in perpendicular direction to the target surface was used to calculate the force feedback and visual change in target size (z direction in setup coordinate system), according to Newton’s third law. Target shift (dz) was proportional to the force applied by the subject, dz = F/k, where k is stiffness. The visual change in target size (in [x, y] plane) depended on this shift according to the laws of projective geometry of the VE. The parameters of projective transformation were as follows: far field, 4.0 mm; near field, 1.3 mm; field of view in Y direction, 44.69 grad; view direction (0, 0, –10); camera position (0, 1, 0).

Experimental Design

All participants performed three blocks of the task in each day (test, training, test; see Figure 2), separated by 5 min rest. Each block included 11 paired comparisons with the standard stiffness value; for each comparison, 10 trials were performed, altogether 110 trials per block (randomly ordered) and 330 trials per session.

Figure 2.

The time course of training and retests. Each day included a single session that consisted of three blocks: test, training, and retest.

Participants were tested on a haptic-only condition before and after the training blocks. During the training blocks, two variables were manipulated between groups (2 × 2 design): trial-by-trial KR (presented as green V for correct answer or red X for wrong answer) and the presence of congruent visual information, resulting in four groups of 12 participants in each, which differed only in the type of training provided (Figure 3).

Figure 3.

Experimental groups. The four conditions represent the type of training provided to participants in each one of the groups (12 participants in each group).

Data Analysis

The ability to discriminate stiffness was calculated as the proportion of correctly discriminated trials for a given comparison pair at each block (percentage correct [PC]). In addition, decision time (DT) was measured for each trial, describing the time from the first contact with one of the targets to the submission of the discrimination response. It is important to mention that there was no explicit request to minimize the DT.

We conducted mixed-measures ANOVA with 11 × 4 × 2 × 2 design (comparison’s difficulty, test block number, KR, VH) for the test blocks to address the time course of haptic learning. To address temporary effects of the training manipulation, we performed additional analysis separately for the two training blocks (ANOVA with 11 × 2 × 2 × 2). These analyses enabled the examination of possible interaction effects between the amount of practice (within and between sessions), difficulty level, and the short- and long-term effects of training with visual cues and/or KR.

Results

Learning Effects Across Haptic Tests

PC

The PC analysis of performance during the haptic test blocks revealed no effects for the type of training (interactions with VH and KR were not significant, nor were main effects of these factors). This result suggests that the addition of visual information and/or trial-by-trial KR during training did not affect haptic discrimination ability. Interestingly, a significant interaction was found between the effect of practice and difficulty levels, F(30, 1320) = 1.5, p = .047, $η_{p}^{2}$ = .03, suggesting that practice effects were not equal across the different comparisons. We investigated the significant interaction further by evaluating the simple effects of practice separately for each difficulty level. The results of the simple effect analysis revealed that whereas in Comparisons 3 through 11, practice effect was significant, in the two most difficult comparisons, it was not (see Figure 4 for a graph and Table 1 for details of the simple effects results).

Figure 4.

Mean percentage correct across all groups as a function of difficulty level at Test 1 and Test 2 (Day 1) and Test 3 and Test 4 (Day 2), with error bars indicating standard error.

Table 1.

Results of the Simple Effects Analysis and the Following Pairwise Comparisons

Difficulty Level	Simple Practice Effect			Improvement Within 1st Day		Improvement Between Days		Improvement Within 2nd Day
	F(3, 1320)	p	$η_{p}^{2}$ Effect Size	Mean Improvement (%)	Cohen’s d Effect Size	Mean Improvement (%)	Cohen’s d Effect Size	Mean Improvement (%)	Cohen’s d Effect Size
1 (hardest)	0.86	.46	.01	2.29	.15	0.83	.06	0.21	.01
2	2.38	.07	.03	−1.88	−.13	2.08	.13	3.96	.24
3	6.37^a	.0003	.09	1.04	.07	5.83^b	.39	1.46	.09
4	7.14^a	<.0001	.12	7.71^b	.52	1.25	.08	0.42	.03
5	12.61^a	<.0001	.18	11.67^b	.83	0.42	.03	−0.83	−.05
6	9.99^a	<.0001	.14	2.92	.17	6.88^b	.39	0.83	.05
7	10.51^a	<.0001	.16	7.29^b	.42	2.71	.17	2.29	.15
8	6.58^a	.0002	.13	6.04^b	.40	2.92	.21	0.00	.00
9	12.65^a	<.0001	.28	8.33^b	.61	2.50	.23	2.71	.29
10	8.25^a	<.0001	.30	6.46^b	.56	3.13^b	.36	0.83	.11
11 (easiest)	6.51^a	.0002	.20	7.08^b	.56	0.63	.06	1.88	.19
Means across significant simple practice effects			.18	6.5	.46	2.9	.22	1.1	.1

Note. The last row presents the averages across all difficulty levels in which the simple practice effect was significant (Levels 3-11).

Significant simple practice effects, according to Bonferroni-corrected alpha of .004.

Represents significant improvement, according to Bonferroni-corrected alpha of .015.

To better understand the effect of practice on performance, we made three pairwise comparisons (for each simple practice effect), examining improvement within the 1st day (Test 1 to 2), between days (Test 2 to 3), and within the 2nd day (Test 3 to 4). As can be seen in Table 1, significant practice effects resulted mainly from improvements within the 1st day (average of 6.5%, medium effect size) and between the 2 days (average of 2.9%, small effect size). No significant improvements were observed during the 2nd day. Taken together, the observed practice effects imply two distinctive consequences of the first training—online, within-session learning and offline consolidation—but these were not equal across difficulty levels and were absent in the most difficult comparisons.

DT

In terms of improvements in DT, again, no effects for the type of training were found (interactions with VH and KR were not significant, nor were main effects of these factors). The interaction between practice and difficulty level was significant, F(30, 1320) = 2.27, p < .001, $η_{p}^{2}$ = .05, suggesting that practice effects of DT were not equal across the difficulty levels. The simple practice effects in all the difficulty levels were highly significant, F(3, 1320) > 155, p < .0001, for all levels. Figure 5 presents DT across difficulty levels and across the test blocks.

Figure 5.

Mean session decision time (in seconds) across all groups as a function of difficulty level at Test 1 and Test 2 (Day 1) and Test 3 and Test 4 (Day 2), with error bars indicating standard error.

To further understand the practice effects in each difficulty level, we conducted pairwise comparisons tests. This analysis revealed significant improvements within the 1st day in all difficulty levels (p < .01), with the magnitude of the improvement increasing as the comparison is easier (average improvement of 1.09 s in the most difficult comparison to 1.43 s in the easiest; Cohen’s d also increased from 0.6 to 0.9, respectively). Interestingly, much of this improvement was lost at the pretraining test of the 2nd day, across all difficulty levels (p < .01). Here too, the effect was not equal across difficulty levels (the range of loss was 0.59 s to 0.74 s, Cohen’s d = 0.38 to 0.51), but there was not a clear pattern across the different difficulty levels. During the 2nd day, participants gained back this loss across all difficulty levels (improvement within the 2nd day of 0.69 s to 0.87 s, Cohen’s d of 0.42 to 0.61, without clear trend across the levels) but did not improve significantly beyond the last test in the 1st day (p > .3 for all levels). Altogether, the DT results show adaptive improvements within session in each day and a significant partial loss between days. The magnitude of these effects is varied across different difficulty levels, and a trend of increased improvements for easier comparisons was observed only during the 1st day.

To summarize, in contrast to the PC results, the DT findings suggest that within-session gains in DT are mainly adaptive and do not show between-session savings in performance. However, as mentioned before, participants were given incentives to exclusively follow the accuracy criterion; no intentional training on DT was induced.

Temporary Effects During Training Blocks

PC

The PC analysis of performance during the training blocks showed minor effects of the training type. These effects were reflected in two significant three-way interactions (the four-way interaction and other three-way interactions were insignificant). The first interaction is between practice, difficulty, and VH, F(10, 440) = 2.6, p = .004, $η_{p}^{2}$ = .06, suggesting that VH effect was not equal across difficulty levels and across the two training blocks. We investigated this interaction by evaluating the simple interaction effects of Difficulty × VH separately for each training block. According to Bonferroni-corrected alpha of .025, the Difficulty × VH interaction was not significant for the first training block, F(10, 440) = 1.95, p = .04, $η_{p}^{2}$ = .03, nor was main effect for VH (p > .1). This result suggests that VH affected performance significantly only in the second training block, and indeed, the Difficulty × VH interaction was significant in the 2nd day, F(10, 440) = 2.53, p = .006, $η_{p}^{2}$ = .05).

To further understand the Difficulty × VH interaction in the 2nd day, we examined simple VH effects in each difficulty level. This analysis revealed that training with VH had mixed effects on performance: VH improved performance in most difficulty levels, but in Comparisons 5 and 7, it decreased performance. However, these simple VH effects were not significant according to Bonferroni-corrected alpha of .004 (p > .05 for all comparisons except Comparison 5, in which p = .02). Taken together, the first three-way interaction reveals that VH training had mixed effects on performance in different comparisons, which was significant only for the second training block (Day 2).

The second three-way interaction that was significant in the general training analysis was Difficulty × VH × KR, F(10, 440) = 2.4, p = .01, $η_{p}^{2}$ = .05. This interaction suggests that the effect of KR was not equal across difficulty levels and was dependent on whether participants received visual cues (VH) or not. Analysis of simple Difficulty × KR interactions was conducted for training with and without visual information separately. In both cases (with and without visual cues), the addition of KR did not have a significant effect (the interaction Difficulty × KR failed to reach significance, as did main effects for KR, p > .2 for all). However, the effect size of the nonsignificant Difficulty × KR interaction with visual cues was less than half compared with the same interaction in training without the addition of the visual information ( $η_{p}^{2}$ = .04 and $η_{p}^{2}$ = .09, respectively). Thus, it seems that the effect of KR on PC performance was complex (varied across difficulty levels in dependence with the addition of visual information) as well as minor and/or noisy (simple KR interactions and main effects were not significant, even when the effect size was medium).

DT

The DT analysis of the training blocks revealed a significant effect of KR on performance (see Figure 6). The interaction between practice, difficulty level, and KR was significant, F(10, 440) = 2.08, p = .02, $η_{p}^{2}$ = .05. Accordingly, simple Difficulty × KR interactions were examined for each training block. In the first training block, the Difficulty × KR interaction was significant, F(10, 440) = 3.45, p = .0002, $η_{p}^{2}$ = .05, suggesting that the effect of KR on DT was not equal across difficulty levels. Further pairwise comparisons revealed a robust decrease in DT for training with KR across all difficulty levels (DT with KR < DT without KR in all comparisons). This decrease varied across comparisons with only marginal significance with Bonferroni-corrected alpha of .004 (.01 < p < .05 for Comparisons 2, 3, 5, 6, 7, and 10; .05 < p < .1 for Comparisons 1, 4, 8, 9, and 11). However, the average difference was 0.8 s (SD = 0.14) and the effect sizes were medium (mean Cohen’s d = .61, SD = .08). In the second training block, the simple Difficulty × KR interaction was not significant, but the main effect of KR in this analysis was highly significant, F(1, 440) = 908, p < .0001, $η_{p}^{2}$ = .07. Altogether, the DT results suggest that training with KR reduces DT within session. As can be seen in Figure 6, this facilitation effect is larger in the first training block, and seems to be less pronounced for easier comparisons, whereas during the second training, the KR effect remains about the same across comparisons.

Figure 6.

Interaction between training blocks, difficulty level, and the addition of KR. The graphs show the mean decision time across difficulty levels and across the two training blocks. To the left, decision times during the first training block (in Day 1) are presented, and to the right are decision times in the second training block (in Day 2).

Examination of temporary effects of training with visual cues showed a significant interaction between VH and practice, F(1, 44) = 5.08, p = .03, $η_{p}^{2}$ = .1, suggesting that the effect of VH was not equal in the two training sessions. The simple effects analysis further revealed slower decisions for VH training in the first block (difference = 0.8 s, p = .047). Following Bonferroni-corrected alpha of .025, this difference is not significant, yet the effect size is medium (Cohen’s d = 0.6). However, in the second training block, this difference disappeared (difference between VH training to no-VH training = −0.16, p > .5, Cohen’s d = 0.1).

Discussion

We used a stiffness discrimination task to study the effects of practice in haptic perceptual learning. The results provided evidence for both immediate and latent learning, reflected by within- and between-sessions improvements in mean haptic stiffness discrimination ability. This finding outlines the principle similarity of haptic perceptual learning to visual and auditory perceptual learning, suggesting that the two distinct learning processes, online and offline, observed in other modalities also underlie haptic stiffness perceptual learning. We found that practice did not induce similar gains at all difficulty levels. Performance of the most difficult comparisons did not improve at all, whereas discrimination of all other comparisons indeed improved.

The improvements on accuracy and DT in stiffness discrimination were shown to have distinct time courses. Within the first session, both the accuracy and the DT improved, showing a robust and concurrent immediate learning. At the beginning of the 2nd day, participants were more accurate but also slower (compared with the end of the 1st day). Within the second session, DT was improved again and slightly beyond the gain during the first session, but accuracy of discrimination remained stable.

These results show that the time course of stiffness discrimination learning is a multistage experience and time-dependent process, analogous to perceptual and motor types of learning (e.g., Karni & Sagi, 1993; Korman, Raz, Flash, & Karni, 2003). Discrepancies in the time courses of accuracy and DT measures are likely to be explained by the fact that no intentional training on DT was induced and participants were asked to perform the discrimination as accurately as possible without addressing the DT issue. This method could cause triggering of two different learning processes: long-term learning of accuracy of discrimination and only adaptive and thus transient improvements in DT. The finding that, between sessions, accuracy was improved but DT slowed down may also reflect a speed–accuracy trade-off (e.g., Wickelgren, 1977), suggesting that at least in part, this improvement in discrimination ability was a result of the increased DT.

The current results showed that both training manipulations that were suggested to facilitate haptic learning—the information feedback (KR) and the multimodal stimulus presentation—had only transient effects during the training blocks themselves and did not affect later haptic stiffness perception. Within the training blocks, the addition of KR resulted in faster decisions, in line with a study from fast haptic perceptual learning (Wagman et al., 2001), which showed that exploratory behavior decreased in duration when KR was present. On the other hand, the addition of visual information resulted in slower decisions. This result appears not to be in line with previous findings (e.g., Hecht & Reiner, 2006) that showed shorter RT for detection of multimodal as compared with unimodal stimuli. However, faster processing of multimodal signals does not necessarily imply that faster discrimination decisions will be made, given different underlying mechanisms involved in detection and discrimination tasks (Sagi & Julesz, 1985).

In addition, VH training resulted in better stiffness discrimination for easy comparisons during the training itself, but this improvement did not transfer to the haptic-only, posttraining retests. This finding challenges the general notion that multisensory training facilitates unisensory learning. This assertion mainly relies on studies showing an enhancement of visual unisensory performance after multisensory training. Nevertheless, as noted by Yechiam and Gopher (2008), visual dominance can cause impaired training effectiveness when the haptic aspects of the task are more important. The only study that demonstrated an advantage for multisensory training on nonvisual unisensory performance (von Kriegstein & Giraud, 2006) involved a voice recognition task, in which associations were explicitly memorized. However, in haptic stiffness discrimination, the role of explicit memory is expected to be minimal. Thus, possibly the involvement of declarative systems in the encoding of the stimuli (given that the task allows such translation in task representation) may mediate the proposed facilitatory effect of multisensory training on unisensory perception.

To conclude, our results suggest that haptic stiffness discrimination learning is a multistage process that does not necessarily benefit from multisensory training. Feedback information has an impact on fast, within-session performance in terms of DT but not on multisession training. These findings may have an important applied meaning in the development and optimization of VE training systems for perceptual motor tasks relying on stiffness perception, such as surgery. In these systems, any combination of modalities can be synthesized for VE training, but successful transfer to the real world condition is critical. This concern was recently raised by Tsuda, Scott, Doyle, and Ones (2009), who proposed that training complex manual skills, such as surgery, in VE needs to be carefully structured and that sensory feedback is one of the key factors throughout the development of a skill. Our findings suggest that training haptic stiffness discrimination in spaced sessions can be beneficial for tasks in which the haptic perception is critical, for example, surgery procedures with occlusion of the visual field, as in MFS. However, training protocols for such tasks should account for low impact of multisensory information and information feedback on improvements in stiffness discrimination.

Key Points

Practice induces significant and persisting multistage improvements in haptic stiffness discrimination ability.

Differential consolidation effects (between days) were found: Whereas accuracy improved, decision time performance partially deteriorated.

The addition of visual information and/or knowledge of results had only temporary effects on decision time, without affecting the time course of haptic discrimination learning.

Footnotes

Acknowledgements

The authors would like to thank Daniel Gopher for his support and Tami Gelfeld and Miriam Reiner for software development.

This work was supported by the SKILLS Integrated Project (IST-FP6 No. #035005; ) funded by the European Commission.

Kinneret Teodorescu was born in Israel in 1982. She received a BA in psychology and economics from Tel-Aviv University in 2008 (magna cum laude) and an MSc in human factors and psychology in the Department of Industrial Engineering from the Technion–Israel Institute of Technology in 2010 (summa cum laude). She is studying toward a PhD in cognitive psychology at the Technion. She participated as a researcher at the European SKILLS project (2009–2011), which explored the acquisition, interpretation, storing, and transfer of human skills by means of multimodal interfaces and virtual environment technologies. Her research interests include learning in simple and complex environments, acquisition of skills, decision making, and quantitative models of human behavior.

Sylvain Bouchigny was born in France in 1974. He received his MEng in scientific instrumentation from the National Engineering School of Caen (ENSI de Caen, France) in 1998 and his PhD in physics from the University of Paris 11 (Orsay, France) in 2004. In 2007, he joined the CEA LIST as a research engineer on physics applied to human interaction. He acted as a project manager on human–machine interface for micromanipulation and surgical applications in real-time interactive virtual environments. He has been a project manager for CEA LIST within the SKILLS project since 2010.

Maria Korman was born in Russia in 1975. She received her BSc degree from the Department of Biology, Ben-Gurion University of the Negev, Israel, in 1995 and her MSc and PhD in 1999 and 2004 from the Neurobiology Department, Weizmann Institute of Science, Rehovot, Israel. She did 4 years of postdoctoral training at the Functional Neuroimaging Unit, University of Montreal, Quebec, Canada. From 2008, she worked as a senior research associate at the Technion–Israel Institute of Technology. Recently she joined the Department of Occupational Therapy at Haifa University. Her research interests include motor learning and control, brain imaging, sleep, aging, and rehabilitation.

References

Adams

J. A.

(1987). Historical review and appraisal of research on the learning, retention, and transfer of human motor skills. Psychological Bulletin, 101, 41–74.

Adams

W. J.

Kerrigan

I. S.

Graf

E. W.

(2010). Efficient visual recalibration from either visual or haptic feedback: The importance of being wrong. Journal of Neuroscience, 30, 14745–14749.

Atienza

Cantero

J. L.

Dominguez-Marin

(2002). The time course of neural changes underlying auditory perceptual learning. Learning & Memory, 9, 138–150.

Ball

Sekuler

(1987). Direction-specific improvement in motion discrimination. Vision Research, 27, 953–965.

Bouchigny

Hoffmann

Mégard

Gosselin

Korman

(2012). Designing a VR training platform for surgeons: Theoretical framework, technological solutions and results. In Bergamasco

Bardy

B. G.

Gopher

(Eds.), Skills training in multimodal virtual environments. London, UK: Taylor & Francis.

Clark

F. J.

Horch

K. W.

(1986). Kinesthesia. In Boff

Kaufman

Thomas

(Eds.), Handbook of perception and human performance (pp. 13:1–13:62). New York, NY: Wiley.

Dinse

H. R.

Wilimzig

Kalisch

(2008). Learning effects in haptic perception. In Grunwald

(Ed.), Human haptic perception: Basics and applications (pp. 165–182). Basel, Switzerland: Birkhäuser Verlag.

Fahle

Edelman

Poggio

(1995). Fast perceptual learning in hyperacuity. Vision Research, 35, 3003–3013.

Gescheider

G. A.

(1997). Psychophysics: The fundamentals (3rd ed.). Mahwah, NJ: Lawrence Erlbaum.

10.

Goldstone

R. L.

(1998). Perceptual learning. Annual Review of Psychology, 49, 585–612.

11.

Hecht

Reiner

(2006). Multimodal virtual environments: Response times, attention, and presence. Presence, 15, 515–523.

12.

Hecht

Reiner

(2009). Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental Brain Research, 193, 307–314.

13.

Herzog

M. H.

Fahle

(1997). The role of feedback in learning a vernier discrimination task. Vision Research, 37, 2133–2141.

14.

Howell

J. N.

Conatser

R. R.

Williams

R. L.

Burns

J. M.

Eland

D. C.

(2008). Palpatory diagnosis training on the virtual haptic back: Performance improvement and user evaluations. Journal of the American Osteopathic Association, 108, 29–36.

15.

Karni

Sagi

(1991). Where practice makes perfect in texture discrimination: Evidence for primary visual cortex plasticity. Proceedings of the National Academy of Sciences, 88, 4966–4970.

16.

Karni

Sagi

(1993). The time course of learning a visual skill. Nature, 365, 250–252.

17.

Karni

Tanne

Rubinstein

B. S.

Askenasy

J. J. M.

Sagi

(1992). No dreams-no memory: The effect of REM sleep deprivation on learning a new perceptual skill. Society for Neuroscience Abstracts, 22, 387.

18.

Korman

Raz

Flash

Karni

(2003). Multiple shifts in the representation of a motor sequence during the acquisition of skilled performance. Proceedings of the National Academy of Sciences, 100, 12492–12497.

19.

Korman

Teodorescu

Cohen

Reiner

Gopher

(2012). Effects of order and sensory modality in stiffness perception. Presence, 21, 295–304.

20.

Kritikos

Brasch

(2008). Visual and tactile integration in action comprehension and execution. Brain Research, 1242, 73–86.

21.

Lacey

Pappas

Kreps

Lee

Sathian

(2009). Perceptual learning of view-independence in visuo-haptic object representations. Experimental Brain Research, 198, 329–337.

22.

Lederman

S. J.

Klatzky

R. L.

(2009). Haptic perception: A tutorial. Attention, Perception, & Psychophysics, 71, 1439–1459.

23.

Lehmann

Murray

M. M.

(2005). The role of multisensory memories in unisensory object discrimination. Cognitive Brain Research, 24, 326–334.

24.

Murray

M. M.

Michel

C. M.

Grave

Peralta

Ortigue

Brunet

Andino

S. G.

Schnider

(2004). Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging. Neuroimage, 21, 125–135.

25.

Norman

J. F.

Clayton

A. M.

Norman

H. F.

Crabtree

C. E.

(2008). Learning to perceive differences in solid shape through vision and touch. Perception, 37, 185–196.

26.

Oldfield

R. C.

(1971). The assessment and analysis of handedness: The Edinburgh inventory. Neuropsychologia, 9, 97–113.

27.

Posner

M. I.

Nissen

M. J.

Klein

R. M.

(1976). Visual dominance: An information-processing account of its origins and significance. Psychological Review, 83, 151–171.

28.

Sagi

Julesz

(1985). Detection versus discrimination of visual orientation. Perception, 14, 619–628.

29.

Sagi

Tanne

(1994). Perceptual learning: Learning to see. Current Opinion in Neurobiology, 4, 195–199.

30.

Salmoni

A. W.

Schmidt

R. A.

Walter

C. B.

(1984). Knowledge of results and motor learning: A review and critical reappraisal. Psychological Bulletin, 95, 355–386.

31.

Seitz

A. R.

Kim

Shams

(2006). Sound facilitates visual learning. Current Biology, 16, 1422–1427.

32.

Seitz

A. R.

Nanez

J. E.

Holloway

Tsushima

Watanabe

(2006). Two cases requiring external reinforcement in perceptual learning. Journal of Vision, 6, 966–973.

33.

Shams

Seitz

A. R.

(2008). Benefits of multisensory learning. Trends in Cognitive Sciences, 12, 411–417.

34.

Sherman

Cavusoglu

M. C.

Tendick

(2000, November). Comparison of teleoperator control architectures for palpation task. Paper presented at the International Mechanical Engineering Congress and Exposition, Orlando, FL.

35.

Shiu

Pashler

(1992). Improvement in line orientation discrimination is retinally local but dependent on cognitive set. Perception & Psychophysics, 52, 582–588.

36.

Srinivasan

M. A.

Beauregard

G. L.

Brock

D. L.

(1996). The impact of visual information on the haptic perception of stiffness in virtual environments. In Proceedings of the ASME Dynamic Systems and Control Division, 58, 555–559.

37.

Stein

B. E.

Meredith

M. A.

(1993). The merging of the senses. Cambridge, MA: MIT Press.

38.

Thorndike

E. L.

(1927). The law of effect. American Journal of Psychology, 39, 212–222.

39.

Tsuda

Scott

Doyle

Ones

D. B.

(2009). Surgical skills training and simulation. Current Problems in Surgery, 46, 271–370.

40.

von Kriegstein

Giraud

A. L.

(2006). Implicit multisensory associations influence voice recognition. PLoS Biology, 4, 1809–1820.

41.

Wagman

J. B.

Shockley

Riley

M. A.

Turvey

M. T.

(2001). Attunement, calibration, and exploration in fast haptic perceptual learning. Journal of Motor Behavior, 33, 323–327.

42.

Wickelgren

(1977). Speed–accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85.

43.

Yechiam

Gopher

(2008). Deliberate suppression of vision as a training tool for multimodal skills. In Avizzano

C. A.

(Ed.), SKILLS: Beyond movement. The history and future of gesture analysis (pp. 80–103). Copenhagen, Denmark: Alinea.