Abstract
Objective
Simulation has assumed a prominent role in education. It is important to explore the effectiveness of different modalities. In this article, we directly compare surgical resident impression of 2 distinct temporal bone simulations (physical and haptic).
Study Design
Research Ethics Board–approved prospective cohort study.
Setting
A haptic voxel-based virtual model (VM) and a physical 3-dimensional printed temporal bone model (PBM) were developed. Participants rated each construct on a number of parameters and performed a direct comparison of the simulations using a survey instrument that employed a 7-point Likert scale and rank lists.
Subjects and Methods
Ten otolaryngology residents dissected anatomically identical, matched physical and virtual models. Data for both simulations originated from 10 unique cadaveric micro–computed tomography images.
Results
Subjects rated the PBM drill quality as being more similar to cadaveric temporal bone than the VM (cortical bone mean: 5.5 vs 3.2, P = .011; trabecular bone mean: 5.2 vs 2.8, P = .004) and with better air cell system representation (mean: 5.4 vs 4.5, P = .003). Subjects strongly agreed that both simulations are effective educational tools, but they rated the PBM higher (mean: 6.7 vs 5.4, P = .019). Notably, subjects agreed that both modalities should be integrated into training, but they were more favorably inclined toward the PBM (mean: 7.0 vs 5.5, P = .002). In direct comparison, the PBM was the preferred simulation in 7 of 9 educational domains.
Conclusions
Appraisal of a PBM and a VM found both to have perceived educational benefit. However, the PBM was considered to have more realistic physical properties and was considered the preferred training instrument.
The cadaveric temporal bone laboratory was pioneering and continues to be a gold standard in virtual surgery. Unfortunately, the challenge in obtaining viable specimens continues to increase. 1 Furthermore, learning is inhibited by work-hour restrictions, monetary pressures, and concerns regarding patient safety. Educational and public safety needs may be best addressed if trainees are permitted mistakes without associated patient risk exposure. Fortuitously, technology now affords some much-needed adjuncts, with promising innovations in image segmentation, computer graphics, haptics, and rapid prototyping and with an expanding body of literature.
Printed temporal bone models (PBMs) using 3-dimensional printing technologies have long been described in the literature.2-5 Initially, they represented an accurate anatomic facsimile; however, the technology is now able to provide a dissectible model that includes select postprocessed soft tissues. Our initial research focus was on internal anatomic fidelity with precise void space generation associated with realistic bone structure and mechanical properties ( Figure 1 ).6,7 The presumptive advantage of the PBM is osseous realism: providing a drill-model contact experience that is highly representative of actual surgery.

Printed temporal bone model: A, Printed temporal bone model in the surgical position. Canal wall up mastoidectomy with posterior tympanotomy has been performed, and the sigmoid sinus (SS), incus (arrowhead), and facial nerve are evident (long arrow). B, Printed temporal bone model visualization from the posterior fossa. The model is derived from micro–computed tomography data with evident porus acousticus, endolymphatic sac (arrowhead), sigmoid sinus (SS), and superior petrosal sinus (long arrow).
Virtual models (VMs) with haptic feedback employ a robotic system to apply forces through an end effector (manipulandum). The user sees a graphical representation of the bone and feels virtual contact forces while holding the haptic manipulandum like an otic drill. Several of these systems are currently available and represent viable training opportunities.8-13 Our VM is similar to previous systems but employs a multicore architecture to accelerate graphic rendering and a distinct voxel-based collision detection and position-locking algorithm to improve drill-bone interaction ( Figure 2 ).14,15 This is in contrast to the commonly used virtual spring model. 16 The apparent advantage of a VM is the ability to objectively monitor and assess trainee actions, providing a basis for formative and summative assessment. Additionally, virtual simulation can be less expensive since costs are predictive without need for a drill system, microscope, and disposables (PBM and drill burr [or burrs]). A drawback of most haptic VMs is their inability to generate surfaces with a stiffness approaching that of cadaveric bone, rendering drill contact forces insufficient during simulation.

Virtual haptic graphic surgical simulation: A, Visualization of the virtual model in the surgical position. The surface contour is generated by Laplacian smoothing and does not appear voxellated. B, Three-dimensional simulation of a canal wall up mastoidectomy. The sigmoid sinus (blue) and carotid artery (red) are apparent. Shadowing of the drill facilitates appreciation of depth.
Our research group has generated both a PBM and a VM for the purposes of resident education and preoperative surgical rehearsal, with long-term goals of competency-based assessments, accreditation examinations, and continuing medical education. While both models may provide successful adjuncts to cadaveric simulation, there has not yet been a direct comparison between modalities. Both simulations present unique virtues and challenges. Depending on perceived educational requirements, it is possible that one system or the other may provide the most useful platform. An understanding of the end user’s perception of the relative merits of each system may guide the direction of future postgraduate education.
In the following, we describe a direct comparison of anatomically identical virtual haptic and 3-dimensional printed simulations derived from the same cadaveric computed tomography data.
Methods
After approval by the university’s Research Ethics Board, 10 resident trainees (from postgraduate year 1 to 5) from the Department of Otolaryngology at the University of Manitoba were recruited to compare surgical training on a PBM and a VM.
Each resident was randomly assigned a unique cadaveric specimen to drill. Data from that distinct cadaveric specimen were then used to create a matching PBM and VM. Residents then randomly dissected either the PBM or the VM, followed by drilling of the alternate model derived from the same assigned cadaveric specimen. Thus, each resident drilled the same anatomy in 3 formats (cadaveric, PBM, and VM), with each set of samples derived from a different anatomic specimen. There was no set time limit for the session, although all subjects completed the exercise in <4 hours.
Following their surgical session, subjects were asked to complete a survey instrument (Likert scale) comparing the VM and PBM experience to cadaveric simulation (see appendix at www.otojournal.org/supplemental). The survey asked subjects to rate physical characteristics, specific anatomic feature representation, usefulness in surgical skills training, and perceived educational value. Participants also directly contrasted both simulations across a range of holistic measures (binary forced choice) and completed a ranking of preferred educational qualities in any form of simulation. Survey data were analyzed using SPSS 22.
Data Acquisition
To create each cadaveric template, 10 human cadaveric temporal bone specimens underwent micro–computed tomography using a SkyScan 1176 microtomograph (Bruker, Billerica, Massachusetts). Image resolution was initially 35 µm but downsampled to 140 µm to match 3-dimensional printer resolution.
Micro–computed tomography data were then segmented using Mimics 14.0.1.7 (Materalize, Leuven, Belgium) into separate structural features. Bone was segmented semiautomatically using Hounsfield unit thresholds. Soft tissue features were manually segmented, including carotid artery, sigmoid sinus, dural plates, endolymphatic sac, endolymphatic duct, otic capsule contents, greater superficial petrosal nerve, chordae tympani, and facial nerve. Segmented features were stored as an individual polygon mesh.
Preparation of Virtual Haptic Models from Cadaveric Specimens
A virtual simulation of each cadaveric specimen was then generated by recombining its individual polygon mesh models into a single voxellated model. 14 Each cadaveric bone specimen therefore had a corresponding haptic simulation that contained anatomy identical in size and shape (isomorphic) to the parent bone.
The simulation used a haptic device (Geomagic Touch, Rock Hill, South Carolina) to control a virtual drill during interaction with the voxellated model. The model was visually displayed on a 165-cm plasma screen (Panasonic TCP65VT30, Panasonic, Osaka, Japan) mounted above and behind the haptic device (1280 × 720 pixel resolution). The drill was activated using an on-off foot-pedal (Scythe, Tokyo, Japan).
Preparation of Printed Bone Models from Cadaveric Specimens
A PBM was generated utilizing the same cadaveric data used for each matched VM pair. To generate the requisite void spaces, multiple polygon meshes, representing soft tissue and bone, were combined into 1 voxellated model. Based on a proprietary algorithm, the computer model was manually sliced into layers. Alignment fiducials were digitally added for postprint reassembly. The slices are output as a series of triangular meshes created by the marching cubes algorithm. 17 Each slice was then printed separately. The void spaces were cleared of remnant material using compressed air, and the slices were infiltrated with a binding agent (cyanoacrylate with hydroquinone) and then recombined to produce a final physical model. 7 The PBM was dissected with an otic drill (Stryker, Kalamazoo, Michigan) and stereo microscope (Zeiss, Jena, Germany).
Results
The mean and standard deviation of resident responses can be seen in the tables for each survey component. Participants considered several of the mechanical properties of the PBM to be more comparable to cadaveric bone than those of the VM ( Table 1 ). Significant differences between the models exist in cortical (P = .011) and trabecular (P = .004) osseous realism, vibrational properties (P = .001), and air cell system generation (P = .003).
Resident Assessment of Virtual Model and Printed Bone Model: Physical Properties as Compared to Cadaveric Bone. a
Values are presented as mean comparison rating ± SD (1 = very dissimilar, 7 = very similar). P values are based on analysis of variance.
P < .05.
Anatomic features were well regarded for both the VM and the PBM ( Table 2 ) with no significance between the modalities.
Resident Assessment of Virtual Model and Printed Bone Model: Anatomic Feature Similarity to Cadaveric Bone. a
Abbreviation: SCC, semicircular canal.
Values are presented as mean similarity rating ± SD (1 = very dissimilar, 7 = very similar). P values are based on analysis of variance.
Residents generally considered both simulations as a productive resource in acquiring surgical skill and approaches to the temporal bone ( Table 3 ). The PBM was ranked significantly more effective for learning cortical mastoidectomy (P = .015) and the posterior tympanotomy/facial recess approach to the middle ear (P = .023).
Resident Perceived Value of Virtual Model and Printed Bone Model in Surgical Skill Acquisition. a
Abbreviations: IAC, internal auditory canal; SCD, semicircular canal dehiscence.
Values are presented as mean value ± SD (1 = not beneficial, 7 = very beneficial). P values are based on analysis of variance.
P < .05.
Residents found the PBM as more educationally effective in certain domains ( Table 4 ). Significance was realized in statements indicating that the training instrument was effective (P < .019), that it should be integrated into resident education (P < .002), and that training with the model would improve confidence during surgery (P < .043).
Resident Appraisal of Virtual Model and Printed Bone Model: Educational Value. a
Values are presented as mean agreement level ± SD (1 = strongly disagree, 7 = strongly agree). P values are based on Mann-Whitney.
P < .05.
In a direct comparison of both simulations, the PBM was considered the superior tool in 7 of 9 domains (P < .05; Table 5 ). Noteworthy differences exist in overall realism, function as an educational tool, use in surgical preoperative rehearsal, learning the mechanics of temporal bone otic drill use, and as a composite preferred simulation. The 2 domains where significance was not attained were for “ease of use” and “use as a visual learning tool.”
Direct Comparison of Simulation Superiority.
Printed bone model/virtual model.
Binomial test.
P < .05.
Discussion
This is a unique study comparing 2 distinct forms of simulation to the cadaveric lab and directly to one another. While some groups have compared VMs to cadaveric dissection, to the best of our knowledge, no direct comparison between PBM and VMs has previously been undertaken.18-21 In consideration of the enormous resources currently being invested in training and in the generation of adjunctive tools, it is imperative that a holistic review of temporal bone simulation technologies be undertaken.
The importance of improved surgical training is increasingly more salient, with increasing focus on surgical outcomes and the pending evolution to competency-based resident education. The results of this study find that both simulations will likely help in the production of competent surgeons, as perceived by the trainees. However, across many domains, the PBM was the preferred simulation. This is likely attributable to the osseous drill qualities of the PBM and the inability of virtual systems in general to generate sufficiently realistic bone stiffness.
It is important to identify that the assessment was undertaken by resident surgeons, in contrast to experts. It would be informative to assess for similar/divergent responses by a cohort of neurotologists. Furthermore, a newly devised survey instrument was used for the comparison and not validated prior to employ.
Of note, our virtual simulation employs a 3-degree-of-freedom, low-stiffness haptic device. We are currently investigating the use of a 6-degree-of-freedom device that can produce moderately stiffer surfaces and multipoint contact forces, rather than the more commonly used 3-degree-of-freedom manipulandum.
While the PBM was the preferred simulation in many domains, the VM has value in anatomic learning and in learning surgical approaches. The benefit is magnified in light of its ease of use and relative nominal expense. Once established in a program, the requirements to maintain VM operation are minimal. Furthermore, the VM provides integrated metrics that are the basis of formative and summative feedback, an aspect of simulation that is paramount to the learning process.
While institutional bias can affect validation studies, both technologies were generated in the same research facility. The VM has not yet been validated, but its development was based on haptic rendering constructs and hardware commonly utilized in other published virtual simulations.8-13 While validated commercial systems exist, in-house design permits the use of the anatomically matched specimens key to our experimental design.
PBMs can be generated quickly and easily, requiring approximately 8 hours of printing and postprocessing time for every 10 composite models. Chimeric model components, including dura and decompressable vertical segment of the facial nerve, are incorporated as constituent bone slices are assembled. Costing of a model and cost comparisons to other simulations are complicated and must incorporate printer depreciation and labor. Comparisons to cadaveric costs are challenging, as these costs are often distributed across specialties and departments. A safety audit conducted by the Health Sciences Centre, Winnipeg Regional Health Authority, provided guidelines to ensure simulation safety.
The study does suffer from a sample size of convenience, and it should be noted that, while significant differences in many categories were not achieved, in only a small number of domains was the VM ranked higher (all within the anatomic features). A larger multicenter trial comparing a validated VM and PBM may alleviate some of the uncertainty within our single-center study. Formal validation of our in-house rating scale would be necessary prior to undertaking such a trial.
This study has several advantages: principal is the use of isomorphic matched simulations within subjects, to remove confounding variations in anatomy across simulations. The added measure of having unique parent cadaveric data for each participant ensures that the source anatomy did not influence outcome. Achieving such a safeguard required control of software and hardware and is not readily achievable with proprietary, validated simulations.
Conclusion
Resident surgeons at a single institution perceived the PBM as a more effective training tool in several important domains as compared to a VM. As this was a comparison of internally generated simulations, the results cannot be generalized across virtual systems. Haptic modeling is a current priority and may prove instrumental in the advance of virtual simulations.
Author Contributions
Disclosures
Footnotes
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
