Abstract
The human temporal bone is a 3-dimensionally complex anatomic region with many unique qualities that make anatomic teaching and learning difficult. Current teaching tools have proved only partially adequate for the needs of the aspiring otologic surgeon in learning this anatomy. We used a variety of computerized image processing and reconstruction techniques to reconstruct an anatomically accurate 3-dimensional computer model of the human temporal bone from serial histologic sections. The model is viewed with a specialized visualization system that allows it to be manipulated easily in a stereoscopic virtual environment. The model may then be interactively studied from any viewpoint, greatly simplifying the task of conceptualizing and learning this anatomy. The system also provides for simultaneous computer networking that can bring distant participants into a single shared virtual teaching environment. Future directions of the project are discussed.
The temporal bone is one of the most complicated areas in the human body. It takes hundreds, even thousands of hours dissecting many, many temporal bones to gain the ability to mentally rotate and visualize in complete three-dimensional perspective all of the structures and their many interrelationships. With this knowledge, the surgeon can always visualize what is just beyond the rotating burr and can bring any desired structure into view.—William F. House, MD 1
The process of learning temporal bone anatomy remains one of the most difficult tasks of otolaryngology residency. The time and monetary expenditure required for teaching and learning this anatomy and otologic surgical procedures remain enormous despite a large number of teaching tools. Individual learning and group teaching are currently provided by anatomic descriptions, illustrations, photographs, histologic and gross sections, CT scans, MRI scans, and sculpted models. Further learning comes in the form of cadaver temporal bone dissection and observation of surgical procedures in the operating room. Despite all of these teaching aids, it takes hundreds of hours of intensive study and a great deal of trial and error in the laboratory before the otolaryngology resident begins to feel comfortable enough with this anatomy to begin drilling in the operating room. Furthermore, well-stocked temporal bone laboratories come at a great cost to residency programs, not only because of the equipment and its maintenance but also because of the number of cadaver temporal bones required for a resident's progression through a training syllabus.
The reason this anatomy is so difficult to learn is quite apparent: the temporal bone contains delicate, tiny anatomic structures in a complex 3-dimensional configuration compacted within dense bone. Most of the important anatomic sites of the temporal bone are hidden from direct sight in the intact temporal bone, only to be revealed by skilled anatomic dissection. Even when some structures of interest are revealed, others remain hidden from sight by the nature of dissection.
Many skilled illustrators have attempted to provide anatomic information in the form of different anatomic viewpoints of a single structure or area, but this method often fails to provide the anatomically accurate 3-dimensional information necessary for a working understanding of this region. Dissection photographs, both 2-dimensional and stereographic, provide accurate anatomic information but lack different perspectives of the dissected area, which is essential for conceptualizing 3-dimensional structures. Advances in radiologic imaging techniques are providing sharper and more detailed cross-sectional information about the temporal bone than ever before. Unfortunately, as a learning and teaching tool, they are limited by the requirement of the student of temporal bone anatomy to mentally reconstruct the cross-sectional cuts into a 3-dimensional model without visual aid. Furthermore, computer-aided reconstructions of scanned temporal bones are not detailed enough to provide effective teaching models.
The ideal teaching aid, in theory, would be a 3-dimen-sional model that the student of temporal bone anatomy could essentially hold in the hand so that the entire model could be easily and instantly viewed from any point of view as the student sees fit. The model would have to be large and detailed enough so that all of the important structures could be seen simultaneously or individually and so that their complex interrelationships could be appreciated with stereoscopic 3-dimensional vision. A computer-generated model has the potential to fulfill these criteria. Several authors have published results of computer-based models of the temporal bone. 2 - 5 Despite some initially promising results, these models had drawbacks that limited their teaching potential. For instance, models created as recently as 5 years ago were being run on, by today's standards, very slow computers that were not able to display very complex or realistic images. Even if accurate 3-dimensional computer models were created, they would still have to be viewed on a standard computer monitor screen. Although high resolution may be obtained on a standard monitor, a 3-dimensional model is still being displayed on a 2-dimensional viewing surface, limiting the viewer's conceptualization of 3-dimensional interrelationships. In addition, most of the previously described attempts to model temporal bone anatomy were only able to display static images of temporal bone reconstructions, not allowing the user to seamlessly and intuitively investigate the model. These limitations have significantly hampered the introduction of computer-based temporal bone models into a routine teaching environment.
Recently, advances in computing speed and 3-dimensional software and hardware development have made it possible to circumvent these limitations. Complexity, interactivity, and stereoscopic visualization are now able to be combined to create a 3-dimensional computer model of the human temporal bone that will change the way we learn anatomy.
PURPOSE
None of the current tools used to learn temporal bone anatomy is able to convey the important 3-dimensional structural spatial relationships of the anatomy within the bone simultaneously and interactively. Our purpose was to provide the student of temporal bone anatomy with a computer-based model designed to provide that crucial information in a way that will aid in the learning process. In turn, the model would reduce the considerable time and monetary expenditure necessary to learn this anatomy. An additional goal was to provide a model that could be used for the teaching of surgical approaches and procedures not only in a local classroom but also over large distances via the Internet.
METHODS AND MATERIAL
The basis of our model was a single human temporal bone from the University of Illinois Eye and Ear Infirmary temporal bone collection. The bone had been vertically sectioned into 630 individual slices in the standard method of preparation. Between sections 75 and 330, every section was mounted. For the remainder of the sections, every fifth slice was mounted. This particular data set was chosen not only for its completeness but also because it included anatomic structures not normally included in a standard temporal bone sectioning.
The principle behind our method of reconstruction is simple: if one were able to restack all of these 2-dimensional slices atop one another in order and in perfect alignment, a complete 3-dimensional temporal bone would be recreated. With a high-fidelity Arcus II transparency scanner (Agfa Corp, Ridgefield Park, NJ), the histologic slides were directly scanned and saved as individual image files. Ideally, alignment of the individual images with respect to one another would have been facilitated by external fiducial markers, as has been described by previous authors. Unfortunately, this data set, while providing a great deal more information than a standard temporal bone sectioning, did not contain preexisting fiducial markings. Therefore a different process of alignment was devised. With Adobe Photoshop 4.0 (Adobe Systems Inc, San Jose, CA) and PC-based Pentium desktop computers, the slides were aligned manually by comparing, translating, and rotating adjacent images with respect to one another by superimposing one image over the next. By changing the degree of image transparency and color inversion, the nearest anatomic alignment could be more easily ascertained. As each image was aligned to its predecessor, it was saved in a consistent format so that all of the slides within the histologic data set were consistently aligned with respect to one another.
Data Segmentation
Histologic sections inherently contain a great deal of artifact and extraneous information not appropriate for a teaching model. Furthermore, the computer does not automatically “know” how to pick out structures within a histologic section to create a coherent and useful model. Therefore it was necessary to manually delineate the structures of the temporal bone that were of anatomic interest and to delete the regions that provided no useful information. This process is termed segmentation. Anatomic structures of interest were chosen for their usefulness to the student of temporal bone anatomy such as major vessels, nerves, and organs. Regions of bone marrow, extraneous air cells, and surrounding tissues would only serve to clutter a teaching model and were thus designated for deletion. Again, with Adobe Photoshop 4.0, the anatomic structures of interest were manually traced within each histologic section directly on the computer screen and uniquely color coded. By this method, extraneous anatomic information was removed, and a new simplified histologic data set was created (Fig 1).
70234-8-fig1.png)
A, Raw histologic data slide, with structures of interest labeled. B, Segmented histologic slide.
Object Reconstruction
This segmented data set was then sent to a Silicon Graphics Maximum Impact 10000 workstation (Silicon Graphics Inc, Mountain View, CA), where the individual image sets were reconstructed into 3-dimensional objects. This process used a “marching cubes” algorithm and newly developed software written in C++ programming language in conjunction with the Visualization Toolkit. 6 This process effectively spreads a “skin” over the skeleton made up by the consecutively stacked 2-dimensional images, creating a group of polygonal models. Because of subtle alignment errors, these raw objects required filtering to smooth the rough image edges and surfaces. This was carried out by placing the image sets through a newly designed pyramid convolution filtering algorithm that averaged out small alignment discrepancies between consecutive images. The degree of filtration possible was balanced by the degree of detail loss caused by filtration.
Because the ossicles were poorly represented within the histologic data set, a different method was required for their reconstruction. Using human ossicles as a template, large ossicle models were sculpted with Styrofoam and plaster. A standard CT scanner was then used to scan these models and produce a cross-sectional data set. With a small amount of modification, these image slices could be imported into the Silicon Graphics workstation and reconstructed into 3-dimen-sional objects in a method similar to that of the histologic set. The external auditory canal and pinna models were created in a similar fashion.
Each individual 3-dimensional anatomic structural model was then recombined into the final model. Each object was uniquely colored and textured: vestibular labyrinth and cochlea light blue, nerves yellow, tensor tympani muscle dark red, sigmoid sinus and jugular vein dark blue, carotid artery red, bones white, and tympanic membrane pink.
RESULTS
The final model is an anatomically accurate human temporal bone that contains many of the important structures encountered during otologic surgery (Fig 2). The model currently contains the following anatomic structures: the pinna and canal, tympanic membrane, ossicles, tensor tympani muscle, vestibular labyrinth, cochlea, facial nerve, geniculate ganglion, cochlear nerve, vestibular nerves, carotid artery, sigmoid sinus, and jugular vein. The tympanic membrane and ossicles are animated so the sound of the user's voice will cause this sound-conductive mechanism to vibrate.
70234-8-fig2.png)
A, External ear.
70234-8-fig3.png)
Virtual Environment
Displaying and interacting with the complete model is an essential part of its usefulness. The primary display engine for the model uses a Silicon Graphics workstation coupled to an Immersadesk (Pyramid Systems, Novi, MI) viewer: a 4 × 5 foot angled rear-projection display screen (Fig 3 A). Three-dimensional stereoscopic visualization of the model is made possible by CAVE (Cave Automatic Virtual Environment) technology, developed at the Electronic Visualization Laboratory at the University of Illinois at Chicago. Created in 1992, CAVE technology creates the perception of “virtual reality” through 3-dimensional imagery involving sight, sound, and touch. Originally designed for engineering and design applications, CAVE technology and its associated visualization systems are rapidly becoming commonplace in major universities involved in architecture, research and development, and art.
The principles behind CAVE technology are as follows. Special goggles with liquid crystal lenses (Stereo-Graphics, Sausalito, CA) alternatively darken and lighten each lens at a rate of 60 times per second, faster than the brain can perceive (Fig 3 3). This oscillation is in perfect synchronization with the also alternating projection of two slightly offset images on the Immersadesk screen. The result is that each eye sees different, slightly offset images which, as in normal stereoscopic vision, are recombined in the brain to allow perception of a 3-dimensional object. As a result, when the user looks at the display, 3-dimensional objects appear to float in space in front of the Immersadesk. In addition, the Immersadesk display actively tracks the user's head position in relation to the computer-generated environment and will alter the 3-dimensional perspective within the environment based on these changes in head position. This allows the use of head movement to investigate the model in an intuitive and realistic fashion, giving the sensation of total immersion in the environment. Overall, this technology provides a new sense of realism that was previously lacking in virtual reality environments, which until now have only consisted of single point-of-view planar representations of 3-dimensional objects.
Within this virtual environment, the user is able to manipulate the model using a special 3-dimensional mouse-like device called the “wand” (Fig 3 C), which is also tracked by the Immersadesk. Using only subtle motions of the wrist and arm, the user has complete real-time control over the model's position in 3-dimen-sional space with unlimited degrees of freedom. The model can thus be twisted and turned in any direction so the user may look at one or many anatomic objects from an infinite number of viewpoints, which is essential for the understanding of complex 3-dimensional interrelationships. The model perspective may also be brought so close to the user that the sensation of walking among the structures of the temporal bone is created. The wand can also be used to manipulate a “virtual pointer”—a 3-dimensional arrow that can be used within the virtual environment to point out objects within the model. Various features such as transparency control and specific object selection allow the user to specifically tailor the model for different self-learning or teaching purposes.
DISCUSSION
Computer-generated models of the temporal bone are not new, but only very recently has it become possible to create a temporal bone model that provides realistic, interactive anatomic information with true stereoscopic visualization. This new teaching tool has far-reaching implications not only for students attempting to learn ear anatomy but also for their teachers and institutions. Although there will never be a true substitute for hands-on drilling in the temporal bone laboratory, we believe that this technology will aid the student to more quickly and easily conceptualize the complexities of temporal bone anatomy. This, in turn, will greatly hasten the student's progression through the laboratory syllabus and allow for a more thorough understanding of anatomic interrelationships. Progression to surgical expertise, in turn, will be greatly enhanced by helping the learning surgeon to visualize the hidden anatomy in the mind's eye while drilling into the temporal bone.
Networking
With the advent of widespread and easily accessible Internet connections, it now becomes possible to link Immersadesk systems by use of existing basic Internet technology. With standard speed data and voice connections that are already in widespread use across the United States and around the world, several Immersadesks can be simultaneously linked together. Once the base program has been installed on each system, the separate sites are linked via the Internet, with one Immersadesk designated as the control system. As the user at the control system moves the model, the positional information is transmitted to the linked systems. This information is used by the linked systems to make simultaneous changes in perspective to match the control system. Thus all users see the same changes in model perspective simultaneously, giving the sensation of sharing a single virtual environment. The sites are also connected by sound such that the participants may carry on a conversation during the session.
Within this environment, several parties at near or distant sites can review and discuss the model simultaneously, each having the capability to alter model perspective and manipulate a pointer within the virtual environment. Thus far, we have successfully demonstrated the capabilities of this setup with participants in 3 separate cities in the United States simultaneously. Despite the distance between the sites, all 3 participants essentially shared a single virtual environment, all participating in an interactive lesson on temporal bone anatomy as if they were standing in the same room. The implications of this capability are enormous: using this technology, an experienced otologic surgeon could provide a lesson to residents throughout the country simultaneously on this highly complex temporal bone model. This would provide much needed instruction for residents who do not have the availability of such an accomplished teacher in the local vicinity.
This model also provides a template for planning complex otologic procedures. Although our current model does not provide for virtual bone drilling, the experienced otologic surgeon can use it to plan surgery or even to create new surgical approaches.
System Costs
The initial startup costs for obtaining a system as described range from $140,000 to $200,000. Smaller desktop systems are currently being developed that would reduce these costs by one half to two thirds. As mentioned previously, many major university centers already own such a system for use in engineering, design, and artistic endeavors, and it may be available for use by medical staff as needed. As applications for virtual reality and CAVE technology expand in the medical field, teaching hospitals and medical schools will have multiple uses for such a system, thereby lowering the costs to any single department. For instance, the Virtual Reality in Medicine Laboratory at the University of Illinois at Chicago regularly accommodates physicians and researchers from otolaryngology, radiology, general surgery, orthopedics, pediatrics, and obstetrics.
Potential costs, however, must be balanced by the savings accrued by such an effective teaching tool. A faster progression through a temporal bone laboratory syllabus would mean fewer temporal bones used, less wear on laboratory equipment, and fewer instructors needed. In addition, the improvement in resident understanding of this complex area may enhance overall surgical skill, reducing the potential for complications and improving quality of patient care.
Future Work
Future directions of this project include an increase in the number and detail of anatomic structures represented within the model. With increased complexity, the model's usefulness as a teaching tool will be greatly enhanced. In addition, work is currently being done to create a “cutting plane” within the environment that will cut the model into cross sections much like a CT scanner does. With real-time control of the cutting process, the user could easily correlate 2-dimensional cross sections in any plane with 3-dimensional anatomy.
Although the current model can run only on the described system, further advances in computing speed and portable visualization technology will no doubt make it possible for this setup, complete with stereoscopic visualization, to be run on a standard desktop computer, making it even more accessible to practitioners and residents worldwide.
Special thanks to Noah Lowenthal, artist and sculptor, and Chet Childs, medical photographer.