Abstract
Observation of surgical procedures performed by experts is extremely important for acquisition and improvement of surgical skills. Smart glasses are small computers, which comprise a head-mounted monitor and video camera, and can be connected to the internet. They can be used for remote observation of surgeries by video streaming. Although Google Glass is the most commonly used smart glasses for medical purposes, it is still unavailable commercially and has some limitations. This article reports the use of a different type of smart glasses, InfoLinker, for surgical video streaming. InfoLinker has been commercially available in Japan for industrial purposes for more than 2 years. It is connected to a video server via wireless internet directly, and streaming video can be seen anywhere an internet connection is available. We have attempted live video streaming of knee arthroplasty operations that were viewed at several different locations, including foreign countries, on a common web browser. Although the quality of video images depended on the resolution and dynamic range of the video camera, speed of internet connection, and the wearer’s attention to minimize image shaking, video streaming could be easily performed throughout the procedure. The wearer could confirm the quality of the video as the video was being shot by the head-mounted display. The time and cost for observation of surgical procedures can be reduced by InfoLinker, and further improvement of hardware as well as the wearer’s video shooting technique is expected. We believe that this can be used in other medical settings.
Acquisition and improvement of surgical skill are an eternal challenge for surgeons. Although many educational materials, including online and offline videos have been available recently, nothing is better than to observe operations performed by experts. As opportunities for experts to visit other hospitals are limited, participants go to the experts’ institutions in most cases. Limitation of time to visit, travel distance, and costs are obstacles to this.
Smart glasses are small computers, which comprise a head-mounted monitor that can display various information and a video camera that records what the wearer is seeing. This is quite suitable for medical purposes and especially for surgery. The smart glasses can be controlled by voice, action, or special devices, keeping the wearer’s hands free. In addition, the visual field is never obstructed by the wearer’s head and is directly transferred to the audience via the internet. This feature of smart glasses can be used for surgical observation by distant surgeons.
Some surgeons have attempted to use Google Glass in various types of surgeries, such as pediatric surgery, 1 dermatological surgery, 2 and plastic surgery. 3 However, Google Glass has not been commercially available up to now; therefore, it is not available for general use by most surgeons.
InfoLinker (Westunitis Inc, Osaka, Japan; Figure 1) glasses are alternative smart glasses that have great potential for surgical use. It was developed for industrial purposes, such as task navigation, remote manufacturing, and know-how sharing systems. 4 It has been available and used in various Japanese industrial settings for more than 2 years. It is not difficult to imagine that it is also applicable for medical purposes. The purpose of this study was to evaluate competence of the smart glasses in video transmission of joint reconstruction surgery using InfoLinker and discuss problems to be solved before further clinical use.
InfoLinker.
Methods
Device
InfoLinker glasses are smart glasses made in Japan. It has a central processing unit (CPU) that runs Android 4.2.2 and a nontransparent WQVGA (428 × 240 in resolution) prism monitor that enables clear visualization under bright circumstances. It also has a 1992 × 1216 front-facing video camera. The monitor and camera set are flexible, and the body can be attached to glasses that the wearer always uses. The monitor displays what the camera sees, so the wearer can check the video image in real time.
The operator wears InfoLinker, and the operation field image is transferred to a secure video server via the internet (Figure 2). A wireless connection (Wi-Fi or Bluetooth) to a portable wireless router allows for the internet connection. No special construction of a network is necessary; therefore, video streaming can be performed in any situation and at any location where wireless internet is available. The video can be seen on a web browser, and access is secured by passwords (Figure 3).
The operator wearing InfoLinker.
Viewing the streaming video on the web browser.
Video Transmission
We transmitted videos during unicompartmental knee arthroplasty operations. Because the operation rooms are sometimes sealed to prevent transmission of X-ray beams, wireless internet connection is not always sufficient. Therefore, a wireless router was placed near a window where the internet Wi-Fi was working well and was connected to a router by a LAN cable temporarily. Although laying of a LAN cable is required, the cost and effort for preparation are minimal and flexible.
Results
We performed video transmission of 3 knee arthroplasty operations. The streaming video was transmitted from Takatsuki General Hospital, Osaka, Japan, and was viewed over the internet in several cities and countries simultaneously such as in Kobe, Hiroshima, Kyoto, Tokyo, Hirosaki, Kamigori, and Asahikawa in Japan and in Korea, China, and France. Observing the video was quite easy on the web browser using secured IDs and passwords anywhere where internet was available.
Image Quality
In general, the quality of the image depended on network speed rather than quality of the camera. In our first trial, the Wi-Fi router was placed in the operating room and video resolution was set to be 320 × 240. The quality was totally fair; the resolution was reasonable, but frame rate was low (around 5/s) and frame drops were apparent. In the second and third trials, an internet router was placed outside of the operation room and an access point was located in the room and connected to the router by a LAN cable. The frame rate improved significantly (up to 15/s), and frame drops were virtually absent. In the third trial, the video resolution was raised to 640 × 480, and frame rate was the same as in the second trial, resulting in improvement of total video quality to practical clinical use.
Battery Life
Battery life is the main factor that limits the work time with InfoLinker. It was less than 1 hour in our first trial. Therefore, we connected a portable battery case to InfoLinker thereafter. The battery life was adequately prolonged.
Heat-down of CPU
Heat-down of CPU was another cause of shortened working time. A surgical helmet was worn during the operation. In the first trial, InfoLinker was down within 1 hour as a result of overheating. A cooling gel sheet was pasted on the body of InfoLinker near the CPU during video transmission. Consequently, no overheating occurred.
Discussion
Advantages of using smart glasses for surgery include the following:
The operation field is not disturbed by the surgeon’s head. Smart glasses have a head-mounted monitor along with a video camera. This means that the camera is placed in front of the operator’s eye; therefore, the visual field of the operation site is not disturbed by the operator’s head.
The operator can verify the recorded video images during the recording by looking at the monitor.
Several head-mounted video cameras are already available. However, the recorded images cannot be verified during surgery. Sometimes, the center of the image diverges from the operation field without the monitor.
InfoLinker is hands free. Operators can record video hands free. Therefore, they are able to keep their hands clean and use both hands during the operation.
There have been several reports of video transmission of surgeries using Google Glass. Google Glass was released about 3 years ago. However, it is still commercially unavailable because of the problem of privacy. Although the quality of the video and monitor of InfoLinker is similar to that of Google Glass, InfoLinker is superior in several aspects. The first point is flexibility. InfoLinker can be attached to any type of glasses. Double glassing is possible as well. The camera head is connected by a flexible arm, which can be bent to adjust the location of the monitor or camera to the best position. The second point is that unlike Google Glass, InfoLinker does not require a smartphone connection and access to the Google drive. It can directly access a video server via wireless internet. This means that InfoLinker is more flexible than Google Glass. Third, InfoLinker is already available in the Japanese market and has been used in manufacturing processes for several years.
Despite these advantages, InfoLinker still has some limitations. The first limitation is video quality. The resolution of the video is 1992 × 1216. It is somewhat lower than that of Google Glass. The angle of view is 14°. Although the angle is equivalent to that of Google Glass, it is still wide for surgery video capturing. A narrower view angle or zooming mechanism is preferable. Furthermore, the dynamic range is still narrow, so over- and underexposure sometimes occurred. It depends on the image-sensing chips within the limited space of the wearable machine.
The second limitation is durability of the glass in InfoLinker. Battery life and heat influence the durability of the glass in InfoLinker. In the first trial, the durability was less than 1 hour as a result of heat-down of the CPU. In the second trial, a cooling gel was adhered to the device and a secondary mobile battery was applied. The working time was prolonged to more than 2 hours.
The third limitation is camera shake. The video image is sensitive to head motion of the wearer. In the first trial, the operator moved his attention between the operation field to the tray of the instrument. Unlike the wearer’s own view, the image on the screen shakes and is a source of great discomfort for the viewer. In the second and later trials, the wearer was instructed to pay attention to minimize movement of his own head, and camera shake decreased. However, there was still some shake, and a mechanism to counteract the shake will need to be developed.
Despite these disadvantages, InfoLinker is an innovative tool for use in surgeries. The time and cost for observation of surgical procedures can be reduced by InfoLinker, and further improvement of hardware as well as the wearer’s video shooting technique is expected. This usability and flexibility well compensates for some residual disadvantages. We believe that InfoLinker can be used in other medical settings.
Footnotes
Author Contributions
Study concept and design: Takafumi Hiranaka
Acquisition of data: Takafumi Hiranaka
Analysis and interpretation: Takafumi Hiranaka, Yuta Nakanishi
Study supervision: Takaaki Fujishiro, Yuichi Hida, Masanori Tsubosaka, Yosaku Shibata, Kenjiro Okimura, Harunobu Uemoto
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.