Three-dimensional Motion Tracking for Beating Heart Surgery Using a Thin-plate Spline Deformable Model

Abstract

Minimally invasive cardiac surgery offers important benefits for the patient but it also imposes several challenges for the surgeon. Robotic assistance has been proposed to overcome many of the difficulties inherent to the minimally invasive procedure, but so far no solutions for compensating physiological motion are present in the existing surgical robotic platforms. In beating heart surgery, cardiac and respiratory motions are important sources of disturbance, hindering the surgeon’s gestures and limiting the types of procedures that can be performed in a minimally invasive fashion. In this context, computer vision techniques can be used for retrieving the heart motion for active motion stabilization, which improves the precision and repeatability of the surgical gestures. However, efficient tracking of the heart surface is a challenging problem due to the heart surface characteristics, large deformations and the complex illumination conditions. In this article, we present an efficient method for active cancellation of cardiac motion where we combine an efficient algorithm for 3D tracking of the heart surface based on a thin-plate spline deformable model and an illumination compensation algorithm able to cope with arbitrary illumination changes. The proposed method has two novelties: the thin-plate spline model for representing the heart surface deformations and an efficient parametrization for 3D tracking of the beating heart using stereo images from a calibrated stereo endoscope. The proposed tracking method has been evaluated offline on in vivo images acquired by a DaVinci surgical robotic platform.

Keywords

medical robots and systems human-centered and life-like robotics visual servoing sensing and perception computer vision visual tracking

References

Bader, T. , Wiedermann, A. , Roberts, K. and Hanebeck, U.D. ( 2007). Model-based motion estimation of elastic surfaces for minimally invasive cardiac surgery. Proceedings of IEEE Conference on Intelligent Robots and Systems (IROS ’07), San Diego, CA, pp. 871-876.

Bartoli, A. and Zisserman, A. ( 2004). Direct estimation of non-rigid registrations. Proceedings of the 15th British Machine Vision Conference, Kingston, UK, Volume 2, pp. 899-908.

Bebek, O. and Cavusoglu, M.C. ( 2007). Intelligent control algorithms for robotic-assisted beating heart surgery. IEEE Transactions on Robotics, 23(3): 468-480.

Benhimane, S. and Malis, E. ( 2007). Homography-based 2D visual tracking and servoing. International Journal of Computer Vision, 26(7): 661-676.

Bookstein, F.L. ( 1989). Principal warps: Thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 11(6): 567-585.

Cavusoglu, M.C. , Rotella, J. , Newman, W.S. , Choi, S. , Ustin, J. and Sastry, S.S. ( 2005). Control algorithms for active relative motion cancelling for robotic assisted off-pump coronary artery bypass graft surgery. Proceedings of the 12th International Conference on Advanced Robotics (ICAR), Seattle, WA, pp. 431-436.

Cuvillon, L. ( 2006). Compensation du battement cardiaque en chirurgie robotisée: Asservissement visuel d’un robot médical avec flexibilités. PhD thesis, Université Louis Pasteur- Strasbourg I.

Ginhoux, R. , Gangloff, J. , de Mathelin, M. , Soler, L. , Sanchez, M.A. and Marescaux, J. ( 2003). Active filtering of physiological motion in robotized surgery using predictive control. IEEE Transactions on Robotics , 21(1): 67-79.

Gröger, M. , Ortmaier, T. , Sepp, W. and Gerd, H. ( 2002). Tracking local motion on the beating heart. Visualization, Image-Guided Procedures and Display, Proceedings SPIE, 4681: 233-241.

10.

Hayashibe, M. , Suzuki, N. and Nakamura, Y. ( 2006). Laser-scan endoscope system for intraoperative geometry aqusition and surgical robot safety management. Medical Image Analysis, 10: 509-519.

11.

Lau, W. , Ramey, N.A. , Corso, J.J. , Thakor, N.V. and Hager, G.D. ( 2004). Stereo-based endoscopic tracking of cardiac surface deformation. Medical Image Computing and Computer-Assisted Intervention (MICCAI ’04) , Saint Malo, France, Vol. 3217, pp. 494-501.

12.

Leclercq, C. and Kass, D.A. ( 2002). Retiming the failing heart: principles and current clinical status of cardiac resynchronization. Journal of the American College of Cardiology, 39(2): 194-201.

13.

Lemma, M. , Mangini, A. , Redaelli, A. and Acocella, F. ( 2005). Do cardiac stabilizers really stabilize? Experimental quantitative analysis of mechanical stabilization. Interactive CardioVascular and Thoracic Surgery, 4: 222-226.

14.

Lim, J. and Yang, M. ( 2005). A direct method for modeling non-rigid motion with thin plate spline. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’05), Washington, DC, pp. 1196-1202.

15.

Malis, E. ( 2007). An efficient unified approach to direct visual tracking of rigid and deformable surfaces. Proceedings of IEEE Conference on Intelligent Robots and Systems (IROS ’07), San Diego, CA, pp. 2729-2734.

16.

Mountney, P. , Lo, B. , Thiemjarus, S. , Stoyanov, D. and Yang, G.Z. ( 2007). A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery. Medical Image Computing and Computer-Assisted Intervention (MICCAI ’07), Brisbane, Australia, Vol. 4792, pp. 34-41.

17.

Nakamura, Y.K. and Kawakami, H. ( 2001). Heartbeat synchronization for robotic cardiac surgery . Proceedings of IEEE International Conference on Robotics and Automation (ICRA ’01), Seoul, Korea, Vol. 2, pp. 2014-2019.

18.

Noce, A. , Triboulet, J. and Poignet, P. ( 2007). Efficient tracking of the heart using texture. Proceedings of IEEE International Conference of the Engineering in Medicine and Biology Society (EMBS ’07), Lyon , France, pp. 4480-4483.

19.

Ortmaier, T. , Groger, M. , Boehm, D.H. , Falk, V. and Hirzinger, G. ( 2005). Motion estimation in beating heart surgery. IEEE Transactions on Biomedical Engineering, 52(10): 1729-1740.

20.

Richa, R. , Bo, A.P. and Poignet, P. ( 2008). Motion prediction for tracking the beating heart. Proceedings of IEEE International Conference of the Engineering in Medicine and Biology Society (EMBS ’08), Vancouver , Canada, pp. 3261- 3264.

21.

Riviere, C.N. , Gangloff, J. and de Mathelin, M. ( 2006). Robotic compensation of biological motion to enhance surgical accuracy. Proceedings of the IEEE, 94(9): 1705-1716.

22.

Silveira, G. and Malis, E. ( 2007). Real-time visual tracking under arbitrary illumination changes. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR ’07), Minneapolis, MN, pp. 1-6.

23.

Stoyanov, D. , Darzi, A. and Yang, G. ( 2005a). A practical approach towards accurate dense 3D depth recovery for robotic laparoscopic surgery. Computer Aided Surgery, 4(10): 199-208.

24.

Stoyanov, D. , Mylonas, G.P. , Deligianni, F. , Darzi, A. and Yang, G.Z. ( 2005b). Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. Medical Image Computing and Computer-Assisted Intervention (MICCAI ’05), Palm Springs, CA, Vol. 3750, pp. 139-146.

25.

Stoyanov, D. and Yang, G.Z. ( 2007). Stabilization of image motion for robotic assisted beating heart surgery. Medical Image Computing and Computer-Assisted Intervention (MICCAI ’07), Brisbane, Australia, Vol. 4791, pp. 417- 424.

26.

Zhang, Z. ( 2000). A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 22(11): 1330-1334.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB