The increase in the use of robotic assisted surgery has provided benefits to both the patient and the surgeon. There are possibilities for training and surgical planning and guidance within this framework but work is needed for this to reach a patient-specific level. Medical images obtained prior to surgery are used clinically to plan operations and there has been much research in their use to build models, giving further knowledge of tissue and organ structures. The incorporation of this data into the surgical environment is a recent development, assisting the surgeon with orientation information, but motion due to respiration or changes to morphology due to intervention begets a need for the incorporation of real-time imaging into the modelling. The current work on pre-operative and intra-operative imaging and modelling methods for robotic assisted surgery has to be reviewed and then the Image Constrained Biomechanical Modelling has to be introduced, whereby patient-specific biomechanical modelling is based on the underlying medical images, reducing the computational complexity involved with traditional biomechani-cal modelling. This article concludes with a discussion of the possible and promising future directions of this work.
DelingetteH.PennecX.SolerL.MarescauxJ.AyacheN., ‘Computational models for image-guided robot-assisted and simulated medical nterventions’Proc. IEEE949 (2006): 1678–1688.
3.
JoleszF. A.NabaviA.KikinisR., ‘Integration of interventional MRI with computer-assisted surgery’J. Magn. Reson. Imaging131 (2001): 69–77.
4.
EdwardsP. J.KingA. P.MaurerC. R.De CunhaD. A.HawkesD. J.HillD. L. G.GastonR. P.FenlonM. R.JusczyzckA.StrongA. J.ChandlerC. L.GleesonM. J., ‘Design and evaluation of a system for microscope-assisted guided interventions (MAGI)’IEEE Trans. Med. Imaging1911 (2000): 1082–1093.
5.
DamanioR. J., ‘Robotics in cardiac surgery: The emperor's new clothes’J. Thoracic Cardiovasc. Surg.1343 (2007): 559–561.
6.
GrimsonW. E. L.EttingerG. J.WhiteS. J.Lozano-PerezT.WellsW. M.KikinisR., ‘An automatic registration method for frameless stereotaxy, image guided surgery, and enhanced reality visualization’Med. Imaging IEEE Trans.152 (1996): 129–140.
LeeW.-N.QianZ.TostiC. L.SwaminathanS. V.BrownT. R.MetaxasD. N.KonofagouE. E., ‘Validation of myocardial elastography using MR tagging in normal and abnormal human hearts in vivo‘ In Proceedings of the 4th IEEE International Symposium on Biomedical imaging from nano to macro, Washington, DC, USA, ISBI 2007, 2007, pp. 384–387.
11.
MarwickT. H., ‘Measurement of strain and strain rate by echocardiography: Ready for prime time?;’J. Am. College Cardiol.477 (2006): 1313–1327.
12.
PellerinD.SharmaR.ElliottP.VeyratC., ‘Tissue Doppler, strain, and strain rate echocardiography for the assessment of left and right systolic ventricular function’Heart89 (2003): 9–17.
13.
BryantD. J.PayneJ. A.FirminD. N.LongmoreD. B., ‘Measurement of flow with NMR imaging using a gradient pulse and phase difference technique’J. Comput. Assisted Tomography84 (1984): 588–593.
14.
AxelL.DoughertyL., ‘MR imaging of motion with spatial modulation of magnetization’Radiology1713 (1989): 841–845.
15.
AletrasA. H.DingS.BalabanR. S.WenH., ‘DENSE: Displacement encoding with stimulated echoes in cardiac functional MRI’J. Magn. Reson.137 (1999): 247–252.
LeeS.-L.WuQ.HuntbatchA.YangG.-Z., ‘Predictive K-PLSR myocardial contractility modelling with phase contrast MR velocity mapping’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, Brisbane, Australia, 2007, pp. 866–873.
18.
LeeS.-L.HuntbatchA.YangG.-Z., ‘Contractile analysis with kriging based on MR myocardial velocity imaging’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2008, New York, NY, USA, 2008, pp. 892–899.
19.
ZhuY.PelcN. J., ‘A spatiotemporal model of cyclic kinematics and its application to analyzing nonrigid motion with MR velocity images’IEEE Trans. Med. Imaging187 (1999): 557–569.
20.
ShiP.RobinsonG.ConstableR. T.SinusasA.DuncanJ., ‘A model-based integrated approach to track myocardial deformation using displacement and velocity constraints’ In Proceedings of the Fifth International Conference on Computer vision, IEEE Computer Society, Cambridge, Massachusetts, USA, 1995.
21.
HunterP. J.BorgT. K., ‘Integration from proteins to organs: The Physiome Project’Nat. Rev. Mol. Cell Biol.43 (2003): 237–243.
22.
GoetzM.KiesslichR., ‘Confocal endomicroscopy: in vivo diagnosis of neoplastic lesions of the gastrointestinal tract’Anticancer Res.281B (2008): 353–360.
23.
GibsonA. P.HebdenJ. C.ArridgeS. R., ‘Recent advances in diffuse optical imaging’Phys. Med. Biol.504 (2005): R1–R43.
HeeM. R.IzattJ. A.SwansonE. A.HuangD.SchumanJ. S.LinC. P.PuliafitoC. A.FujimotoJ. G., ‘Optical coherence tomography of the human retina’Arch. Ophthalmol.1133 (1995): 325–332.
26.
MountneyP.LoB.ThiemjarusS.StoyanovD.Zhong-YangG., ‘A probabilistic framework for tracking deformable soft tissue in minimally invasive surgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, Brisbane, Australia, 2007, pp. 34–41.
27.
StoyanovD.MylonasG. P.DeligianniF.DarziA.YangG. Z., ‘Soft-tissue motion tracking and structure estimation for robotic-assisted MIS procedures’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2005, Palm Springs, California, USA, 2005, pp. 139–146.
28.
MourguesF.DevemayF.Coste-ManiereE., ‘3D reconstruction of the operating field for image overlay in 3D-endoscopic surgery’ In Proceedings of the IEEE and ACM International Symposium on Augmented reality, New York, NY, USA, 2001, pp. 191–192.
29.
GinhouxR.GangloffJ. A.de MathelinM. F.SolerL.SanchezM. M. A.MarescauxJ., ‘Beating heart tracking in robotic surgery using 500 Hz visual serving, model predictive control and an adaptive observer’ In Proceedings of the IEEE International Conference on Robotics and automation, ICRA 2004, Barcelona, Spain, 2004, vol. 271, pp. 274–279.
30.
SauvéeM.NoceA.PoignetP.TribouletJ.DombreE., ‘Three-dimensional heart motion estimation using endoscopic monocular vision system: From artificial landmarks to texture analysis’Biomed. Signal Proc. Control23 (2007): 199–207.
31.
LoB.ScarzanellaM.StoyanovD.YangG.-Z., ‘Belief propagation for depth cue fusion in minimally invasive surgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2008, New York, NY, USA, 2008, pp. 104–112.
32.
StoyanovD.MylonasG.YangG.-Z., ‘Gaze-contingent 3D control for focused energy ablation in robotic-assisted surgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2008, New York, NY, USA, 2008, pp. 347–355.
33.
SozaG.GrossoR.NimskyC.GreinerG.HastreiterP., ‘Estimating mechanical brain tissue properties with simulation and registration’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2004, St Malo, France, 2004, pp. 276–283.
34.
ShiinaT.YamakawaM., ‘Fast reconstruction of tissue elastic modulus image by ultrasound’ In Proceedings of the 27th Annual International Conference of the Engineering on medicine and biology society, IEEE-EMBS 2005, Shanghai, P. R. China, 2005, pp. 976–980.
35.
FabriceC.EhrlacherA.MohamedR.ThoM. C. H. B.TranV., ‘The in vivo characterization of the mechanical properties of human skin by MRI and indentation techniques’Comput. Meth. Biomech. Biomed. Eng.101 (2007): 121–122.
36.
KauerM.VuskovicV.DualJ.SzékelyG.BajkaM., ‘Inverse finite element characterization of soft tissues’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2001, Utrecht, The Netherlands, 2001, pp. 128–136.
37.
KarimiR.ChanR.HouserS.BoumaB. E.Kaazempur MofradM. R., ‘A novel framework for elastography and modulus estimation: Integration of tissue mechanics with imaging’ In Proceedings of the 3rd IEEE International Symposium on Biomedical imaging, nano to macro, Arlington, Virginia, USA, 2006, pp. 602–605.
38.
OmataS.MurayamaY.ConstantinouC. E., ‘Real time robotic tactile sensor system for the determination of the physical properties of biomaterials’Sens. Actuators A: Phys.1122–3 (2004): 278–285.
39.
WeisslederR.PittetM. J., ‘Imaging in the era of molecular oncology’Nature4527187 (2008): 580–589.
40.
SanduleanuS.DriessenA.Gomez-GarciaE.HameetemanW.de BruïneA.MascleeA., ‘In vivo diagnosis and classification of colorectal neoplasia by chromoendoscopy-guided confocal laser endomicroscopy’Clin. Gastroenterol. Hepatol84371–378.
41.
ZagaynovaE.GladkovaN.ShakhovaN.GelikonovG.GelikonovV., ‘Endoscopic OCT with forward-looking probe: Clinical studies in urology and gastroenterology’J. Biophotonics12 (2008): 114–128.
42.
Wachsmann-HogiuS.WeeksT.HuserT., ‘Chemical analysis in vivo and in vitro by Raman spectroscopy–from single cells to humans’Curr. Opin. Biotechnol.201 (2009): 63–73.
43.
WoodT.ThiemjarusS.KohK.ElsonD.YangG.-Z., ‘Optimal feature selection applied to multispectral fluorescence imaging’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2008, New York, NY, USA, 2008, pp. 222–229.
44.
GueziecA.KazanzidesP.WilliamsonB.TaylorR. H., ‘Anatomy-based registration of CT-scan and intraoperative X-ray images for guiding a surgical robot’IEEE Trans. Med. Imaging175 (1998): 715–728.
45.
KozakJ.NesperM.FischerM.LutzeT.GöggelmannA.HassfeldS.WetterT., ‘Semiautomated registration using new markers for assessing the accuracy of a navigation system’Comput. Aided Surg.71 (2002): 11–24.
46.
BettingF.FeldmarJ.AyacheN.DevernayF., ‘A new framework for fusing stereo images with volumetric medical images’Comput. Vision Virtual Reality Robot. Med. LNCS 905 (1995): 30–39.
47.
SamaniA.BishopJ.YaffeM. J.PlewesD. B., ‘Biomechanical 3-D finite element modelling of the human breast using MRI data’IEEE Trans. Med. Imaging204 (2001): 271–279.
48.
WittekA.MillerK.KikinisR.WarfieldS. K., ‘Patient-specific model of brain deformation: Application to medical image registration’J. Biomech.404 (2007): 919–929.
49.
KleinG. J., ‘Forward deformation of PET volumes using non-uniform elastic material constraints’ In Proceedings of the Information Processing in Medical Imaging (Eds KubaA.SamalM.ToddPokropekA.), Visegrád, Hungary, 1999, pp. 358–363 (Springer-Verlag, Berlin).
50.
SchnabelJ. A.TannerC.Castellano-SmithA. D.DegenhardA.LeachM. O.HoseD. R.HillD. L.HawkesD. J., ‘Validation of nonrigid image registration using finite-element methods: Application to breast MR images’IEEE Trans. Med. Imaging222 (2003): 238–247.
51.
AblittN. A.JianxinG.KeeganJ.SteggerL.FirminD. N.Guang-ZhongY., ‘Predictive cardiac motion modelling and correction with partial least squares regression’IEEE Trans. Med. Imaging2310 (2004): 1315–1324.
52.
KingA. P.RhodeK. S.RazaviR.SchaeffterT., ‘An adaptive and predictive respiratory motion model for image-guided interventions: Theory and first clinical application’IEEE Trans. Med. Imaging (2009): (e-pub ahead of print).
53.
BaderT.WiedemannA.RobertsK.HanebeckU. D., ‘Model-based motion estimation of elastic surfaces for minimally invasive cardiac surgery’ In Proceedings of the IEEE International Conference on Robotics and automation, 2007, pp. 2261–2266.
54.
LeroticM.ChungA.MylonasG.YangG.-Z., ‘pq-space based non-photorealistic rendering for augmented reality’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, Brisbane, Australia, 2007, pp. 102–109.
55.
BichlmeierC.HeiningS. M.FeuersteinM.NavabN., ‘The virtual mirror: A new interaction paradigm for augmented reality environments’IEEE Trans. Med. Imaging289 (2009): 1498–1510.
56.
StoyanovD.YangG.-Z., ‘Stabilization of image motion for robotic-assisted beating heart surgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, 2007, pp. 417–424.
57.
MylonasG.StoyanovD.DarziA.YangG.-Z., ‘Assessment of perceptual quality for gaze-contingent motion stabilization in robotic-assisted minimally invasive surgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, Brisba0ne, Australia, 2007, pp. 660–667.
58.
LimY.-J.JinW.DeS., ‘On some recent advances in multimodal surgery simulation: A hybrid approach to surgical cutting and the use of video images for enhanced realism’Presence: Teleoperators and Virtual Environments166 (2007): 563–583.
59.
SkrinjarO.NabaviA.DuncanJ., ‘A stereo-guided biomechanical model for volumetric deformation analysis’ In Proceedings of the Mathematical Methods in Biomedical Image Analysis, MMBIA 2001, Kauai, Hawaii, USA, 2001, pp. 95–102.
60.
SunH.KennedyF. E.CarlsonE. J.HartovA.RobertsD. W.PaulsenK. D., ‘Modelling of brain tissue retraction using intraoperative data’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2004, St Malo, France, 2004, pp. 225–233.
61.
CarterT.TannerC.CrumW.Beechey-NewmanN.HawkesD., ‘A framework for image-guided breast surgery’Med. Imaging Augmented Reality4091 (2006): 203–210.
62.
CashD. M.MigaM. I.SinhaT. K.GallowayR. L.ChapmanW. C., ‘Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements’IEEE Trans. Med. Imaging2411 (2005): 1479–1491.
63.
KobashiK.PapademetrisX.DuncanJ. S., ‘A new biomechanical model based approach on brain shift compensation’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2003, Montreal, Canada, 2003, pp. 59–66.
64.
KemperC. A.TalosI.-F.GolbyA.BlackP. M.KikinisR.GrimsonW. E. L.WarfieldS. K., ‘An anisotropic material model for image guided neurosurgery’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2004, St Malo, France, 2004, pp. 267–275.
65.
LunnK. E.PaulsenK. D.LiuF. H.KennedyF. E.HartovA.RobertsD. W., ‘Data-guided brain deformation modelling: Evaluation of a 3-D adjoint inversion method in porcine studies’IEEE Trans. Biomed. Eng.5310 (2006): 1893–1900.
66.
PrattP., ‘Image guidance and surgery simulation using inverse nonlinear finite element methods’ In Proceedings of the International Symposium on Biomedical simulation (Eds BelloF.EdwardsP. J.), Paris, France, 2008, pp. 185–190.
SueblinvongV.WeissD. J., ‘Cell therapy approaches for lung diseases: Current status’Curr. Opin. Pharmacol.93 (2009): 268–273.
69.
ClatzO.SermesantM.BondiauP. Y.DelingetteH.WarfieldS. K.MalandainG.AyacheN., ‘Realistic simulation of the 3-D growth of brain tumors in MR images coupling diffusion with biomechanical deformation’IEEE Trans. Med. Imaging2410 (2005): 1334–1346.
70.
KyriacouS. K.DavatzikosC.ZinreichS. J.BryanR. N., ‘Nonlinear elastic registration of brain images with tumor pathology using a biomechanical model’IEEE Trans. Med. Imaging187 (1999): 580–592.
71.
RosenbergL. B., ‘Virtual fixtures: Perceptual tools for telerobotic manipulation’ In Proceedings of the IEEE Virtual Reality Annual International Symposium, Seattle, Washington, USA, 1993, pp. 76–82.
72.
RenJ.PatelR. V.McIsaacK. A.GuiraudonG.PetersT. M., ‘Dynamic 3-D virtual fixtures for minimally invasive beating heart procedures’IEEE Trans. Med. Imaging278 (2008): 1061–1070.
73.
KwokK.-W.MylonasG.SunL.LeroticM.ClarkJ.AthanasiouT.DarziA.YangG.-Z., ‘Dynamic active constraints for hyper-redundant flexible robots’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2009, London, UK, 2009, pp. 410–417.
74.
AllardJ.CotinS.FaureF.BensoussanP. J.PoyerF.DuriezC.DelingetteH.GrisoniL., ‘SOFA – an open source framework for medical simulation. In’Medicine meets virtual reality (2007): 13–18.
75.
TaylorZ.ChengM.OurselinS., ‘Real-time nonlinear finite element analysis for surgical simulation using graphics processing units’ In Proceedings of the Medical Image Computing and Computer-Assisted Intervention, MICCAI 2007, Brisbane, Australia, 2007, pp. 701–708.
