Model-based passive bilateral teleoperation with time delay

Abstract

Noisy behaviour of time domain passivity control (TDPC) at low velocity is a well known problem for delayed teleoperation systems. This paper presents a novel adaptive model-based passive controller to alleviate some of the noise problems associated with delayed teleoperation system using the TDPC approach. By composing an online estimation of phantom human tissue model and passivity observer on the master side, a high transparency of teleoperation can be achieved while the system passivity is maintained by modifying the damping of the master. The performance of the developed approach was validated using one-degree-of-freedom master–slave robot system with constant time delay. Results show that the phantom tissue parameters can be accurately estimated. In addition, the passivity of the system is observed by the passivity observer using the output of the identified tissue model. Results demonstrate that stable teleoperation with time delay can be achieved and the environment force can be accurately reflected to the operator without chattering.

Keywords

Bilateral teleoperation environment parameters identification model-based approach passivity observer time delay

Introduction

Recently, robots in master–slave configurations have been introduced in minimally invasive surgery. In order to accomplish safer and more delicate surgical tasks using robotic manipulators, accurate and reliable reproduction of the haptic sensations to the surgeon are essential (Wagner and Howe, 2007). This requirement is generally characterized as the transparency of the master–slave system. Another major objective for teleoperation system design is stability. The system is required to be stable with respect to a set of uncertainties introduced by operator, communication channel, remote environment and sensors. In practical applications, particularly where the master–slave system performs over a great distance, the existence of communication delay creates some of the most challenging problems. In master–slave surgical robot systems, it has been confirmed that the surgeon effectively loses his/her ability to operate when the time delay exceeds 500 ms, even where there is only visual feedback (Butner and Ghodoussi, 2003). In the presence of the haptic feedback, the surgeon’s performance decreases significantly due to the time delay.

To date, several approaches have been proposed in the literature to deal with this problem. The first systemized stabilization method of bilateral teleoperation systems was proposed by Anderson and Spong (1989) based on scattering theory, and studied further by Niemeyer and Slotine (1991). Kosuge et al. (1998) proposed a virtual time-delay method and combined it with scattering transformations, while Munir and Book (2002) utilized a modified Smith predictor in the wave-variable framework. For time-varying-delay, Yokokohji et al. (1999) presented a method of compensation of position drifts due to time-varying-delay using standard delay time, while Natori et al. (2010) proposed and applied a disturbance observer for the compensation of time delay. Recently, Franken et al. (2011) proposed a two-layer approach to guarantee the stability of a bilateral teleoperation system in the presence of destabilizing factors such as time-varying communication delays. Besides these studies, there are also other approaches based on robust control theory. Leung et al. (1995) proposed a bilateral controller for time delay based on the H_∞ optimal controller and the μ-synthesis frameworks. Lee and Spong (2006) proposed a PD-based controller for constant time delay.

Hannaford and Ryu (2002) proposed time domain passivity control (TDPC) for guaranteeing the passivity of haptic interfaces. Based on TDPC, Hou and Luecke (2005) proposed a bilateral controller for teleoperation systems with a time delay, considering the slave and environment as a big one-port network system. Li et al. (2011) and Kawashima et al. (2008) proposed a model-based passivity approach for bilateral control of a teleoperation system with TDPC. However, TDPC suffers from a sudden force change when the time domain passivity condition is breached (Ryu et al., 2010). This noisy behaviour of TDPC is the consequences of the ‘see-then-action’ nature of this approach (Lee and Huang, 2010). Jafari et al. (2013) tried to improve the system transparency by adding a force error term to conventional passivity architecture. Amini et al. (2013) proposed a control scheme including a human contact force at the master side and a remote slave contact force-by-force estimation approach. They proved that system stability is guaranteed even in presence of time varying contact forces.

One alternative approach to compensate for the time delay in the transmission channel is the model-based approach (Passenberg et al., 2010). In this method, a remote environment model is created on the master side to duplicate the essential features of the remote slave robot and environment. Mitra and Niemeyer (2008) proposed a model-mediated approach to bilateral teleoperation under large communication delays. An adaptive impedance control scheme to alleviate the problems of time delays is presented by Velanas and Tzafestas in (2010). A virtual environment method and the correction to deal with the time delay were proposed by Li and Song (2007). However, this approach relies heavily on the accuracy of the model and the type of environment it is modelling.

In this paper, we advance a novel adaptive model-based passivity approach for achieving stable force perception over a teleoperation system with time delay. More precisely, an environment model is built on the master side online based on the identification parameters. This virtual model can duplicate the performance of the real environment and can be used for calculation of the passivity observer. The integration of an adaptive model with a passivity observer is an important step forward from the TDPC, since the passivity observer does not need to split the system energy into an incoming and outgoing energy flow (Ryu et al., 2010). As a result, the system energy can be calculated at the same time and environment parameters can be transmitted. Moreover, high teleopeation transparency is achieved for non-delayed force feedback from the model.

In addition, the system passivity is kept by modifying the master damping coefficient according to the output of passivity observer rather than direct modification of the feedback force in TDPC. A stable and smooth manipulation of the master device is achieved. Under this method, the sudden large force change to the master manipulator can totally diminished, which is a serious problem for the TDPC when a big passivity controller output is needed at the end contact. Experimental results are presented that demonstrate the feasibility of the approach.

Model-based teleoperation controller

Time domain passivity control

The passivity formalism represents a mathematical description of the intuitive physical concepts of power and energy. It provides a simple tool for system stability analysis (Slotine and Li, 1991). A system will be called passive if and only if

E (t) = {\int_{0}}^{t} P (τ) d τ + E (0) \geq 0

(1)

where E(t) is the total energy of the system at time t. P(t) denotes the net power at input and output ports. E(0) is the initial stored energy of the system at t = 0.

The energy E(t) of a two-port network can be characterized by an equation as follows:

E (t) = \int_{0}^{t} P (τ) d τ = \int_{0}^{t} (f_{md} (τ) v_{m} (τ) - f_{s} (τ) v_{s} (τ)) d τ \geq 0

(2)

which states that the energy supplied to a passive system must be non-negative for all time. Here, f_md(t), f_s(t), v_m(t) and v_s(t) are the port variables denoting force and velocity. The passivity condition in Equation (2) motivates the idea of time domain passivity control. The idea uses a passivity observer (PO) to monitor E(t) in real time. Depending on the operating conditions and the specifics of the element’s dynamics, the passivity observer may or may not be negative at a particular time. If it is negative at any time, the uncertainty of the may then be contributing to instability. Moreover, since the exact amount of the generated energy is known, the required amount of energy can be dissipated by a time-varying damping element, called a passivity controller.

Teleoperation control scheme

The goal of the control in the master–slave system is to obtain high transparency and provide good manoeuvrability so that the operators can feel the virtual tasks as accurate representations (Hannaford, 1998; Yokokohji and Yoshikawa, 1994). For this purpose, it may be required that both the position response and the force response be identical, despite the object dynamics. By modelling the teleoperation system as in Figure 1, the dynamics of a one-degree-of-freedom (1-DOF) master and slave system are given by the following equations:

master : f_{m} = M_{m} {\overset{\cdot\cdot}{x}}_{m} + B_{m} {\overset{\cdot}{x}}_{m}

(3)

slave : f_{sc} = M_{s} {\overset{\cdot\cdot}{x}}_{s} + B_{s} {\overset{\cdot}{x}}_{s}

(4)

where $M_{i}$ , $B_{i}$ and $x_{i}$ are the manipulator’s inertia, damping coefficient and displacement, respectively (i = m, s indicates the master, m or slave, s, manipulator). In Figure 1, T₁ is the time delay from the master to slave while T₂ is the delay from slave to master.

Figure 1.

Physical model of the position–force type teleoperation system.

The forces $f_{m}$ and $f_{sc}$ applied to the manipulators depend both on the interaction with the operator/environment and on the control action. In general, these forces can be defined as

f_{m} = f_{h} - f_{md}, f_{sc} = f_{s} - f_{e}

(5)

where $f_{md} (τ) = f_{s} (τ - T_{2})$ , $f_{h}$ , $f_{e}$ are the forces imposed by the human operator and by the environment, respectively. In Equation (5), $f_{s}$ is the force computed by the control algorithms as

f_{s} = K_{cs} (x_{md} - x_{s}) + B_{cs} ({\overset{\cdot}{x}}_{md} - {\overset{\cdot}{x}}_{s})

(6)

where $K_{cs}$ and $B_{cs}$ are parameters of the control system.

From the master–slave system model, the block diagram of the controller is shown in Figure 2. As there is no force sensor at the master manipulator, the manipulating force of operator $f_{h}$ is calculated by a PD controller, which is given as

f_{h} = K_{h} (x_{h} - x_{m}) + B_{h} ({\overset{\cdot}{x}}_{h} - {\overset{\cdot}{x}}_{m})

(7)

where K_h and B_h are parameters of the controller, and x_h is the displacement of the operator hand.

Figure 2.

Block diagram of impedance control of master–slave system.

Figure 3 illustrates equivalent transformation of Figure 2. The communication channel transmits the velocity of the master ${\overset{\cdot}{x}}_{m}$ for the slave to follow, and it returns force feedback f_s of the slave to the master. Here, G(s) is given as

G (s) = \frac{(K_{cs} + B_{cs} s) Z_{e} (s)}{(M_{s} s^{2} + B_{s} s) + (K_{cs} + B_{cs} s) + Z_{e} (s) s}

(8)

where Z_e(s) is the transfer function of the environment object.

Figure 3.

Equivalent block diagram of Figure 2.

Proposed control method

A general structure of the proposed model-based force-reflecting teleoperation system is shown in Figure 4. During teleoperation, the online identification method is used to obtain the parameters of the environment ${\hat{Z}}_{e}$ according to the data of slave velocity v_e and contact force f_e, and then it is transmitted back through communication channel to the master side. In this case, a virtual environment is created on the master side, which duplicates the essential features of the remote environment. This virtual environment is used to provide the passivity observer with no delayed feedback force ${\tilde{f}}_{s}$ . Lastly, the damping coefficient of the master manipulator is modified based on the output of the PO to satisfy the passivity criterion.

Figure 4.

Block diagram of the proposed model-based teleoperation controller.

As already discussed in previous section, the system is called passive and becomes stable if:

E (t) = \int_{0}^{t} P (t) d τ = \int_{0}^{t} (f_{md} (τ) {\overset{\cdot}{x}}_{m} (τ) - f_{s} (τ) {\overset{\cdot}{x}}_{md} (τ)) d τ \geq 0

(9)

where $f_{md} (τ) = f_{s} (τ - T_{2})$ and ${\overset{\cdot}{x}}_{md} (τ) = {\overset{\cdot}{x}}_{m} (τ - T_{1})$ .

The energy of the system E(t) in Equation (9) can be rewritten as

\tilde{E} (t) = \int_{0}^{t} (f_{md} (τ) {\overset{\cdot}{x}}_{m} (τ) - {\tilde{f}}_{s} (τ) {\overset{\cdot}{x}}_{md} (τ)) d τ

(10)

Here, a slave model is virtually implemented at the master side (virtual reflector $\tilde{P} (s)$ ) to calculate the force with no time delay ${\tilde{f}}_{s} (s)$ .We assume ${\tilde{f}}_{s} (t) = f_{s} (t)$ to calculate Equation (10), as $f_{s} (t)$ cannot be measured at the master side. As a result, the following equation can be obtained.

\tilde{E} (t) = \int_{0}^{t} (f_{s} (τ - T_{2}) {\overset{\cdot}{x}}_{m} (τ) - {\tilde{f}}_{s} (τ) {\overset{\cdot}{x}}_{m} (τ - T_{1})) d τ

(11)

$\tilde{E}$ (t) is called the passivity observer and is set at the master side for monitoring the energy flow of the bilateral controller.

In order to increase the stability robustness of the whole system, increasing the master robot damping is suggested by Daniel and McAree (1998) and Love and Book (2004). The increased damping coefficient can absorb the energy generated by the non-passive communication line during contact. This provides the motivation for adapting the master robot’s damping to variations according to the passivity of the teleoperation system. As a result, the master velocity ${\overset{\cdot}{x}}_{m}$ is given in the following equation.

{\overset{\cdot}{x}}_{m} (s) = \frac{f_{h} - f_{s} e^{- T_{2} s}}{M_{m} s + B_{m} + B} = \frac{f_{h} - f_{md}}{M_{m} s + B_{m} + B}

(12)

where B is a variable value calculated from the output of the passivity observer. As a result, the damping coefficient of the master manipulator can be changed by this variable. The following relationship is used to calculate damping coefficient B in real time.

B = {\begin{matrix} - W (t) / γ, W (t) < S (t) \\ const ., W (t) \geq S (t) \end{matrix}

(13)

where $γ$ is a parameter that can be used to turn the rate at how much the energy is used as damping coefficient for the master. S(t) is the reference energy and W(t) is the value of passivity observer calculated by Equation (11).

Here, the constant energy value S(t), rather than the fixed threshold of the PO at zero, is used as the threshold for calculating parameter B. As we know that traditional TDPC suffers from a sudden force change when the threshold of PO is fixed at zero (Ryu et al., 2005). For the purpose of surgical application, especially surgical objects of human tissues with soft and deformable characteristics, the traditional TDPC with reference energy threshold is preferred. As we have identified the environment model online, S(t) is set as the energy storage of the environment. Then S(t) can be easily calculated by

S (t) = \frac{1}{2} {\tilde{K}}_{e} {x_{s}}^{2} (t)

(14)

Here, ${\tilde{K}}_{e}$ is the environment stiffness and x_s(t) is the slave position. As the environment parameters are identified online (discussed later), S(t) can be easily calculated. For simplicity, we use the largest value of S(t) when slave position x_s(t) achieves its maximum value.

During teleoperation, if the passivity observer becomes smaller than the reference energy S(t) due to the long time delay, that means the system has the trend of becoming active, and B is calculated from Equation (13) to satisfy the passivity requirements. The compensation is negligible when the output of the passivity observer is larger than S(t) and B equals a constant value, calculating from S(t). Comparing the proposed method with other conventional approaches, such as wave variable-based approaches, our method at least provides more transparency, as the reflecting force from the slave is less modified (experimental comparison will be shown below).

Environment modelling error and system passivity

The main problem of model-based teleoperation arises from the model uncertainties (Passenberg et al., 2010), and until now it has not been addressed from a theoretical point of view in the literature.

In Equation (11), the estimated force ${\tilde{f}}_{s} (t)$ is used instead of the actual contact force $f_{s} (t)$ for the calculation of the passivity observer. There must be some differences between ${\tilde{f}}_{s} (t)$ and $f_{s} (t)$ because of the existence of parameter estimation errors. Suppose the environment parameters are stiffness K_e and damping B_e,

{\tilde{K}}_{e} = K_{e} + Δ K_{e}, {\tilde{B}}_{e} = B_{e} + Δ B_{e}

(15)

Here ${\tilde{K}}_{e}$ , ${\tilde{B}}_{e}$ are estimated values and $Δ K_{e}$ , $Δ B_{e}$ are estimation errors. The estimated environment force is given by ${\tilde{f}}_{s} (t) = f_{s} (t) + Δ f_{s} (t)$ , and $Δ f_{s} (t) = Δ K_{e} x_{s} + Δ B_{e} {\overset{\cdot}{x}}_{s}$ is the force error.

Then the passivity observer in Equation (11) can be written as

\begin{matrix} \tilde{E} (t) = \int_{0}^{t} (f_{md} (τ) {\overset{\cdot}{x}}_{m} (τ) - {\tilde{f}}_{s} (τ) {\overset{\cdot}{x}}_{md} (τ)) d τ \\ = \int_{0}^{t} (f_{md} (τ) {\overset{\cdot}{x}}_{m} (τ) - (f_{s} (τ) + Δ f_{s} (τ)) {\overset{\cdot}{x}}_{md} (τ)) d τ \\ = E (t) - \int_{0}^{t} Δ f_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \end{matrix}

(16)

In order to satisfy the passivity criterion, the following inequality should be forced.

E (t) = \tilde{E} (t) + \int_{0}^{t} Δ f_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \geq 0

(17)

In the upper section, we used environment storage energy $S (t) = \frac{1}{2} K_{e} {x_{s}}^{2}$ as the threshold for calculating proposed viscosity value B, which means when

E (t) = S (t) + \int_{0}^{t} Δ f_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \geq 0

(18)

the whole system passivity can be maintained. For clarity, we only use the environment stiffness estimation error $Δ K_{e}$ and neglect the environment damping $Δ B_{e}$ for calculating the force error $Δ f_{s} (t)$ . Then Equation (18) will be:

\begin{matrix} E (t) = S (t) + \int_{0}^{t} Δ f_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \\ = \frac{1}{2} {\tilde{K}}_{e} {x_{s}}^{2} (t) + \int_{0}^{t} Δ K_{e} x_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \\ = \frac{1}{2} (K_{e} + Δ K_{e}) {x_{s}}^{2} (t) + \int_{0}^{t} Δ K_{e} x_{s} (τ) {\overset{\cdot}{x}}_{md} (τ) d τ \end{matrix}

(19)

From a practical viewpoint, it is reasonable to assume that the slave robot is equipped with an ‘almost perfect’ position control loop so that the commanded velocity ${\overset{\cdot}{x}}_{md}$ is achieved with negligible dynamics, i.e. ${\overset{\cdot}{x}}_{md} \approx {\overset{\cdot}{x}}_{s}$ . As a result, the energy calculation in Equation (19) is given by

\begin{matrix} E (t) = \frac{1}{2} (K_{e} + Δ K_{e}) {x_{s}}^{2} (t) + \int_{0}^{t} Δ K_{e} x_{s} (τ) {\overset{\cdot}{x}}_{s} (τ) d τ \\ = \frac{1}{2} (K_{e} + Δ K_{e}) {x_{s}}^{2} (t) + \frac{1}{2} Δ K_{e} {x_{s}}^{2} (t) \\ = (\frac{1}{2} K_{e} + Δ K_{e}) {x_{s}}^{2} (t) \end{matrix}

(20)

From Equation (20), we can clearly see that when the environment estimation error $Δ K_{e}$ satisfies

| Δ K_{e} | \leq \frac{1}{2} K_{e}

(21)

the system energy $E (t) \geq 0$ . This means that the whole system passivity can be maintained when the estimation errors obey inequality (21). In practice, the condition shown in inequality (21) can be easily achieved by environment parameter identification algorithms such as recursive least-squares.

Thus we have demonstrated that even if there are some identification errors in environment parameters, the whole system passivity is maintained and the system stability remains.

Environment modelling and estimation technique

In the previous analysis it was illustrated how knowledge of the robot’s environment can improve the performance of the teleoperation system. This section describes a method of modelling and identifying the impedance of the environment.

We assume that the slave robot is in contact with an environment modelled as a surface with stiffness coefficient K_e and damping coefficient B_e. The one-dimensional version of the environment model is shown schematically in Figure 5. A simple model representing the environment of the slave robot is

f_{e} = B_{e} {\overset{\cdot}{x}}_{s} + K_{e} (x_{s} - x_{0})

(22)

where x_s is the actual slave robot position and x₀ models the constraint surface in the robot’s workspace. Here the equilibrium position of the environment x₀ is assumed to be zero. For simplicity, the dynamic characteristics (inertia, friction) of the environment are neglected, supposing relatively slow motion throughout the teleoperation task.

Figure 5.

One-dimensional version of the environment model.

In commonly used position–force bilateral teleoperation systems, this force f_e is reflected directly on the master controller, and is displayed via a haptic interface on the human operator. If there are some time delays in the communication channel, this will leads to inconsistencies in the displayed feedback forces with respect to the position from the master, causing severe degradation of the teleoperation transparency, as well as system instabilities.

As described above in Equation (22), the impedance parameters of the environment, K_e and B_e, are unknown within the master system. When the teleoperation system is subject to communication time delays, the forces applied to the PO are calculated using estimated impedance values, ${\tilde{B}}_{e}$ and ${\tilde{K}}_{e}$ , as follows:

{\tilde{f}}_{e} = {\tilde{B}}_{e} {\overset{\cdot}{x}}_{s} + {\tilde{K}}_{e} x_{s}

(23)

In order to obtain the environment parameters, recursive least square schemes allowing for a fast converging and stable estimation are employed. They are used to provide an online estimate of the actual remote environment characteristics K_e and B_e. In general, the least square estimation can be described as an optimization problem where the estimation error, $e_{f} = {\tilde{f}}_{e} - f_{e}$ , between measured and estimated system output has to be minimized. Define the estimated reaction force ${\tilde{f}}_{e}$ expressed in Equation (23). A linear system can be expressed by the linear transformation in the following equation.

y_{k} = φ^{T} (k) θ

(24)

where $y_{k}$ is the system output, $φ^{T} (k)$ is the regression vector and θ is the parameter vector. Here we define the environment parameter vector, $θ = {[{\tilde{B}}_{e}, {\tilde{K}}_{e}]}^{T}$ , and a regression vector, $φ (k) = {[{\overset{\cdot}{x}}_{s} (k), x_{s} (k)]}^{T}$ , so that Equation (23) is rewritten as :

{\tilde{f}}_{e} = {\tilde{B}}_{e} {\overset{\cdot}{x}}_{s} + {\tilde{K}}_{e} x_{s} = φ^{T} (k) θ

(25)

Recursive least squares with exponential forgetting is based on the following cost function:

J_{n} = \sum_{i = 1}^{n} λ^{n - i} {(f (i) - φ {(i)}^{T} θ (i))}^{2}

(26)

The solution to the linear system is then computed by a recursive algorithm. The parameter vector θ is calculated every sample time as (Ljung, 1987):

{\tilde{θ}}_{n} = {\tilde{θ}}_{n - 1} + L_{n} [f_{n} - {φ_{n}}^{T} θ_{n - 1}]

(27)

where $L_{n} = Y_{n - 1} φ_{n} {(β + {φ_{n}}^{T} Y_{n - 1} φ_{n})}^{- 1}$

Y_{n} = β^{- 1} [I - L_{n} {φ_{n}}^{T}] Y_{n - 1}

and where $β (0 < β \leq 1)$ is a forgetting factor and the subscript n denotes t = nT. In each sample period, the output vector f_n and the regression vector $φ_{n}$ are updated. The initial guess for the adaptation gain matrix Y and the forgetting factor $β$ must be specified by the user. The measurable output of the system is the corresponding force calculated from the PD controller. Then this force is used as the variable to formulate the cost function.

Equation (27) is used to compute online updates for the estimates of the environment parameters. As the slave side robot manipulates its environment, these estimates are then reflected back to the haptic display controller at master side, as shown in Figure 4, and constitute the local environment model used to compute the forces applied to the PO according to Equation (23). The overall block diagram of the proposed control method is shown in Figure 6.

Figure 6.

Block diagram of the proposed mode-based passive controller. The damping of the master manipulator is modified according to the output of PO. The parameters of the environment are identified online then transmitted to the local site through communication channel.

Experimental results

Experimental set-up

To validate and assess the performance of the proposed model-based teleoperation control scheme presented in the previous section, an experimental evaluation approach is carried out. The schematic configuration of the teleoperation system is shown in Figure 7 and the actual experimental set-up is shown in Figure 8. The set-up consists of two identical 1-DOF devices powered by a DC motor without a gearbox (Figure 8a). As shown in Figure 8(b), the phantom human tissue is placed on the base environment. A Maxon DC motor is employed as the master equipped with an encoder (Microtech Laboratory Inc. MEH-12-2000PST16), with a resolution of 32,000 pulses/rev.

Figure 7.

Configuration of teleoperation system.

Figure 8.

Experimental set-up: (a) the whole teleoperation set-up consists of two identical 1-DOF devices powered by an electromotor without a gearbox. The position of each manipulator is recorded with a high-precision encoder. (b) Slave manipulator and phantom human tissue.

Both devices are controlled from the same controller running a real-time Linux distribution. The sampling frequency of the control loop is 1 kHz. The time delay is emulated using a buffering algorithm. The parameters used in the experiments are shown in Table 1. The master was operated by human hand and the slave robot is driven to contact with phantom human tissue with stiffness K_e and viscosity B_e located at the position x_e.

Table 1.

Parameters for experiments.

M_m	0.01 kg	M_s	0.01 kg
B_m	0.002 Ns/mm	B_s	0.002 Ns/mm
K_h	1.0 N/mm	K_cs	0.25 N/mm
B_h	0.005 Ns/mm	B_cs	0.002 Ns/mm
β	1.0	γ	0.01 mm²/s

Experimental results

Without model-based passivity controller

We firstly assume that there is no communication delay between the master and slave stations. The slave manipulator was driven by master to contact that phantom human tissue several times and the experimental results are shown in Figure 9. The position and force responses of the master and slave manipulators show stable interaction (Figure 9a,b). The output energy of the system stays almost zero (Figure 9c), demonstrating a passive teleoperation system.

Figure 9.

Experimental results with phantom tissue under no time delay: (a) position response of the master–slave manipulator; (b) force tracking performance; (c) the energy of the whole teleoperation system.

Conversely, if we assume a round-trip delay of 200 ms, then the respective results we obtain are shown in Figure 10. Due to the delay, the phase delay can be seen in both the position and force signals (Figure 10a,b). Even the operator controlling the master to keep stable, the results shown in Figure 10(c), suggests that the energy becomes a negative value that the passivity of the whole system is not satisfied by the controller.

Figure 10.

Contact with phantom tissue under 200-ms round-trip time delay without model-based predictive control: (a) position response of the master and slave; (b) force tracking performance; (c) system energy with time delay.

It is evident from these results that the presence of even a small time delay in the communication lines may make a classical position–force teleoperation system not passive and then unstable.

Tracking in contact motion with proposed controller

In this paragraph, we apply the model-based passivity controller scheme proposed in previous section. We use again a 1-DOF slave robot with phantom human tissue as the remote environment. For simplicity, the forward and backward communication delays T₁ and T₂ were set to the same value. The constant reference energy S(t) is calculated from Equation (14) and set as 0.48 N/mm.

As we have described previously, the time delay in the transmission channels of the master–slave surgical robot system makes vivid sensation feedback to the surgeon difficult. Note that for the application considering a surgical environment, manipulating soft tissues accounts for 25–35% of the time spent on most surgical procedures (Ljung, 1987). Motivated by this, the proposed model-based passivity approach is tested in contact with soft phantom tissue.

The experiment was performed with the proposed control method when the slave side is in contact with the phantom tissue. The teleoperation task was defined, in which the operator was asked to press repeatedly against the target environment for 10 s and then remove the slave from the environment. Please note that this repeated contact was a voluntary motion. The initial values of the covariance matrix Y₀ of the recursive least square method (Equation 10) are set as:

Y_{0} = [\begin{matrix} 10^{6} & 0 \\ 0 & 10^{6} \end{matrix}]

(32)

The teleoperation results for conventional position–force controller are displayed in Figures 11 and 12. In this experiment, initial conditions for the environment stiffness and damping coefficient were set as zero. Also, the estimator is active only when the slave robot comes in contact with environment. As seen in Figure 11, the least square method was able stably to predict the environment stiffness and viscosity. The estimated environment parameters converge to the real values in a short time, although there are some oscillations in the first 2 s.

Figure 11.

Experimental results of phantom tissue parameters estimation for round-trip 200-ms time delay: (a) stiffness estimation result; (b) viscosity estimation results.

At the first 15 s of this experiment, the operator was asked to push against the phantom human tissue all the time; the parameters of the environment were identified and sustained when the slave is out of contact. The sustained parameters were then sent to the master side with a time delay. Then a virtual model of the environment was built based on these parameters on the master side. At the same time, the energy observer of the system is built using the output force of the virtual environment.

Figure 12 shows the experimental results. The slave position and force follows the master position and force with a 200-ms round-trip time delay, as shown in Figure 12(a) and (b). The system energy, as shown in Figure 12(d), is always positive. The estimated force from the virtual model is used as the reflecting forces to the PO as shown in Figure 12(b). The damping coefficient of master manipulator (Figure 12c) is modified according to the output of PO (Figure 12d). From Figure 12(c), we can see the proposed method of Equation (13) described previously is used to activate the calculation of parameter B. When the PO becomes smaller than the reference energy, the control method is activated and the parameter B is calculated by Equation (13); otherwise value B is constant.

Figure 12.

Experimental results for round-trip 200-ms time delay with phantom tissue: (a) position tracking; (b) force tracking; (c) parameter b; (d) system energy.

Tracking in contact motion with longer time delay

In order to check the performance of the proposed approach for a longer time delay, a similar experiment was carried out with the same experimental conditions as the above experiment. Here, the round time delay in the communication channel is set as 400 ms.

Figure 13 shows the experimental results. Similar to the previous experimental results shown in Figure 12, the position responses of the master and slave (x_m and x_s) were stable (Figure 13a). The proposed control method is activated (Figure 13c) when the output of the PO becomes smaller than reference energy S(t) (Figure 13d). The stable performances of the force at master side and the contacting force from the virtual environment model are shown in Figure 13(b); the largest contact force is about 0.5 N. From the output energy of the system shown in Figure 13(d), the whole system becomes passive after activation of the control approach, and thus the system is stable during operation.

Figure 13.

Experimental results for round-trip time delay 400 ms with phantom tissue: (a) position tracking; (b) force tracking; (c) parameter b; (d) system energy.

During actual surgical operation, the contacting force f_s (shown in Figure 2) of the surgical manipulator and environment will be reflected to the master side. This kind of reflecting force will significantly increase the transparency of the master–slave surgical robot system. However, the system will become unstable if there are some time delays in the transmission lines. Most importantly, the operator will feel oscillations and lose his/her manoeuvrability when contacting with the environment. In our method, the damping coefficient of the master manipulator is modified according to the PO instead of modifying the reflected force f_s. Thus, the operator feels less heavy when manipulating the master manipulator due to the increased damping coefficient of the master manipulator (Figures 12c and 13c).

Experimental comparison with wave-variable-based approach

To date, several approaches have been proposed in the literature to deal with time delay problems in teleoperation systems. The passivity-based wave variable method by Niemeyer and Slotine (1991) was used, as it is the most popular and well-studied method for ensuring a passive communication channel between the master and the slave. Recently, there have been a number of studies on variable-based control design techniques for bilateral teleoperation (Bate et al., 2011; Ye and Liu, 2009). It is desirable to compare the performance of our new proposed controller with those popular and recent studies. During the following comparison studies, the inertias, damping coefficients of the master–slave manipulators and control parameters of the PD controller are set the same as in the previous experimental conditions shown in Table 1 to keep things fair. Here we just show the experimental result of the wave variable-based teleoperation system. Please see Niemeyer and Slotine (1991), Ye and Liu (2009) and Bate et al. (2011) for more details about this approach.

Wave-variable-based approach

This verifies the effective of the traditional wave-variable teleoperator proposed by Niemeyer and Slotine (1991). In our experiments, the operator can apparently feel the force fluctuations of the master manipulator during the experiment. During contact motion, the operator cannot feel the environment contact force very well. In the literature, this kind of fluctuations is called wave reflection, which is a well known phenomenon in wave-variable-based teleoperation. The experimental results are neglected in order to save space here.

A second implementation of the wave-variable-based scheme is proposed by Bate et al. (2011). In this approach, the force and velocity information is decoupled at the slave side to prevent circulating wave reflections. The experimental results of this approach are shown in Figure 14. The energy of the system is always positive, as shown in Figure 14(c), which means that the system passivity is maintained. From the position tracking shown in Figure 14(a), we can see some position drift during the free motion and some force fluctuations when the slave starts to contact with the environment or release from the object (Figure 14b). Even with modifications, the bias terms of the position and force tracking still exist.

Figure 14.

Experimental results of modified wave-variable-based teleoperator (Bate et al., 2011) with 400-ms round-trip time delay: (a) position response of the master–slave manipulator; (b) force tracking performance; (c) power of the system.

The above two experimental results demonstrated that the wave variable-based method has the benefit of guaranteeing stability under time delays (Figure 14c). However, the algorithm, even the modified one by Bate et al. (2011), also reflects parts of the transmitted signal on the master and slave side, causing significant oscillation and poor transient response.

System transparency

In the most general way, Yokakahji and Yoshikawa (1994) defined transparency in the case where the positions (velocities) and forces at the master and slave are identical.

f_{h} = f_{s}, {\overset{\cdot}{x}}_{m} = {\overset{\cdot}{x}}_{s}

(33)

Here, this definition is used to check the transparency of the proposed method by checking the position tracking errors and errors of the force felt by the operator and slave contact force from the environment model. Please note that here we use the position errors instead of velocity errors.

Here, two experimental results with the same 400-ms time delay are compared. In Figure 15, the black solid lines show the position and force tracking errors of wave-variable-based approach proposed by Bate et al. (2011), while the dashed lines stand for the experimental errors of our earlier proposed method. Compared with the experimental results shown in Figure 15, the proposed approach has a higher transparency than the wave-variable-based method, as larger position and force errors can be easily observed.

Figure 15.

Experimental error comparisons of the modified wave-variable-based approach (Bate et al., 2011) and the proposed model-based method with 400-ms round-trip time delay: (a) position tracking error; (b) force tracking error.

Conclusion and future work

This paper introduces a novel adaptive model-based passivity approach, which can deal with time delay for stable telesurgery. Experimental results show that environment parameters can be accurately identified and stable teleoperaion can be achieved under a time delay while the remote slave contact force is accurately displayed to the operator. Thus a high transparency of the master–slave system is achieved while the system passivity is maintained. Experimental results also indicate that modification of the master viscosity according to the system passivity observer can keep the system passivity effectively. Following our feasibility study on time delay estimations for telesurgery and phantom human tissue, it can be concluded that the adaptive model-based approach using a passivity observer is effective and accurate. If applied in a master–slave surgical robot system over a great distance, it has the potential to aid surgeons considerably in procedures that involve the accurate reproduction of the haptic sensations on the slave side, as well as improving their intraoperative diagnostic and interventional decisions.

To improve the performance of the proposed model-based passivity approach further, future development will be conducted on the following aspects. First, since in the long term we aim to model real human tissues for surgical application, the simplest spring model will be replaced by non-linear and more complex tissue models, having an exponential-like biomechanical response to applied forces (Brown, 2003; Fung, 2004).

Moreover, experiments with devices containing multiple DOFs, such as the surgical manipulator using pneumatic artificial muscle (Li et al., 2013) and applying an online parameter estimation algorithm to real human tissue have to be conducted. Our eventual goal is to acquire a patient model online during performance of master–slave surgical robots with a time delay so that the model can be used to improve teleoperation transparency while system stability is maintained.

Footnotes

Conflict of interest

The author declares that there is no conflict of interest.

Funding

This research was partially supported under Shanghai Science and Technology Innovation Action Plan and the Research Fund for the Doctoral Program of Higher Education of China.

References

Amini

Rezaei

Zareinejad . (2013) Enhanced time delayed linear bilateral teleoperation system by external force estimation. Transactions of the Institute of Measurement and Control 35(5): 638–647.

Anderson

Spong

(1989) Bilateral control of teleoperation with time delay. IEEE Transaction on Automatic Control 34(5): 494–501.

Bate

Cook

(2011) Reducing wave-based teleoperator reflections for unknown environments. IEEE Transactions on Industrial Electronics 58(2): 392–397.

Brown

(2003) In-vivo and postmortem biomechatronics of abdominal organs under compressive loads: experimental approach in a laparoscopic surgery setup. PhD thesis, University of Washington.

Butner

Ghodoussi

(2003) Transforming a surgical robot for human telesurgery. IEEE Transactions on Robotics and Automation 19(5) :818–824.

Daniel

McAree

(1998) Fundamental limits of performance for force reflecting teleoperation. The International Journal of Robotics Research 17(8): 811–830.

Franken

Stramigioli

Misra

. (2011) Bilateral telemanipulation with time delays: a two-layer approach combining passivity and transparency. IEEE Transactions on Robotics 27(4): 741–756.

Fung

(2004) Biomechanics: Mechanical Properties of Living Tissues. 2nd ed. Berlin: Springer.

Hannaford

(1998) A design framework for teleoperations with kinematic feedback. IEEE Transactions on Robotics and Automation 5(4): 426–434.

10.

Hannaford

Ryu

(2002) Time-domain passivity control of haptic interfaces. IEEE Transactions on Robotics and Automations 18(1): 1–10.

11.

Hou

Luecke

(2005) Time delayed teleoperation system control, a passivity based method. International Conference on Advanced Robotics: 796–802.

12.

Jafari

Rezaei

Ghidary

(2013) A stable perturbation estimator in force-reflecting passivity-based teleoperation. Transactions of the Institute of Measurement and Control 35(2): 147–156.

13.

Kawashima

Tadano

Sankaranarayanan

. (2008) Model-based passivity control for bilateral teleoperation of a surgical robot with time delay. Proceedings of the IEEE/RSJ IROS: 1427–1432.

14.

Kosuge

Murayama

Takeo

(1998) Bilateral feedback control of telemanipulators via computer network. Proceedings of the IEEE/RSJ IROS 3: 1380–1385.

15.

Lee

Huang

(2010) Passive-set-position-modulation framwork for interactive robotic systems. IEEE Transactions on Robotics 26(2): 354–369.

16.

Lee

Spong

(2006) Passive bilateral teleoperation with constant time delay. IEEE Transactions on Robotics 22(2): 269–281.

17.

Leung

GMH

Francis

Apkarian

(1995) Bilateral controller for teleoperators with time delay via μ-synthesis. IEEE Transaction on Robotics and Automation 11(1): 105–116.

18.

Song

(2007) Virtual-environment modeling and correction for force-reflecting teleoperation with time delay. IEEE Transactions on Industrial Electronics 54(2): 1227–1233.

19.

Kawashima

Tadano

. (2011) Model-based passivity control for teleoperation of robots with force display. Proceedings of the ASME IMECE2011, Denver, CO, 11–17 November.

20.

Kawashima

Tadano

. (2013) Achieving haptic perception in forceps’ manipulator using pneumatic artificial muscle. IEEE/ASME Transactions on Mechatronics 18(1): 74–85.

21.

Ljung

(1987) System identification: Theory for the User. Englewood Cliffs, NJ: Prentice-Hall.

22.

Love

Book

(2004) Force reflecting teleoperation with adaptive impedance control. IEEE Transactions on Systems, Man, and Cybernetics 34(1): 159–165.

23.

Mitra

Niemeyer

(2008) Model-mediated telemanipulation. The International Journal of Robotics Research 27(2): 253–262.

24.

Munir

Book

(2002) Internet-based teleoperation using wave variable with prediction. IEEE/ASME Transactions on Mechatronics 7(2):124–133.

25.

Natori

Tsuji

Ohnishi

. (2010) Time-delay compensation by communication disturbance observer for bilateral teleoperation under time-varying delay. IEEE Transactions on Industrial Electronics 57(3): 1050–1062.

26.

Niemeyer

Slotine

JJE

(1991) Stable adaptive teleoperation. IEEE Journal of Oceanic Engineering 16(1): 152–162.

27.

Passenberg

Peer

Buss

(2010) A survey of environment-, operator-, and task-adapted controllers for teleoperation systems. Journal of Mechatronics 20: 787–801.

28.

Ryu

Artigas

Preusche

(2010) A passive bilateral control scheme for a teleoperator with time-varying communication delay. Mechatronics 20(7): 812–823.

29.

Ryu

Preusche

Hannaford

. (2005) Time domain passivity control with reference energy following. IEEE Transactions on Control Systems Technology 13(5): 737–742.

30.

Slotine

JJE

(1991) Applied Nonlinear Control. Englewood Cliffs, NJ: Prentice-Hall.

31.

Velanas

Tzafestas

(2010) Human telehaptic perception of stiffness using an adaptive impedance reflection bilateral teleoperation control scheme. IEEE International Symposium on Robot and Human Interactive Communication: 21–26.

32.

Wagner

Howe

(2007) Force feedback benefit depends on experience in multiple degree of freedom robotic surgery task. IEEE Transactions on Robotics 23(6): 1235–1240.

33.

Liu

(2009) Improving haptic feedback fidelity in wave-variable-based teleoperation orientated to telemedical applications. IEEE/ASME Transactions on Mechatronics 58(8): 374–384.

34.

Yokokohji

Imaida

Yoshikawa

(1999) Bilateral teleoperation under time-varying communication delay. Proceedings of the IEEE/RSJ IROS 3: 1854–1859.

35.

Yokokohji

Yoshikawa

(1994) Bilateral control of master–slave manipulators for ideal kinesthetic coupling-formulation and experiment. IEEE Transactions on Robotics and Automation 10(5): 605–620.