Work with me,not for me: Relationship between robotic assistance and performance in subacute and chronic stroke patients

Abstract

Introduction

Studies in robotic therapy which applied the performance enhancement approach report improvements in motor performance during training, though these improvements do not always transfer to motor learning.

Objectives

We postulate that there exists an assistance threshold for which performance saturates. Above this threshold, the robot’s input outweighs the patient’s input and likely learning is not fostered. This study investigated the relationship between assistance and performance changes in stroke patients to find the assistance threshold for performance saturation.

Methods

Twelve subacute and chronic stroke patients engaged in five sessions (over two weeks, each 60 min) in which they performed a reaching task with the rehabilitation robot H-Man in presence of varying levels of haptic assistance (50 N/m to 290 N/m, randomized order). In two additional sessions, a therapist manually tuned the assistance to promote maximal motor learning.

Results

Higher levels of assistance resulted in smoother and faster performance that saturated at assistance levels with K ≥ 110 N/m. Also, the therapist selected assistance levels of K = 175 N/m or below.

Conclusion

The findings of the study indicate that low levels of assistance (K ≤ 175 N/m) can sufficiently induce a significant change in performance.

Keywords

Assistive technology decentralized care neurorehabilitation robotic rehabilitation robotic assistance stroke rehabilitation

Introduction

The world’s population is aging, with the number of people aged 65 years or older expected to reach 1.6 billion (17% of the global total) by 2050.¹ The risk of stroke increases with age,² with incidences doubling each decade after the age of 55.^3,4 Healthy older adults and those affected by stroke often suffer from deficits in upper extremity sensorimotor dysfunction due to changes in both the central and peripheral nervous systems.⁵ Results from studies exploring the time course of recovery report that approximately up to 70% of the patients have residual impairment in the upper extremity six months post stroke.^6–9 The prevalence of motor dysfunction in elderly and stroke populations has motivated research groups to develop technology-assisted systems that can decrease the workload of clinicians, while also facilitating motor re-learning. In the last few decades, multiple robotic solutions have been developed that promote sensorimotor learning in populations with sensorimotor impairments such as stroke.^10–12 Overall, results of clinical studies have demonstrated that robot-assisted training is at least as effective as conventional physical therapy.^13–17

In addition to considerations regarding the mechanical design of the robotic systems, there has been ample interest in elucidating robotic interactive control algorithms that can positively influence motor learning (i.e. the processes associated with practice or experience that leads to long-term changes in the ability to perform a skill¹⁸). In a very common training scheme (known as the ‘performance enhancement approach’), the movements of a patient are haptically guided or constrained in some fashion,^19–21 with the goal of enhancing the patient’s motor performance (i.e. an observable and measurable change in motor skill during training²²) during the task. The assistance provided by the rehabilitation robot enables patients to perform otherwise inexecutable movements, which is said to stimulate brain plasticity and sensorimotor learning processes.^23,24 Prior research has demonstrated that robotic assistance improves task performance during training,^20,21 but these effects are often short-lived and do not translate to long-term learning.²¹ For example, Liu et al.²¹ conducted a study in which healthy subjects were first guided in a tracing task (i.e. training phase) in such a way that the subjects were not required to actively support the movement. Then, when the subjects were asked to replicate the movement without any assistance (i.e. recall phase) from the robot immediately afterwards, the subjects made large tracing errors. This finding is congruent with the ‘guidance hypothesis’,²⁵ which argues that when guidance is provided very frequently during new skill acquisition, the user relies on said guidance to perform the task and/or learns an altered task, and when the feedback is absent, there is a noticeable decline in performance quality.^26,27 Thus, for the case of robotic rehabilitation, haptic guidance may result in suboptimal motor learning if too much support or assistance is provided during training.^21,26 In addition, too much assistance may motivate slacking in the user^28,29 since the user can rely on the robot completing the task, but effort is considered crucial for motor learning.^30–32

Taken together, there is strong evidence that performance and learning are not directly related³³: a proficient motor performance during training (in presence of assistance) does not necessarily result in a proficient motor performance when assistance is removed (i.e. learning). We agree that robotic assistance influences motor performance (metrics sensitive to the motor condition, as e.g. smoothness), but argue that there exists an assistance threshold for performance saturation (for a given robotic device) for which assistance levels higher than such a threshold do not result in further performance improvements. Likely, in these cases, the robot does most of the task and this, in consequence, results in a slacking response by the user. Concluding from the precedent on the relationship between motor performance and motor learning, assistance above that threshold does, therefore, not provoke any further learning gain. To the best of our knowledge, this threshold has not been systematically determined yet for stroke patients with upper limb dysfunction.

Knowledge about the maximal assistance requirements would directly translate to the power requirements of a rehabilitation device. Early upper limb robotic systems were designed to fully support a patient’s movements (e.g. MIT-Manus,³⁴ ARMin,³⁵ HapticMaster³⁶), which resulted in complex, high-powered setups. The high cost and safety issues of these complex rehabilitation robotics restrict their use to centralized care facilities (e.g. hospitals), and thus limit their application to decentralized environments (i.e. community centers or patient’s home).³⁷ Fortunately, there is growing interest in designing lower powered neurorehabilitation robots that can be used in decentralized locations^38,39 (e.g. hCAAR,³⁸ H-Man⁴⁰), which in contrast to early systems are likely to be more accessible for patients due to their reduced costs and inherent safety. Knowing the maximal range of assistance required for performance saturation, and, hence, the maximal required power supply is a crucial step for the future development of devices aiming to provide therapy in decentralized settings while avoiding unnecessary power disposability.

As such, the aim of the present study was to find this assistance threshold by investigating the relationship between five different robotic assistance levels and motor performance in presence of assistance in 12 subacute and chronic stroke patients. To achieve this aim, we utilized a two degrees of freedom robotic manipulator (hereafter referred to as ‘H-Man’) designed for upper limb assessment and rehabilitation training.⁴⁰ Stroke patients performed an upper extremity reaching task in five sessions (each 60 min) over a period of two weeks, under varying levels of robotic assistance (i.e. 50, 110, 170, 230, and 290 N/m). Differences in standard kinematic performance metrics (i.e. spectral arc length (SAL) and normalized total time (T_norm)) were examined. A secondary aim of this study was to gain an understanding of optimal robotic assistance for motor learning from the rehabilitation therapist’ viewpoint. Thus, after the two-week rehabilitation program, patients completed another two sessions with the H-Man robot (each 60 min), during which the rehabilitation therapist tuned the assistance levels to induce a maximum learning effect based on motor behavioral characteristics of the participant. These results are the first step in elucidating an optimal assistance threshold for the H-Man, with the aim to develop guidelines for future developments of rehabilitative devices employed in decentralized care settings.

Methods

Participants

Twelve subacute and chronic stroke patients (age: 55.8 ± 10.0 years, 7 males, time since stroke: 11.3 ± 6.5 months) participated in the present study (Table 1). Study inclusion criteria were first-ever clinical stroke (ischaemic or haemorrhagic) confirmed by brain imaging, post-stroke duration of 3 to 24 months, with shoulder abduction and elbow flexion greater or equal to 3/5 on the Medical Research Council scale for muscle strength, and a Fugl–Meyer Upper Extremity Motor Assessment (FMA)⁴¹ score of 20–50 or predominant motor ataxia or incoordination (FMA > 50). Participants were excluded if they had any non-stroke related arm impairment, moderate arm spasticity as indicated by the Modified Ashworth Scale⁴² (MAS > 2), moderate shoulder pain (VAS > 5/10), visual impairment (hemianopia), visual-spatial neglect, and/or cognitive impairments (Mini Mental State Exam (MMSE)⁴³ < 26/30).

Table 1.

Stroke patient characteristics.

Age (years)	Gender	Time since stroke (months)	Stroke type	Affected arm	FMA (0–66)
FMA ≥40
66	M	6	Ischaemic	R	64^a
54	M	22	Ischaemic	R	55^a
75	M	4	Ischaemic	L	48
57	F	7	Ischaemic	R	46
45	M	13	Haemorrhagic	L	45
52	F	5	Haemorrhagic	L	43
56	F	11	Haemorrhagic	R	43
57.9 ± 9.8	4M, 3F	9.7 ± 6.3	4I, 3H	3L, 4R	49.1 ± 7.7
FMA < 40
52	M	20	Haemorrhagic	R	30
51	F	7	Haemorrhagic	R	29
38	F	16	Ischaemic	R	29
57	M	6	Ischaemic	R	28
67	M	19	Ischaemic	L	20
53.0 ± 10.5	3M, 2F	13.6 ± 6.7	2H, 3I	1L, 4R	27.2 ± 4.1

Indicates pre-dominant motor ataxia.Note. Italics represent the averages of the above lines.

Prior to subject recruitment, ethical approval was obtained from the Domain Specific Institutional Review Board (IRB) of the National Healthcare Group (NHG), Singapore. All subjects gave written informed consent prior to screening procedures and recruitment (clinical-trial ID: NCT02188628 – clinicaltrials.gov). Also, written informed consent was provided by all patients for patient information to be published. The study was conducted in accordance with the declaration of Helsinki.

Apparatus and protocol

The experimental apparatus used for the study is the rehabilitation robot H-Man⁴⁰ (Figure 1): a compact planar, upper extremity robot designed for the use in rehabilitation settings and for human motor control experiments in stroke and neurologically healthy participants.^44,45 The participant was seated in a height-adjustable chair in front of H-Man that was placed on a fixed table, such that the center of the sternum was aligned with the handle of the H-Man robot and the elbow bent at 90°. A display was used to provide visual feedback and the representation of the task. The visual stimuli consisted of the start and target positions, the cursor position and task instructions. The participant’s trunk was physically restrained to limit trunk movements during the task. At the start of each trial, a target was visually displayed on the computer monitor in one of the contralateral, ipsilateral and sagittal plane directions (angles of −45°, 0°, and +45° from the vertical axis, respectively) at a distance of 16 cm to the initial position. The participant grasped the robot’s handle (if needed a wrist strap was provided) and moved the cursor from the start position to the target (point-to-point reaching task).

Figure 1.

A participant using the current version of H-Man. The visual interface illustrates the reaching task by providing visual cues for the start position (‘Start’), the current position (‘Cursor’) and the target position (‘Target’).

During the point-to-point reaching task, assistance was provided via a target attraction impedance controller^46,47 by the equation:

F = K (x - x_{target}) + B \dot{x}

where K is the stiffness, B the damping, x the present position (user-controlled cursor) and x_target the final position. The assistance was rendered as a pulling force between the user-controlled cursor position and the target position (virtual spring-damper system). The protocol was carried out in five training sessions, each of 60 min, over a period of two weeks with supervision from one occupational therapist and engineers. In each of the five sessions, the point-to-point reaching task was performed with a different, but fixed, level of assistance, i.e. different levels of stiffness K (50, 110, 170, 230, and 290 N/m). The order of the assistance levels was randomized for each participant.

After the two-week program, patients performed two additional sessions (each 60 min), during which one rehabilitation therapist (the same therapist for all subjects who had many years of experience of occupational and robotic therapy with stroke patients) was asked to tune the assistance levels to induce a maximum learning effect for the respective participant. Starting from a medium level of assistance for each patient, the therapist could adjust (increasing or decreasing) the assistance level at any time while observing the patient’s movements, if considered useful for learning.

Data and statistical analysis

Participants were divided into two impairment groups based on the FMA score before the commencement of the intervention¹⁷: Five patients were assigned to the moderately to highly impaired (FMA <40) group and seven to the mildly impaired group (FMA ≥40).

For the first part of the study, in which assistance levels were systematically varied, the raw kinematic data (position and velocity) were filtered using a low pass filter (Butterworth: 6th order, cut-off frequency Fc: 20 Hz, sampling rate Fs: 1000 Hz). The filtered data were used in offline data processing to calculate the task performance indices adopted from the literature.⁴⁸ For each level of assistance, the data across the three directions were considered as tasks on a planar workspace and hence were combined in the analysis. Task performance in presence of the different levels of assistance was evaluated based on metrics that are considered sensitive to the motor condition and thus are of importance for movement evaluation.^49,50 As such, a smoothness metric spectral arc length (SAL)⁵¹ and a temporal performance metric normalized total time (T_norm) were chosen. The smoothness metric SAL is a dimensionless measure of the length of the frequency spectrum curve of a speed profile over the bandwidth appropriate for the action. Movement smoothness is considered as an important indicator for motor re-learning in stroke patients, allowing for the quantification of sub-movements and thus of movement efficiency. There is ample evidence that reaching movements get smoother with motor learning⁵² and post-stroke motor recovery.⁵⁰ T_norm is a measure of the temporal performance of each trial defined as the total time needed for the completion of a trial divided by the maximum distance covered in the respective trial. Temporal performance serves as an indicator for paresis⁵³ and somatosensory loss,⁵⁴ and is expected to improve with recovery.⁴⁹

Differences in task performance metrics due to the different assistance levels and impairment groups were evaluated using a two-way analysis of variance (ANOVA) with group as the between-subject factor and assistance level as the within-subject factor.

Significant main effects and interactions were compared using Tukey's honest significant difference test (HSD).

For the analysis of the therapist’s tuning of the robotic assistance, we analyzed the final assistance level that the therapist considered optimal for maximal learning for the respective patient.

Results

Task performance

Spectral arc length (SAL) values as a function of assistance level and group are shown in Figure 2. Smoother movements were observed for higher levels of assistance, F(4,50) = 8.60, p < 0.001. The change in smoothness appeared to reach a plateau with an increase in the level of assistance. This was verified by Tukey’s HSD test that indicated that movements were smoother for the lowest assistance level (SAL(level 1) = −2.72 ± 0.41) compared to all other assistance levels (SAL(level 2) = −2.45 ± 0.30, p_Level1-2 < 0.05; SAL(level 3) = −2.31 ± 0.18, p_Level1-3 < 0.001; SAL(level 4) = −2.31 ± 0.14, p_Level1-4 < 0.001; SAL(level 5) = −2.22 ± 0.12, p_Level1-5 < 0.001). Differences in SAL between all other levels did not reach significance (all p’s > 0.05). In terms of inter-group performance variations, movements performed by the mildly impaired group were smoother than those performed by the highly impaired group (SAL(FMA ≥ 40) = −2.34 ± 0.26 and SAL(FMA < 40) = −2.48 ± 0.35), F(1,50) = 4.88, p < 0.05. The interaction between the level of assistance and impairment group was found non-significant, F(4,50) = 1.20, p = 0.322.

Figure 2.

Smoothness performance (mean and standard deviation) in terms of spectral arc length (SAL) in presence of different levels of assistance of both impairment groups. Smoothness performance saturated from level 2 of assistance onwards (*indicates significance: p < 0.05).

Normalized total time (T_norm) values dependent on assistance level and group are shown in Figure 3. In general, movements were completed in a shorter time period as assistance level increased, F(4,50) = 6.46, p < 0.001. As for the smoothness performance, post hoc analysis showed that performance in terms of T_norm was significantly worse for the lowest assistance level (T_norm(level 1) = 14.32 ± 7.64 s/m) than for all other levels (T_norm(level 2) = 10.76 ± 4.26 s/m, p_Level1-2 < 0.05; T_norm(level 3) = 9.93 ± 4.04 s/m, p_Level1-3 < 0.05; T_norm(level 4) = 9.51 ± 3.27 s/m, p_Level1-4 < 0.05; T_norm(level 5) = 8.06 ± 2.65 s/m, p_Level1-5 < 0.001). Differences in T_norm between all other levels were not found to be significant (all p’s > 0.05). There was also a significant main effect of impairment group on T_norm (F(1,50) = 42.12, p < 0.001), such that the mildly impaired group exhibited shorter total movement times than the moderately to highly impaired group (T_norm(FMA ≥ 40) = 8.07 ± 3.10 s/m and T_norm(FMA < 40) = 13.95 ± 5.20 s/m). The interaction between the level of assistance and impairment group was non-significant, F(4,50) = 2.39, p > 0.05.

Figure 3.

Performance (mean and standard deviation) in terms of total time normalized (T_norm) in presence of different assistance levels for both impairment groups. T_norm in the lowest level of assistance was significantly higher than in all other levels (levels 2–5), hence assistance levels higher than level 2 did not provoke further T_norm performance changes (*indicates significance: p < 0.05).

Therapist’s tuning of robotic assistance

For most patients (91.7%, n = 11), the therapist tuned the final assistance levels to a level lower than or equal to 175 N/m. The only case in which the therapist adjusted the assistance to a level higher than 175 N/m was for the patient with the lowest FMA score (FMA = 20).

Discussion

This study investigated the relationship between robotic assistance and performance in 12 subacute and chronic stroke patients to find an assistance threshold after which no further performance gain can be achieved. Further, the study yielded an understanding of robotic assistance for motor learning from the rehabilitation therapist’s viewpoint.

Overall, we observed a significant difference between groups for both performance metrics, whereby the mildly impaired stroke patients exhibited smoother movements and shorter movement times than the moderately to highly impaired patients. In addition, there was a trend toward smoother movements and shorter movement times as the level of robotic assistance increased. However, statistical analysis indicated that motor performance reached saturation at K =110 N/m, after which higher levels of robotic assistance did not yield further improvements in either movement smoothness or movement times. The observation of this performance saturation indicates that when higher assistance levels are utilized in a robotic rehabilitation protocol for post-stroke upper limb dysfunction, the robot’s input outweighs that of the patient (i.e. the robot is taking over most of the work required to complete the task). This, arguably, results in a reduction of overall effort by the user who allows the robotic device to move the upper limb to the target with minimal participation. Consequently, patients may modulate their force production based on the applied assistive force during task performance (i.e. slacking),^55,56 which ultimately reduces the possibility that robotic rehabilitation training would provoke somatosensory stimulation and initiate brain plasticity^23,24 and hence learning. This explanation is consistent with the work of Jarrassé et al.⁵⁷ in which the interaction between the patient and the rehabilitation robot is described akin to a teacher–student relationship. In this framework, the main purpose of the teacher (i.e. robot) is to assist the student (i.e. patient) in building his/her own capacity, rather than the robot exerting unidirectional control over the task performance (master-slave interaction). An assistance level above the threshold would arguably induce a master–slave interaction instead of the desired teacher–student relationship.

Moreover, results from the two manual tuning sessions indicated that the rehabilitation therapist selected values lower (or equal to) 175 N/m as the final assistance level for all but the most impaired stroke patient (i.e. FMA = 20). Although this value is higher than the assistance threshold obtained from the kinematic data, it is apparent from the viewpoint of a therapist with extensive robotic rehabilitation experience, that levels of assistance above 175 N/m are not required to induce optimal motor learning for a post- stroke population with FMA higher than 20. We do, however, acknowledge that the manual tuning data are preliminary, especially in light of the fact that we received input from a single rehabilitation clinician, and that we conducted only two manual tuning sessions. Nonetheless, clinicians are a critical partner in the delivery of decentralized rehabilitation, and our future work will investigate inter-therapist variations in manual tuning as a function of clinical experience and patient upper limb dysfunction.

Both parts of the study reveal that high assistance is not required for motor learning. Our study aimed to investigate a threshold for the performance enhancement approach. Combining the findings of both parts of the study, a stiffness threshold of 175 N/m seems appealing. As with any study, the present experiment comes with some limitations: First, while the results of our study suggest that high levels of robotic assistance do not improve motor performance, there is the possibility that the stiffness threshold may be different for robotic devices with dissimilar mechanical structures.

Moreover, the assistance levels used in the present study were based on the available power range of the H-Man device, and thus we cannot rule out the possibility that stiffness values greater than 270 N/m would not promote further motor performance. Second, the present study focused on motor performance and, therefore, we cannot directly translate our findings to motor learning since performance and learning are not necessarily related. Based on the current knowledge of motor learning, we assume that assistance levels higher than 175 N/m will not further foster learning. This hypothesis, however, needs to be confirmed in future work. Next, our findings cannot be generalized to the whole population of stroke patients given that the sample size was relatively small (n = 12) and patients with severe levels of upper limb weakness (i.e. FMA < 20) and comorbid difficulties were not eligible to participate. Given the heterogeneous nature of stroke characteristics and post-stroke upper limb impairments, future research will focus on a larger number of stroke patients across a broader neurological profile (e.g. FMA < 20) in order to fully evaluate the relationship between motor performance and robotic assistance.

Despite these limitations, the findings of our study have great implications for the design of future rehabilitation robotic systems aiming for decentralized care. Lower powered devices may suffice in providing the required assistance for optimal motor learning. Although a quantifiable power safety limit for devices employed in decentralized settings cannot be provided yet, it is indisputable that the understanding that high assistance levels are unnecessary makes decentralized care more realizable.

Conclusion

This paper investigated motor performance variations (smoothness and movement time) in presence of different levels of haptic assistance with the upper limb rehabilitation robot H-Man. Results show a performance saturation for high levels of assistance (K ≥ 110 N/m) as those assistance levels did not yield further performance improvements. We postulate that the performance saturation (level 2–5 [K = 110–290 N/m]) is a result of the robot taking over most of the work required to complete the task. Likely, this promotes slacking in the user and consequently, learning is not further promoted. The manual tuning behavior of the therapist points in the same direction since the final assistance level was set to maximal K = 175 N/m for all but the most impaired patient. These findings are of great importance for the development of robots that target decentralized care: Lower assistance levels directly translate to the power requirements of a device.

Lower powered devices indisputably make decentralized care more realizable.

Footnotes

Authors' Note

Asif Hussain is also affiliated with ARTICARES Pte. Ltd.

Declaration of conflicting interests

AH and DC hold equity positions in ARTICARES Pte. Ltd, a company that manufactures this type of technology under license from Nanyang Technological University, Singapore.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the ‘H-Man’ project (NMRC/BnB/0006b/2013), Ministry of Health, Singapore, and NUS Graduate School for Integrative Sciences and Engineering, National University of Singapore, Singapore.

Guarantor

SK.

Contributorship

AH, AB, WD, CH, CK, LX, KC, DC conceptualized the study. KC, DC acquired the funding. LY, KC, DC administered the project. AH, AB, WD developed the software. VD, CK, CYN, LY, KC recruited the patients. AH, AB, WD, CH, VD, CK, CYN conducted the experiments. MA, KC, DC supervised the project. SK, AH, LX analyzed the data.

SK, AH, CH wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Acknowledgements

We would like to thank all patients for their participation.

ORCID iDs

Simone Kager

Domenico Campolo

References

Goodkind

Kowal

PR.

An aging world: 2015. Washington, DC: United States Census Bureau, 2016.

Feigin

Lawes

CMM

Bennett

, et al. Stroke epidemiology: a review of population-based studies of incidence, prevalence, and case-fatality in the late 20th century. Lancet Neurol 2003; 2: 43–53.

Ovbiagele

Nguyen-Huynh

MN.

Stroke epidemiology: advancing our understanding of disease mechanism and therapy.

Neurotherapeutics 2011; 8: 319–329.

Chong

Sacco

RL.

Risk factors for stroke, assessing risk, and the mass and high-risk approaches for stroke prevention.

Contin Lifelong Learn Neurol 2013; 11: 18–34.

Hughes

CML

Tommasino

Budhota

, et al. Upper extremity proprioception in healthy aging and stroke populations, and the effects of therapist- and robot-based rehabilitation therapies on proprioceptive function. Front Hum Neurosci 2015; 9: 120.

Kwakkel

Kollen

Wagenaar

RC.

Therapy impact on functional recovery in stroke rehabilitation: a critical review of the literature.

Physiotherapy 1999; 85: 377–391.

Gresham

Fitzpatrick

Wolf

, et al. Residual disability in survivors of stroke – the Framingham study. N Engl J Med 1975; 293: 954–956.

Wade

Wood

Langton-Hewer

, et al. The hemiplegic arm after stroke: measurement and recovery. J Neurol Neurosurg Psychiatry 1983; 46: 521–524.

Lawrence

Coshall

Dundas

, et al. Estimates of the prevalence of acute stroke impairments and disability in a multiethnic population. Stroke 2001; 32: 1279–1284.

10.

Babaiasl

Mahdioun

Jaryani

, et al. A review of technological and clinical aspects of robot-aided rehabilitation of upper-extremity after stroke. Disabil Rehabil Assist Technol 2016; 11: 263–280.

11.

Maciejasz

Eschweiler

Gerlach-Hahn

, et al. A survey on robotic devices for upper limb rehabilitation. J Neuroeng Rehabil 2014; 11: 3.

12.

Brackenridge

V Bradnam

Lennon

, et al. A review of rehabilitation devices to promote upper limb function following stroke. Neurosci Biomed Eng 2016; 4: 25–42.

13.

Brokaw

Murray

Nef

, et al. Retraining of interjoint arm coordination after stroke using robot-assisted time-independent functional training. J Rehabil Res Dev 2011; 48: 299–316.

14.

Guarino

Richards

, et al. Robot-assisted therapy for long-term upper-limb impairment after stroke. N Engl J Med 2010; 362: 1772–1783.

15.

Kwakkel

Kollen

Krebs

HI.

Effects of robot-assisted therapy on upper limb recovery after stroke: a systematic review.

Neurorehabil Neural Repair 2008; 22: 111–21.

16.

Norouzi-Gheidari

Archambault

Fung

Effects of robot-assisted therapy on stroke rehabilitation in upper limbs: systematic review and meta-analysis of the literature.

J Rehabil Res Dev 2012; 49: 479–96.

17.

Lum

Burgar

Shor

, et al.

Robot-assisted movement training compared with conventional therapy techniques for the rehabilitation of upper-limb motor function after stroke.

Arch Phys Med Rehabil 2002; 83: 952–959.

18.

Winstein

CJ.

Knowledge of results and motor learning- implications for physical therapy.

Phys Ther 1991; 71: 140–149.

19.

Milot

Marchal-Crespo

Green

, et al. Comparison of error-amplification and haptic-guidance training techniques for learning of a timing-based motor task by healthy individuals. Exp Brain Res 2010; 201: 119–131.

20.

Feygin

Keehner

Tendick

Haptic guidance: experimental evaluation of a haptic training method for a perceptual motor skill. In: Proceedings 10th symposium on haptic interfaces for virtual environment and teleoperator systems, Orlando, FL, USA, March 2002, pp. 40–47. IEEE.

21.

Liu

Cramer

Reinkensmeyer

DJ.

Learning to perform a new movement with robotic assistance: comparison of haptic guidance and visual demonstration.

J Neuroeng Rehabil 2006; 3: 20.

22.

Christina

Bjork

RA.

Optimizing long-term retention and transfer. In: Druckman

Bjork

(eds) In the mind’s eye: Enhancing human performance. Washington DC: National Academy Press, 1991, pp. 23–56.

23.

Poon

C-S.

Sensorimotor learning and information processing by Bayesian internal models.

Conf Proc IEEE Eng Med Biol Soc 2004; 6: 4481–4482.

24.

Rossini

Dal Forno

Integrated technology for evaluation of brain function and neural plasticity.

Phys Med Rehabil Clin 2004; 15: 263–306.

25.

Schmidt

Bjork

RA.

New conceptualizations of practice: common principles in three paradigms suggest new concepts for training. Psychol Sci 1992; 3: 207–217.

26.

Williams

Carnahan

Motor learning perspectives on haptic training for the upper extremities.

IEEE Trans Haptics 2014; 7: 240–250.

27.

Heuer

Lüttgen

Robot assistance of motor learning: a neuro-cognitive perspective.

Neurosci Biobehav Rev 2015; 56: 222–240.

28.

Emken

Benitez

Reinkensmeyer

DJ.

Human-robot cooperative movement training: learning a novel sensory motor transformation during walking with robotic assistance-as-needed.

J Neuroeng Rehabil 2007; 4: 8.

29.

Franklin

Burdet

Peng Tee

, et al. CNS learns stable, accurate, and efficient movements using a simple algorithm. J Neurosci 2008; 28: 11165–11173.

30.

Lotze

Braun

Birbaumer

, et al. Motor learning elicited by voluntary drive. Brain 2003; 126: 866–872.

31.

Perez

Lungholt

BKS

Nyborg

, et al. Motor skill training induces changes in the excitability of the leg cortical area in healthy humans. Exp Brain Res 2004; 159: 197–205.

32.

Wolbrecht

Chan

, et al. Real-time computer modeling of weakness following stroke optimizes robotic assistance for movement therapy. In: 3rd international IEEE/EMBS conference on neural engineering, Kohala Coast, HI, USA, May 2007, pp.152–158. IEEE.

33.

Guadagnoli

Lee

TD.

Challenge point: a framework for conceptualizing the effects of various practice conditions in motor learning.

J Mot Behav 2004; 36: 212–224.

34.

Hogan

Krebs

Charnnarong

, et al. MIT-MANUS: a workstation for manual therapy and training. I. In: Proceedings of the IEEE international work on robot human communication, Tokyo, Japan, September 1992, pp. 161--165. IEEE.

35.

Nef

Mihelj

Colombo

, et al. ARMin-robot for rehabilitation of the upper extremities. In: Proceedings of IEEE international conference on robotics and automation, Orlando, FL, USA, May 2006, pp. 3152–3157. IEEE.

36.

Linde

R Van Der

Lammertse

Frederiksen

, et al. The HapticMaster, a new high-performance haptic interface. In: Proceeding of EuroHaptics, Edinburgh, UK, July 2002, pp.1–5. Scotland, UK, University of Edinburgh and Edinburgh College of Art.

37.

Kahn

Lum

Rymer

, et al. Robot-assisted movement training for the stroke-impaired arm: does it matter what the robot does? J Rehabil Res Dev 2006; 43: 619–630.

38.

Sivan

Gallagher

Makower

, et al. Home-based computer assisted arm rehabilitation (hCAAR) robotic device for upper limb exercise after stroke: results of a feasibility study in home setting. J Neuroeng Rehabil 2014; 11: 163.

39.

Perry

Zabaleta

Belloso

, et al. ArmAssist: development of a functional prototype for at-home telerehabilitation of post-stroke arm impairment. In: 4th IEEE RAS & EMBS international conference on biomedical robotics and biomechatronics (BioRob), Rome, Italy, June 2012 pp. 1561–1566. New York: IEEE.

40.

Campolo

Tommasino

Gamage

, et al. H-Man: a planar, H-shape cabled differential robotic manipulandum for experiments on human motor control. J Neurosci Methods 2014; 235: 285–297.

41.

Fugl-Meyer

Jääskö

Leyman

, et al. The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand J Rehabil Med 1975; 7: 13–31.

42.

Bohannon

Smith

MB.

Interrater reliability of a modified ashworth scale of muscle spasticity.

Phys Ther 1987; 67: 206–207.

43.

Folstein

McHugh

PR.

“Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician.

J Psychiatr Res 1975; 12: 189–198.

44.

Hussain

Tommasino

Hughes

CML

, et al. Technology intervention in neurorehabilitation – a practical approach to teaching. In: International conference on robotics in education, Yverdon-les-Bains, Switzerland, 20--22 May 2015, pp. 116–119.

45.

Hussain

Dailey

Hughes

, et al. Quantitative motor assessment of upperlimb after unilateral stroke: a preliminary feasibility study with H-Man, a planar robot. In: IEEE international conference on rehabilitation robotics, Singapore, August 2015, pp. 654–659. IEEE.

46.

Marchal-Crespo

Reinkensmeyer

DJ.

Review of control strategies for robotic movement training after neurologic injury.

J Neuroeng Rehabil 2009; 6: 20.

47.

Hogan

Impedance control: an approach to manipulation. In: Proceedings of American control conference, San Diego, CA, USA, June 1984, pp.304–313. IEEE.

48.

Balasubramanian

Colombo

Sterpi

, et al. Robotic assessment of upper limb motorfunction after stroke. Am J Phys Med Rehabil 2012; 91: S255–S269.

49.

Nordin

Xie

Wünsche

Assessment of movement quality in robot- assisted upper limb rehabilitation after stroke: a review.

J Neuroeng Rehabil 2014; 11: 137.

50.

Bosecker

Dipietro

Volpe

, et al. Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabil Neural Repair 2010; 24: 62–69.

51.

Balasubramanian

Melendez-Calderon

Burdet

A robust and sensitive metric for quantifying movement smoothness.

IEEE Trans Biomed Eng 2012; 59: 2126–2136.

52.

Berthier

Keen

Development of reaching in infancy.

Exp Brain Res 2006; 169: 507–518.

53.

Lang

Bland

Bailey

, et al. Assessment of upper extremity impairment, function, and activity after stroke: foundations for clinical decision making. J Hand Ther 2013; 26: 104–115.

54.

Coderre

Zeid

Dukelow

, et al. Assessment of upper-limb sensorimotor function of subacute stroke patients using visually guided reaching. Neurorehabil Neural Repair 2010; 24: 528–541.

55.

Reinkensmeyer

Wolbrecht

Bobrow

A computational model of human- robot load sharing during robot-assisted arm movement training after stroke.

Annu Int Conf IEEE Eng Med Biol 2007; 3975: 4019–4023.

56.

Casadio

Sanguineti

Learning, retention, and slacking: a model of the dynamics of recovery in robot therapy. IEEE Trans Neural Syst Rehabil Eng 2012; 20: 286--296.

57.

Jarrassé

Charalambous

Burdet

A framework to describe, analyze and generate interactive motor behaviors. PLoS One 2012; 7: e49945.