The Effects of Robot-Enhanced Psychotherapy: A Meta-Analysis

Abstract

Through this meta-analysis we aimed to provide an estimation of the overall effect of robot-enhanced therapy on psychological outcome for different populations, to provide average effect sizes on different outcomes, such as cognitive, behavioral and subjective, and to test possible moderators of effect size. From a total of 861 considered studies for this meta-analysis, only 12 were included because of the lack of studies that have reported quantitative data in this area and because of their primary focus on describing the process of robotic development rather than measuring psychological outcomes. We calculated Cohen's d effect sizes for every outcome measure for which sufficient data were reported. The results show that robot-enhanced therapy yielded a medium effect size overall and, specifically on the behavioral level, indicating that 69% of patients in the control groups did worse than the average number of participants in the intervention group. More studies are needed with regard to specific outcomes to prove the efficacy of robot-enhanced therapy, but the overall results clearly support the use of robot-enhanced therapy for different populations.

Keywords

robot-enhanced/assisted therapy psychological outcomes meta-analysis

Rapid progress in the development of interactive technologies and their accessibility offer the possibility for innovation in psychotherapy for individuals with mental health disorders. Some of the technological tools used have already undergone systematic testing and their effectiveness have been synthesized in meta-analytical studies; see the case of online delivered cognitive–behavioral therapy (CBT) and computer-based CBT (Mureşan, Montgomery, & David, 2012; Reger & Gahm, 2009), as well as virtual reality-based CBT (Opriş et al., 2012; Powers & Emmelkamp, 2008). Recent advances in robotics have also enabled social robots to fulfill a variety of functions in the psychotherapeutic process.

A social robot may be defined as an artificially intelligent system that has a physical embodiment, is autonomous or semiautonomous, and interacts and communicates with humans by following the behavioral norms expected by the people with whom the robot is intended to interact (Bartneck & Forlizzi, 2004). In the paradigm developed by Feil-Seifer and Mataric (2005), “the robot's goal is to create close and effective interaction with a human user for the purpose of giving assistance and achieving measurable progress in convalescence, rehabilitation, learning, and so forth (p. 465).” Libin and Libin (2004) also tried to define the role of the robot in human–robot interactions and they introduced the term “robotherapy,” defined “as a framework of human–robotic creature interactions aimed at the reconstruction of a person's negative experiences through the development of coping strategies, mediated by technological tools, to provide a platform for building new positive life skills” (pp. 1792–1793). David, Matu, and David (2014) suggested that the term robotherapy should be replaced by robot-assisted/enhanced therapy and defined as “the use of robots in a personalized evidence-based psychotherapy framework, where the robot should be seen as a technological tool that can help the psychotherapists to accomplish their clinical roles and aims” (p. 4).

The technological progress in robotics has focused on the development of special characteristics of physically embodied agents to meet the special needs of children, adults or elderly with cognitive, physical or social disabilities. (Libin & Libin, 2004). For example, by 2060, 30% of the population of Europe will be 65 years of age or older, compared with 17% in 2010 (Eurostat, 2010; http://epp.eurostat.ec.europa.eu/). This demographic shift will have enormous economic impact (e.g., health, pensions, long-term care) and create an unprecedented demand on younger citizens to care for the elderly. The studies on human–robot interaction may help alleviate this burden (Prescott et al., 2012). In this context, most studies in this field are focusing on human optimization for the elderly or individuals with autism.

Human Optimization

Human optimization refers to a system of strategies designed to help people improve their skills; achieve their maximum potential, productivity, and performance, while also enhancing well-being. Investigations of robotic interactions for future use in daily life have increased intensively in recent years, in areas such as health care, education or entertainment. There are several studies that have investigated a robot's influence on a person's behavior and performance, for example, on the level of enjoyment (Kidd & Breazeal, 2004), task engagement (Jung & Lee, 2004; Wainer, Feil-Seifer, Shell, & Mataric, 2007), trust and respect toward the robot (Bickmore & Cassell, 2001), and also several that investigated the general perception of the presence of social robots (Wainer et al., 2007; Jung & Lee, 2004; Takayama, & Pantofaru, 2009). Previous work has shown that physically embodied agents are consistently perceived as more engaging than a character on a video display and sometimes as engaging as a human (Bainbridge, Hart, Kim, & Scassellati, 2008). Also, studies have shown that the presence of the robot may elicit more task engagement or better task performance (Jung & Lee, 2004; Burgoon et al., 2004). On the other hand, some studies have shown that any type of presence might impair the subjects’ cognitive performance, especially when a task is new or difficult (social facilitation paradigm; Riether, Hegel, Wrede, & Hortman, 2012; Hoyt, Blascovich, & Swinth, 2003; Zanbaka, Ulinski, Goolkasian, & Hodges, 2007). These studies were conducted on both virtual and robotic agents and showed that an agent is able to elicit the same social facilitation effects (enhancement on easy tasks, impairment on complex ones) as a human confederate when compared with the alone condition (Park & Catrambone, 2007).

Elderly with Social Problems

Many different studies have also reported positive reactions of elder persons to assistive social robots. As a wide variety of research designs were used, and many of these studies indicate positive outcomes of the effect of companion robots on the elderly, results are discussed in several specialized reviews (Bemelmans, Gelderblom, Jonker, & de Witte, 2012; Broekens, Heerink, & Rosendal, 2009; Mordoch, Osterreicher, Guse, Roger, & Thompson, 2013). These reviews reveal that social robots seem to have positive effects on the well-being of the elderly population, on social problems and physical and cognitive impairments, even for those diagnosed with dementia. Researchers have also investigated the use of robots to improve well-being of the elderly, reduce their emotional problems, and increase their social interaction with peers. For example, Wada, Shibata, Saito, and Tanie (2002) found that the robot Paro was able to improve the mood of elderly participants who had spent time interacting with it over the course of a 6-week period. In 2006, Kidd, Taggart, and Turkle observed that seniors who had the robot with them in a group were more likely to interact socially with each other when the robot was present, compared with when it was absent. However, more rigorous research is needed to clarify the effectiveness (in terms of effect size, clinical relevance) and the mechanisms of change in the case of social robots for the elderly.

Autism-Spectrum Disorder (ASD)

Another important type of population in the human–robot interaction field is represented by individuals with ASD. Many research groups have studied in detail how social robots positively affect social interaction in children with ASD. Recently, Diehl, Schmitt, Villano and Crowell mentioned in a review in 2012 that individuals with ASD (a) exhibit strengths in understanding the physical (object-related) world and exhibit weaknesses in understanding the social world; (b) are more responsive to feedback, even social feedback, when administered through technology rather than via human interaction; and (c) are more intrinsically interested in treatment when it involves electronic or robotic tools. Considering these characteristics of the autistic condition, several important programs have been developed in the field, such as (a) the AuRoRA Project (Dautenhahn, 1999; Dautenhahn & Werry, 2004) and (b) the IROMEC Project (Interactive Robotic Social Mediators as Companions), both of which investigate the use of robots as tools. Some research has also looked into the educational or therapeutic role of robotics for children with ASD (Besio, Caprino, & Laudanna, 2008).

Among these projects and other studies, it has been shown that children with ASD who benefit from human–robot interaction may exhibit (a) increased levels of engagement, (b) a wide range of positive social behaviors (e.g., spontaneous initiations, social play behaviors), and (c) increased attention during the child–robot interaction sessions (Scassellati, Admoni, & Mataric, 2012; Ricks & Colton, 2010; Michaud & Clavet, 2001; Robins, Amirabdollahian, Ji, & Dautenhahn, 2010; Kozima, Nakagawa, & Yasuda, 2005; Vanderborght et al., 2012; Tapus et al., 2012). Yet most of the support for the use of social robots in therapy is based on case studies and designs with major limitations; thus, such use lacks support for the generalization of the improved skills (Ricks & Colton, 2010; Diehl et al., 2012).

Although there are a few reviews in this domain that have investigated the effects of robot-assisted therapy on different pathologies or populations, e.g., effects on upper limb recovery after stroke (Kwakkel, Kollen, & Krebs, 2008), effects on different abilities of children with ASD (Diehl et al., 2012) or on the elderly (Broekens, Heerink, & Rosendal, 2009), there have been no attempts to quantitatively assess the effect of robot-enhanced therapy on psychological outcomes. Also, there are studies that have tried to explain how different characteristics of the human–robot relation, for example, trust in robots, may influence the interaction between the two agents (Hancock et al., 2011). But, unlike our study, they focus only on a small part of this process. In this context, the study we propose is the first quantitative meta-analysis on social robots, which takes into consideration the effects of robot-enhanced therapy on psychological outcomes.

Scope of the Present Meta-Analysis

This study outlines the current status of the field and takes an important step forward by including existing studies in a quantitative meta-analysis. To ensure appropriate development of social robot applications in psychology, professionals must have a clear understanding of the opportunities and challenges such applications will provide in professional practice. Moreover, we consider that clinical studies are very relevant to the development of social robots for their use in psychotherapy.

Through this meta-analysis we aimed to (a) provide an estimation of the overall effect of robot-enhanced therapy on psychological outcomes for different populations; (b) provide average effect sizes on different outcomes, such as cognitive, behavioral, and subjective; and (c) test possible moderators of effect size. Also, in the context of the current modalities in which social robots are being used to address different types of clinical problems, there are still some questions that might be answered through our study, for example, what type of tasks we should use in human–robot interactions or what type of outcome is more influenced by the use of social robots in psychotherapy.

Method

Inclusion of Studies

We have included in our meta-analysis studies that report quantitative data regarding the use of social robots in specific tasks that have as the outcome psychological measures. We have also compared the use of this type of agent with other types of intervention that did not include a social robot. The dependent variables that we focused on were (a) cognitive performance (e.g., anagrams, puzzles), (b) behavioral level (e.g., prosocial behaviors), and (c) subjective level (e.g., mood, perceived pain).

The data-collection process consisted of a systematic search of PubMED, PsycINFO, and IEEExplore (http://ieeexplore.ieee.org) for records from 1990 until June, 2013 to identify all the studies that aimed to assess the effects of robot-enhanced therapy. Because the majority of the studies in this field are published in technological journals and through conference proceedings and not in psychological journals, we have included a representative sample of papers from IEEExplore database. These databases were searched using the following terms: robot psychology, robotherapy, robot psychotherapy, robot autism, robot elderly, robot-assisted learning, robot-assisted therapy. We also systematically searched the references from recent studies and reviews on the topic (Diehl et al., 2012; Broekens, Heerink & Rosendal, 2009).

The inclusion criteria were (a) to report psychological outcomes that resulted from a comparison between the effects of robot-enhanced therapy and interaction with a human or a nonrobotic object, (b) to have multiple participants to form a group, (c) to report quantitative data to calculate the effect size, and (d) to be written in English. We did not include studies that reported case studies or single-case experiments that used robots in both conditions (experimental and control) or studies that only applied pretest and posttest measurements for a single group of subjects.

After the initial search we identified a total of 955 records from databases and added 17 more records, which we have considered to be relevant from other sources, including references from other relevant papers. We removed 111 duplicates (papers that appeared twice in our database) and then we screened 861 records through their abstracts. A total of 103 articles were retained to be assessed for eligibility (see Figure 1). Only 12 studies were included in the meta-analysis. The other studies were excluded because they were either theoretical reviews or descriptions of the technological process of developing the robot or case studies. We also excluded studies because researchers neither included a group with an alternative condition nor provided sufficient data to calculate effect size.

Figure 1

PRISMA (preferred reporting items for systematic reviews and meta-analyses) flow chart.

Moderators

There are a number of potential moderators of the effect of robot-enhanced therapy on the psychological outcomes, as identified through literature search and suggestions from previous studies (David et al., 2014). After analyzing potential studies for our meta-analysis, we also decided (a posterori) to consider the following moderators:

a. Function of the robot in the session

Possible roles of robotic agents in psychotherapy as described by David et al. (2014, p. 6) such as

1. Mediator

“The robot mediates the activities of the therapists; to be able to implement the activities the therapist needs the meditating role of the robot (if the robot is not used the therapist activities cannot be implemented and/or are less efficient; e.g., in this case the robot acts as a necessary and specific “catalyst” that enables or accelerates treatment progress, by mediating the interaction between the therapist and his or her clients).”

2. Therapist

“The robot replaces the therapist, having a direct relation with the client; the actions of the robot are programmed and supervised by the therapist (e.g., robots can virtually function as psychotherapists and even completely replace therapists when they are unavailable).”

3. Assistant

“The robot facilitates the activities of the therapist; the robot is seen as a possible tool that optimizes the therapist's activities, although the optimization of these activities can be based on a variety of tools (e.g., the robot in this case may or may not be used in the psychotherapeutically process, since they are not being used as principal vehicle for providing psychotherapy services).”

b. The type of the control condition

Computer, human, no help, toy.

c. Robot type

Human-like face, nonhuman-like face. To categorize the robots, we took their facial appearance as a distinction criterion, whether or not the robot had the face of a human.

d. Design

Experimental, quasi-experimental.

e. Population

Clinical, nonclinical.

Coding

For every study included in the meta-analysis we retained the following variables: the study identification data (author, year of publication), the type of outcome reported (cognitive, behavioral, subjective), the sample size, the age of participants, the function of the robot in the task, the type of control condition (computer, human, no help, toy), the robot type (human-like face; nonhuman-like face), the design (experimental, quasi-experimental) and the type of population (clinical, nonclinical). The dependent variables were classified considering the reported outcomes of a standard psychological intervention: (a) cognitive level, (b) behavioral level, and (c) subjective level (see Table 1).

Table 1

Characteristics of the Studies Included in the Meta-Analysis

Study	N	Type of population	Task description	Measure outcome	Robot used	Effect size (Cohen's d)
Admoni, Bank, Tan, Toneva, and Scassellati (2010)	70	Nonclinical	Joint visual attention task; attention shifts elicited by five directional stimuli: a photograph of a human face, a line drawing of a human face, Zeno's gaze, Keepon's gaze, and an arrow.	Response time to attention shifts to another agent's gaze.	Zeno human-like face Keepon nonhuman-like face	-0.097
Bainbridge, Hart, Kim, and Scassellati (2008)	65	Nonclinical	Cooperation task: Participants had to interpret the agents' pointing gestures and to move a pile of books from one location to another, pointed out by the agent.	Response time to the requests (to move objects) of the agent.	Nico human-like face	0.579
Bekele, Lahiri, Swanson, Crittendon, Warren, and Sarkar (2013)	12	Nonclinical	Joint attention task: There were six levels of prompts in each trial. These hierarchical prompts were administered stepwise if no or an inappropriate response was detected and progressed in least-to-most fashion.	Preferential looking towards the robot (percentage of time). Frequency of looking to target and number of required level of success.	Nao human-like face	0.153
Beran, Ramirez-Serrano, Vanderkooib, and Kuhn (2013)	57	Nonclinical	Interaction task with children during their vaccination: Children had to blow on some toys and watch how the agent placed the toys in the container, or simply watched the toys.	Avoidance and distress. (The Faces Pain Scale-Revised). Pain. (Behavioral Approach-Avoidance Distress Scale).	Nao human-like face	0.795
Han, Jo, Jones, and Jo (2008)	90	Nonclinical	Learning task: Participants had to listen to some English stories, and then there were some conversation exercises.	Concentration (Likert scale). Achievement (knowledge test).	Irobi human-like face	1.545
Kim, Berkovitz, Bernier, Leyzberg, Shic, Paul, and Scassellati (2012)	24	Clinical	The task consisted of manipulating blocks: multicolored, magnetic-linking tiles in the robot condition; multicolored, interlocking blocks in the adult condition, and tangrams in the computer-game condition.	Total utterances. Total utterances directed to the confederate. Total utterances directed to the conversational agent (measured in frequency).	Pleo nonhuman-like face	0.773
Leyzberg, Spaulding, Toneva, and Scassellati (2012)	100	Nonclinical	Puzzle-solving task: a nonogram puzzle. Participants had to find a pattern of shaded boxes on a blank board such that the number of consecutively shaded boxes in each row and column appeared as specified in length and order by the numbers that are printed to the left of each row and above each column.	Response time to puzzle solving.	Keepon nonhuman-like face	0.550
Pop, Simut, Pintea, Saldien, Rusu, Vanderfaeillie, David, Lefeber, and Vanderborght (2013)	13	Clinical	The task consisted of (a) listening to a social story, (b) answering some comprehensive questions, and (c) exercising the social ability targeted in the social story (e.g., sharing toys).	Level of prompt (the level of help needed to accomplish the social task). Likert Scale.	Probo nonhuman-like face	0.998
Pop, Pintea, Vanderborght, and David (2014)	11	Clinical	The task consisted in a role play where the agent was the patient, the participant was the doctor and they were supposed to help the agent to feel better. Several aches/needs were expressed by the agent to encourage the child to use the correct tools needed to recover.	Collaborative play. Eye contact, (measured in seconds). Engagement in play (Likert scale). Play performance. Positive emotions. Stereotype behaviors. Verbal initiations, (measured in frequency).	Probo non-human-like face	0.949
Riether, Hegel, Wrede, and Horstmann (2012)	106	Nonclinical	Four computerized tasks of cognitive and motoric nature (anagram solving, numerical distance, finger tapping, and motoric tracking), which were administered both in an easy and complex version.	Reaction time (time from stimulus presentation until first key press of the correctly solved anagram). Accuracy (ration of correct solution to total number of anagrams).	Flobi human-like face	-0.113
Stanton, Kahn, Severson, Ruckert, and Gill (2008)	11	Clinical	The experimenter placed the agent in front of the child and told the child it was okay to pet the artifact, and then he let the child engage in self-directed exploratory play for a few minutes. Next the experimenter asked a series of questions, while the child continued to play, and invited the child to engage in certain behavioral interactions with the artifact, such as holding it and rolling the ball to it.	Affection. Animating artefact. Authentic interaction. Autistic Behaviors. Duration of the session. Percentage of interaction. Reciprocal interaction. Spoken words. Verbal engagement.	Aibo nonhuman-like face	0.363
Wood, Dautenhahn, Rainer, Robins, Lehmann, and Syrdal (2013)	22	Nonclinical	An interview regarding a familiar event for the children. The interviews had four phases in the following order: (a) establishing a report, (b) asking for free narrative recall, (c) asking questions, and (d) closure.	Child response duration. Eye gaze. Interview duration. Remembering names and objects from the event. Proportionate word count. Response time interview. Word count.	Kaspar human-like face	0.195

Note. N= number of participants; we didn't include in our analyses all the outcomes of the studies because for some of them, no quantitative was provided and others were not relevant for our research.

The analyses were conducted using Comprehensive Meta-Analysis, Version 2.2.046 (Borenstein, Hedges, Higgins, & Rothstein, 2005). Comprehensive Meta-Analysis is a program developed specifically for meta-analyses. As such, it includes functions to automatically compute effect sizes to perform basic and advanced meta-analyses (for details, see http://www.meta-analysis.com/index.php). For analyzing the data, we chose Cohen's d as a measure of effect size. We calculated Cohen's d effect sizes for every outcome measure for which sufficient data were reported and were relevant for our study. All the effect sizes were coded such that a positive value of Cohen's d indicated a greater improvement in the robot-enhanced therapy group than in the control group. For the calculation of effect sizes, the following data were used: means and standard deviations, given that these data were available; Cohen's d reported in the original study; precise p values; and sample sizes. We computed d values using the aforementioned indicators for 96 effects.

The studies selected for this meta-analysis were originally conducted using different types of control groups, different types of interventions, and different types of outcome measures. Taking into consideration these differences, we could not assume a single true effect size for all studies selected. Therefore, we decided to use a random-effects model to analyze the data (e.g., Borenstein et al., 2009; Hunter & Schmidt, 2004). To test the assumption that the effect sizes included in each dataset estimated the same population mean, we tested for homogeneity of effect sizes using the Q statistic and the I² statistic (Borenstein et al., 2005). To address the publication bias, we calculated a fail-safe N for all effect-size subsets (Rosenthal, 1991).

Results

The Robot-Enhanced Therapy Overall Effect

The overall effect of robot-enhanced therapy, including the three levels (cognitive, behavioral, and subjective), was calculated from 12 studies including 581 participants. The results showed a medium significant effect of the robot-enhanced therapy, Cohen's d = 0.523, variance of d (Var d) = 0.022, p < .01, 95% CI [0.233, 0.814] when compared with the nonrobotic condition (e.g., human condition). There was evidence of a moderate level of heterogeneity, as shown by the following indicators, Q(11) = 21.556, p = .028, I² = 48,969; in this case we analyzed whether one of the potential moderator variables could have explained the heterogeneity found on the overall effects. We found no significant moderator for the effect of robot-enhanced therapy on the overall outcome.

These results indicate that 69% of participants from the alternative condition performed more poorly than the average performance of the participants from the robot-enhanced therapy condition (McGough & Faraone, 2009). We computed a fail-safe N for the effect of robot-enhanced therapy on the overall effect. The number of studies that would reduce the effect size to nonsignificance was 65. Rosenthal (1991) stated that the computed, fail-safe N should be larger than 5K + 10 (where K is the number of studies included in meta-analysis) to indicate a robust effect size. The computed fail-safe N did not support the robustness of the computed effect sizes, and as we included 12 studies reporting data, the fail-safe N would be expected to be more than 70.

The Effects of Robot-Enhanced Therapy on Behavioral Level

The effect of robot-enhanced therapy on behavioral level was calculated from nine studies including 247 participants. The results showed a medium significant effect of the robot-enhanced therapy, Cohen's d = 0.543, Var d = 0.014, p = .00, 95% CI [0.314, 0.722], when compared with the nonrobotic condition (e.g., human condition) and there was no evidence of heterogeneity, Q(8) = 7.579, p = .476, I² = 0.000. These results indicate that 69% of participants in the alternative condition had poorer performance than the average performance of participants from the robot-enhanced therapy condition (McGough & Faraone, 2009). We computed a fail-safe N for the effect of robot-enhanced therapy on the behavioral level and the number of studies that would reduce the effect size to nonsignificance was 65. The computed fail-safe N did not support the robustness of the computed effect sizes; as we included 9 studies reporting data, the fail-safe N would be expected to be more than 55. We found no significant moderator for the effect of robot-enhanced therapy on behavioral level.

The Effects of Robot-Enhanced Therapy on Subjective Level

The effect of robot-enhanced therapy on subjective level was calculated from three studies including 79 participants. The results showed a nonsignificant effect of the robot-enhanced therapy, Cohen's d = 0.446, Var d = 0.319, p = .162, 95% CI [−0.179, 1.072], when compared with nonrobotic condition (e.g., human condition) and there was no evidence of heterogeneity, Q(2) = 3.506, p = .173, I² = 42.952. We found no significant moderator for the effect of robot-enhanced therapy on subjective level.

The Effects of Robot-Enhanced Therapy on Cognitive Performance

The effect of robot-enhanced therapy on cognitive level was calculated from five studies including 387 participants. The results showed a small nonsignificant effect of robot-enhanced therapy on cognitive performance, Cohen's d = 0.373, Var d = 0.087, p = .207, 95% CI [−0.206, 0.952], and there was evidence of a high level of heterogeneity, as shown by the following indicators, Q(4) = 17.155, p = .002, I² = 76.683, and in this case, we analyzed whether one of the potential moderator variables could have explained the heterogeneity found on the cognitive level. We found no significant moderator for the effect of robot-enhanced therapy on the cognitive outcome.

Discussion

The results of this meta-analysis show that there is a medium significant effect of robot-enhanced therapy on improving performance on the three levels (behavioral, cognitive, and subjective) taken together, such that 69% of participants in the alternative condition had poorer performance than the average performance of the participants from robot-enhanced therapy condition (McGough & Faraone, 2009). Our findings are in line with other studies and reviews that emphasize the effectiveness of robot-enhanced therapy on specific populations or outcomes (e.g., Wada, Shibata, Saito, & Tanie, 2004; Ricks & Colton, 2010; Diehl et al., 2012).

When analyzing the data separately on the three levels considered in our study, we found a significant effect of the robot-enhanced therapy on improving the performances on the behavioral level (Cohen's d = 0.543) Similar to the findings for general performance, 69% of participants in the alternative condition had poorer behavioral performance than the average performance of participants in robot-enhanced therapy condition (McGough & Faraone, 2009). This finding addresses one important research question formulated in our meta-analysis: What type of outcome does the use of a social robot in psychotherapy impact more? We can conclude that outcomes measured at a behavioral level seem to be influenced most by robot-enhanced therapy. These findings are consistent with the results of other studies (especially from the autism field) that show great improvement in adaptive behaviors of children who receive robot-enhanced therapy (Robins et al., 2010; Kozima, Nakagawa, & Yasuda, 2005; Vanderborght et al., 2012; Tapus et al., 2012).

In contrast to the results for outcomes measured at the behavioral level, we found no difference between the effects of robot-enhanced therapy and nonrobotic therapy (e.g., human condition) on outcomes measured at the cognitive level (Cohen's d = 0.373), suggesting that clinicians may choose from the two types of interventions that which is more suitable for their clients. Furthermore, due to the recent improvements in technology, clients’ preferences tend to be more and more in favor of using technological tools in clinical contexts (Costescu & David, 2014). Therefore, by developing semiautonomous or autonomous robots, robot-enhanced therapy has the potential to reduce the cost of the therapy and to reduce the workload of the therapist.

Moreover, considering the general findings for robot-enhanced therapy on performance at the cognitive level, we examined studies included in the category of cognitive level. Of the five studies included in this category, we found that two reported negative outcomes for robot-enhanced therapy (see Table 1). Possible explanations for these negative effects may include the manner that outcomes were assessed (e.g., response times or a context in which the robot presence may have served as a distractor from the task). Also, the study conducted by Riether et al. (2012) investigated the cognitive performance of the participants in two types of tasks: easy and complex. Previous studies have shown that the presence of an agent during completion of some cognitive tasks may impair performance, especially when a task is new or difficult (Hoyt et al., 2003; Zanbaka, Ulinski, Goolkasian & Hodges, 2007; Park & Catrambone, 2007). This effect could partially explain the results obtained on the cognitive performance level. This suggests that it may be informative for future researchers to examine what specific cognitive mechanisms might be targeted or affected by robot versus human interactions (Diehl et al., 2012).

We also found no significant effect of robot-enhanced therapy on improving the performance at the subjective level (Cohen's d = 0.373). In the literature, there are a few important studies, especially on the elderly (e.g., Wada et al., 2004), which have shown great improvements on subjective levels in patients interacting with different robots (e.g., Paro). Unfortunately, we could not include those studies in our meta-analysis because the studies did not meet our inclusion criteria.

The finding of no further advantage of robot-enhanced therapy on effects of cognitive and subjective levels may be due to the variability of the outcomes and to the differences in the modalities in which these outcomes were measured. Moreover, the number of studies included in the calculation of the effect of robot-enhanced therapy on the subjective level is very small and the results are not stable. However, our results showing that robot-enhanced therapy is as effective as interventions with humans support the idea that social robots can be used as a complementary tool in therapy for specific types of populations, such as for ASD children or for specific tasks. In these instances, robot-enhanced therapy has the potential to reduce the workload of the therapist, the cost of the therapy, and to improve the specific skills of children with ASD (Thill, Pop, Belpaeme, Ziemke, & Vanderborght, 2012). Taking these results into consideration, researchers and professionals may benefit from a clear understanding of the opportunities and challenges associated with robot-enhanced therapy in professional practice.

Our results showed significant heterogeneity in the case of the investigated outcomes; we therefore conducted moderation analyses. We found no significant moderator for the effect of robot-enhanced therapy on any level. However, we could identify a trend regarding the role of the robot in therapy both on the overall effect and also on the behavioral level. We found that the most efficient interventions were those in which the robot is used as a mediator in therapy. In other words, this emphasizes the importance of the actions developed by the robot (e.g., providing feedback, responding to participants’ behaviors) compared with the simple presence of a robot in the task. In conclusion, the data suggest social robots should be used in therapy, mostly to mediate the activities of the therapists (David et al., 2014; Vanderborght et al., 2012).

One of the most important issues in this area is the small number of studies that provide quantitative data regarding the use of social robots in psychotherapy, because the majority of the studies investigating human–robot interactions are published in robotic journals. It has to be mentioned that it is difficult to investigate the clinical significance of any empirical data included in these papers because they focus on technical details of the robots rather than on psychologically relevant aspects of their methodology.

In sum, our results show that robot-enhanced therapy represents a great potential for improving standard interventions for several types of problems, including clinical populations. As mentioned above, the role of social robots in psychotherapy is to improve the therapeutic process, to reduce the symptoms associated with different psychopathologies, and to improve the quality of life for the clients. From the psychologist's point of view, in clinical practice, social robots may help them reach their objectives in psychotherapy easily and reduce the workload (especially when working with a clinical population, such as individuals with ASD).

Future Directions

Large-scale clinical trials and longitudinal studies are required to have a clearer understanding of the effectiveness of robot-enhanced therapy (Simut et al., 2012). As our results suggest that the most efficient interventions are those in which the robot is used as a mediator, it may be useful in future research to focus on developing semiautonomous or autonomous robots to reduce the cost of psychological interventions.

One limitation of our study is that we could not identify any significant source of heterogeneity of the overall effect. There seem to be other variables, in addition to those we considered, that have an influence on the overall effect of robot-enhanced therapy. Further investigations are clearly needed to identify these moderators.

The small number of articles and participants included in the meta-analysis reflects the paucity of studies in this area reporting quantitative data. In future studies, researchers should include quantitative measures and they should compare the efficacy of the robot-enhanced therapy with evidence-based treatments.

Future research on the effects of robot-enhanced therapy should also focus on other types of pathologies (e.g., anxiety, depression) to test the effects of robot-enhanced therapy on reducing these symptoms, as the current research mostly focuses on ASD and dementia in the elderly. Until now, the majority of studies investigated the outcomes of therapy; future studies should also investigate the mechanisms of change and should elaborate cost-effectiveness analysis regarding robot-enhanced therapy.

References

References marked with an asterisk (*) denote articles included in the meta-analysis.

* Admoni

, Bank

, Tan

, Toneva

, & Scassellati

(2011). Robot gaze does not reflexively cue human attention. In Proceedings of the 33rd Annual Conference of the Cognitive Science Society (pp. 1983–1988). Boston, MA: CSS.

Bainbridge

W. A.

, Hart

, Kim

E. S.

, & Scassellati

(2008). The effect of presence on human–robot interaction. In The 17th IEEE International Symposium on Robot and Human Interactive Communication, 2008 (pp. 701–706). Piscataway, NJ: IEEE.

* Bainbridge

W. A.

, Hart

J. W.

, Kim

E. S.

, & Scassellati

(2011). The benefits of interactions with physically present robots over video-displayed agents. International Journal of Social Robotics, 3, 41–52. doi:10.1007/s12369-010-0082-7

Bartneck

, & Forlizzi

(2004). A design-centered framework for social human-robot interaction. In 13th IEEE International Workshop on Robot and Human Interactive Communication, 2004 (pp. 591–594). Piscataway, NJ: IEEE.

* Bekele

E. T.

, Lahiri

, Swanson

A. R.

, Crittendon

J. A.

, Warren

Z. E.

, & Sarkar

(2013). A step towards developing adaptive robot-mediated intervention architecture (ARIA) for children with autism. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 21, 289–299.

Bemelmans

, Gelderblom

G. J.

, Jonker

, & de Witte

(2012). Socially assistive robots in elderly care: A systematic review into effects and effectiveness. JAMDA: The Journal of Post-Acute and Long Term Care Medicine, 13, 114–120. doi:10.1016/j.jamda.2010.10.002

* Beran

T. N.

, Ramirez-Serrano

, Vanderkooi

O. G.

, & Kuhn

(2013). Reducing children's pain and distress towards flu vaccinations: A novel and effective application of humanoid robotics. Vaccine, 31, 2772–2777. doi:10.1016/j.vaccine.2013.03.056

Besio

, Caprino

, & Laudanna

(2008). Profiling robot-mediated play for children with disabilities through ICF-CY: The example of the European project IROMEC. In Miesenberger

, Klaus

, Zagler

, & Karshmer

(), Computers helping people with special needs (pp. 545–552). Berlin: Springer.

10.

Bickmore

, & Cassell

(2001). Relational agents: A model and implementation of building user trust. Human Factors: The Journal of the Human Factors and Ergonomics Society, 396–403.

11.

Borenstein

, Hedges

, Higgins

, & Rothstein

(2005). Comprehensive Meta-Analysis Version 2. Englewood, NJ: Biostat.

12.

Broekens

, Heerink

, & Rosendal

(2009). Assistive social robots in elderly care: A review. Gerontechnology, 8, 94–103. doi:10.4017/gt.2009.08.02.002.00

13.

Burgoon

J. K.

, Bonito

J. A.

, Bengtsson

, Cederberg

, Lundeberg

, & Allspach

(2000). Interactivity in human–computer interaction: A study of credibility, understanding, and influence. Computers in Human Behavior, 16, 553–574. doi:10.1016/S0747-5632(00)00029-7

14.

Costescu

, & David

(in press). Attitudes toward using social robots in psychotherapy. Transylvanian Journal of Psychology.

15.

Dautenhahn

(1999). Robots as social actors: Aurora and the case of autism. In Proceedings Third International Cognitive Technology Conference CT99 (Vol. 359, p. 374). Retrieved from http://www.cyber.rdg.ac.uk/people/kd/WWW/home.html

16.

Dautenhahn

, & Werry

(2004). Towards interactive robots in autism therapy: Background, motivation and challenges. Pragmatics & Cognition, 12, 1–35. doi:10.1075/pc.12.1.03dau

17.

David

, Matu

, & David

(2014). Robot-based psychotherapy: Concepts development, state of the art, and new directions. The case of robot-based cognitive behavior therapy. Journal of Cognitive Psychology, 7, 192–210.

18.

Diehl

J. J.

, Schmitt

L. M.

, Villano

, & Crowell

C. R.

(2012). The clinical use of robots for individuals with autism spectrum disorders: A critical review. Research in Autism Spectrum Disorders, 6, 249–262. doi:10.1016/j.rasd.2011.05.006

19.

Eurostat. (2010). Demography report 2010. Retrieved from http://epp.eurostat.ec.europa.eu/

20.

Feil-Seifer

, & Mataric

M. J.

(2005). Defining socially assistive robotics. In 9th International Conference on Rehabilitation Robotics, 2005. (pp. 465–468). Piscataway, NJ: IEEE.

21.

* Han

, Jo

, Jones

, & Jo

J. H.

(2008). Comparative study on the educational use of home robots for children. Journal of Information Processing Systems, 4, 159–168. doi:10.3745/JIPS.2008.4.4.159

22.

Hancock

P. A.

, Billings

D. R.

, Schaefer

K. E.

, Chen

J. Y.

, De Visser

E. J.

, & Parasuraman

(2011). A meta-analysis of factors affecting trust in human–robot interaction. Human Factors, 53, 517–527. doi:10.1177/0018720811417254

23.

Hoyt

C. L.

, Blascovich

, & Swinth

K. R.

(2003). Social inhibition in immersive virtual environments. Presence: Teleoperators and Virtual Environments, 12, 183–195. doi:10.1162/105474603321640932

24.

Hunter

J. E.

, & Schmidt

F. L.

(1990). Methods of meta-analysis. Newbury Park, CA: Sage.

25.

Jung

, & Lee

K. M.

(2004). Effects of physical embodiment on social presence of social robots. In Proceedings of PRESENCE (pp. 80–87). Valencia, Spain: ISPR.

26.

Kidd

C. D.

, & Breazeal

(2004). Effect of a robot on user perceptions. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2004: Proceedings (Vol. 4, pp. 3559–3564). Piscataway, NJ: IEEE.

27.

Kidd

C. D.

, Taggart

, & Turkle

(2006). A sociable robot to encourage social interaction among the elderly. In Proceedings of the 2006 IEEE International Conference on Robotics and Automation, 2006. (pp. 3972–3976). Piscataway, NJ: IEEE.

28.

* Kim

E. S.

, Berkovits

L. D.

, Bernier

E. P.

, Leyzberg

, Shic

, Paul

, & Scassellati

(2013). Social robots as embedded reinforcers of social behavior in children with autism. Journal of Autism and Developmental Disorders, 43, 1038–1049.

29.

Kozima

, Nakagawa

, & Yasuda

(2005). Interactive robots for communication- care: A case-study in autism therapy. In IEEE International Workshop on Robot and Human Interactive Communication, 2005 (ROMAN 2005; pp. 341–346). Piscataway, NJ: IEEE.

30.

Kwakkel

, Kollen

B. J.

, & Krebs

H. I.

(2008). Effects of robot-assisted therapy on upper limb recovery after stroke: A systematic review. Neurorehabilitation and Neural Repair, 22, 111–121. doi:10.1177/1545968307305457

31.

* Leyzberg

, Spaulding

, Toneva

, & Scassellati

(2012). The physical presence of a robot tutor increases cognitive learning gains. In Proceedings of the 34th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society.

32.

Libin

A. V.

, & Libin

E. V.

(2004). Person–robot interactions from the robopsychologists’ point of view: The robotic psychology and robotherapy approach. Proceedings of the IEEE, 92, 1789–1803. doi:10.1109/JPROC.2004.835366

33.

McGough

J. J.

, & Faraone

S. V.

(2009). Estimating the size of treatment effects: Moving beyond p values. Psychiatry (Edgmont), 6, 21–29.

34.

Michaud

, & Clavet

(2001). Robotoy contest-designing mobile robotic toys for autistic children. In Proceedings of the American Society for Engineering Education (ASEE’01). Washington, DC: ASEE.

35.

Mordoch

, Osterreicher

, Guse

, Roger

, & Thompson

(2013). Use of social commitment robots in the care of elderly people with dementia: A literature review. Maturitas, 74, 14–20. doi:10.1016/j.maturitas.2012.10.015

36.

Mureşan

, Montgomery

G. H.

, & David

(2012). Emotional outcomes and mechanisms of change in online cognitive–behavioral interventions: A quantitative meta-analysis of clinical controlled studies. Journal of Technology in Human Services, 30, 1–13. doi:10.1080/15228835.2011.653290

37.

Opriş

, Pintea

, Garcia-Palacios

, Botella

, Szamoskozi

, & David

(2012). Virtual reality exposure therapy in anxiety disorders: A quantitative meta-analysis. Depression and Anxiety, 29, 85–93. doi:10.1002/da.20910

38.

Park

, & Catrambone

(2007). Social facilitation effects of virtual humans. Human Factors: The Journal of the Human Factors and Ergonomics Society, 49, 1054–1060. doi:10.1518/001872007X249910

39.

* Pop

, Pintea

, Vanderborght

, & David

(2014). Enhancing play skills, engagement and social skills in a play task in ASD children by using robot-based interventions. A pilot study. Interaction Studies, 15, 292–320.

40.

* Pop

, Simut

, Pintea

, Saldien

, Rusu

, Vanderfaeillie

, … Vanderborght

(2013). Social robots vs. computer display: Does the way social stories are delivered make a difference for their effectiveness on ASD children? Journal of Educational Computing Research, 49, 381–401. doi:10.2190/EC.49.3.f

41.

Powers

M. B.

, & Emmelkamp

P. M.

(2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. Journal of Anxiety Disorders, 22, 561–569. doi:10.1016/j.janxdis.2007.04.006

42.

Prescott

T. J.

, Epton

, Evers

, McKee

, Hawley

, Webb

T. L.

, … Dario

(2012). Robot companion for citizens: Roadmapping the potential for future robots in empowering older people. In Bridging Research into Ageing and ICT Development. Prague, Czech Republic: Institute for Informatics and Digital Innovation.

43.

Reger

M. A.

, & Gahm

G. A.

(2009). A meta-analysis of the effects of internet-and computer-based cognitive–behavioral treatments for anxiety. Journal of Clinical Psychology, 65, 53–75. doi:10.1002/jclp.20536

44.

Ricks

D. J.

, & Colton

M. B.

(2010). Trends and considerations in robot-assisted autism therapy. In 2010 IEEE International Conference on Robotics and Automation (ICRA; pp. 4354–4359). Piscataway, NJ: IEEE.

45.

* Riether

, Hegel

, Wrede

, & Horstmann

(2012). Social facilitation with social robots? In Human–robot Interaction (HRI), 2012 7th ACM/IEEE International Conference on (pp. 41–47). Piscataway, NJ: IEEE.

46.

Robins

, Amirabdollahian

, Ji

, & Dautenhahn

(2010). Tactile interaction with a humanoid robot for children with autism: A case study analysis involving user requirements and results of an initial implementation. In IEEE International Symposium on Robot and Human Interactive Communication: Proceedings, 2010 (pp. 704–711). Piscataway, NJ: IEEE. doi:10.1109/ROMAN.2010.5598641

47.

Rosenthal

(1991). Meta-analytic procedures for social research: Vol. 6. Thousand Oaks, CA: Sage. doi:10.4135/9781412984997

48.

Scassellati

, Admoni

, & Mataric

(2012). Robots for use in autism research. Annual Review of Biomedical Engineering, 14, 275–294. doi:10.1146/annurev-bioeng-071811-150036

49.

Simut

, Pop

, Vanderfaeillie

, Lefeber

, & Vanderborght

(2012, July). Trends and future of social robots for ASD therapies: Potential and limits in interaction. Paper presented at the International Conference on Innovative Technologies for Autism Spectrum Disorders: Tools, Trends and Testimonials, Valencia, Spain.

50.

* Stanton

C. M.

, Kahn

P. H.

, Severson

R. L.

, Ruckert

J. H.

, & Gill

B. T.

(2008, March). Robotic animals might aid in the social development of children with autism. In Proceedings of the 3rd ACM/IEEE International Conference on Human–robot Interaction, (pp. 271–278). New York, NY: ACM.

51.

Takayama

, & Pantofaru

(2009). Influences on proxemic behaviors in human–robot interaction. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009: Proceedings (pp. 5495–5502). Piscataway, NJ: IEEE.

52.

Tapus

, Peca

, Aly

, Pop

, Jisa

, Pintea

, … David

D. O.

(2012). Children with autism social engagement in interaction with Nao, an imitative robot A series of single case experiments. Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems, 13, 315–347. doi:10.1075/is.13.3.01tap

53.

Thill

, Pop

C. A.

, Belpaeme

, Ziemke

, & Vanderborght

(2012). Robot-assisted therapy for autism spectrum disorders with (partially) autonomous control: Challenges and outlook. Paladyn, 3, 209–217. doi:10.2478/s13230-013-0107-7

54.

Vanderborght

, Simut

, Saldien

, Pop

, Rusu

A. S.

, Pintea

, Dirk

, & David

D. O.

(2012). Using the social robot probo as a social story telling agent for children with ASD. Interaction Studies: Social Behaviour and Communication in Biological and Artificial Systems, 13, 348–372. doi:10.1075/is.13.3.02van

55.

Wada

, Shibata

, Saito

, & Tanie

(2002). Analysis of factors that bring mental effects to elderly people in robot assisted activity. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 2002. (Vol. 2, pp. 1152–1157). Piscataway, NJ: IEEE.

56.

Wada

, Shibata

, Saito

, & Tanie

(2004). Effects of robot-assisted activity for elderly people and nurses at a day service center. Proceedings of the IEEE, 92, 1780–1788. doi:10.1109/JPROC.2004.835378

57.

Wainer

, Feil-Seifer

D. J.

, Shell

D. A.

, & Mataric

M. J.

(2007). Embodiment and human–robot interaction: A task-based perspective. In The 16th IEEE International Symposium on Robot and Human Interactive Communication: Proceedings, 2007 (pp. 872–877). Piscataway, NJ: IEEE.

58.

* Wood

L. J.

, Dautenhahn

, Rainer

, Robins

, Lehmann

, & Syrdal

D. S.

(2013). Robot-mediated interviews-how effective is a humanoid robot as a tool for interviewing young children? PloS One, 8, e59448. doi:10.1371/journal.pone.0059448

59.

Zanbaka

C. A.

, Ulinski

A. C.

, Goolkasian

, & Hodges

L. F.

(2007). Social responses to virtual humans: Implications for future interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1561–1570). New York, NY: Association for Computing Machinery.