Predictors and outcomes associated with therapeutic alliance in cognitive behaviour therapy for children with autism

Participants

Participants included 48 children (91.7% male) between the ages of 8 and 12 years (M = 9.60, SD = 1.25), who took part in a RCT of an emotion regulation focused CBT intervention (Weiss et al., 2018). All children (1) demonstrated at least average intellectual functioning (IQ ⩾ 79) based on the Wechsler Abbreviated Scale of Intelligence–Second Edition (WASI-II; Wechsler, 2011); (2) exhibited some degree of willingness to participate in therapy; (3) had a documented autism diagnosis from a qualified healthcare professional; and (4) met cut-offs on either the parent-report versions of the Social Communication Questionnaire (SCQ cut-off > 14; Rutter & Bailey, 2003) or the Social Responsiveness Scale–Second Edition (SRS-2 Total T score cut-off > 59, Constantino, 2012). For children whose parent could not provide documentation and did not meet cut-off scores for either screening tool (n = 2), the Autism Diagnostic Observation Schedule–Second Edition (ADOS-2; Lord et al., 2012) Module 3 was administered. Children were not eligible to participate in the study if they had a recent history of aggressive behaviour towards others or self-injurious behaviours that could potentially be a serious safety concern, or if they were currently receiving CBT or another therapy targeting emotion regulation.

For the current sample, child Full-Scale IQ–2 (FSIQ-2) scores ranged from 79 to 140 (M = 105.00, SD = 14.64). Children varied in autism symptom severity both on the SRS-2 total T scores (M = 74.40, SD = 9.33) and SCQ total scores (M = 21.62, SD = 4.45). The majority of parents identified their children as White/Caucasian (78.6%) and reported themselves as being married (91.5%), having post-secondary education (90.7%), and having an annual family income of at least CAD$100,000 before taxes (65.1%; 18.6% preferred not to disclose).

Measures

Procedures

The larger RCT from which this study was derived began in September 2013. This research has received ethics review and approval by the York University’s Ethics Review Board (#e2013-229). Thirty-two children were excluded for not meeting study inclusion criteria, and 11 declined to participate despite being eligible. Sixty-eight children were eligible to participate in the larger RCT. Once eligibility was determined, parents provided written informed consent and children provided either written or verbal informed assent. For the purposes of this study, all children who received the intervention in its entirety were combined to form one treatment group regardless of whether they were initially randomized to treatment immediate or waitlist. Measures were administered within the 2 weeks prior to treatment receipt (pre-treatment) and within 1 week of completing therapy (post-treatment).

Of the 68 eligible families, 13 withdrew from the study after confirming enrolment (nine children dropped out before beginning the intervention and four dropped out part way through the programme). Data were not available for an additional six children because parents did not consent to having session videos used for research purposes (n = 4) or technical issues caused session videos to be unusable (n = 2). Finally, data for one participant was excluded due to a change in therapist midway through treatment, leaving a final sample of n = 48.

Analysis plan

All analyses were conducted using IBM SPSS Statistics version 24. Spearman-rho correlations were calculated to test the hypothesis that pre-treatment characteristics would be associated with subsequent measures of early and late bond and task-collaboration at the bivariate level. Multiple regressions were then calculated to determine if pre-treatment child characteristics that were significant at the bivariate level predicted therapeutic alliance. If predictor variables were largely correlated with each other (i.e. r > 0.50), only the variable with the strongest association with bond and task-collaboration was included in the model to maintain power to detect moderate to large effects.

A series of hierarchical regression analyses were conducted to test the hypothesis that early and late alliance would predict treatment outcomes. The first step controlled for time in treatment, child IQ and baseline levels of an outcome measure (McLeod & Weisz, 2005). The second block consisted of either early or late bond and task-collaboration scores. Separate analyses were run for each outcome variable.

We recognize that conducting these analyses raises questions about issues related to multiplicity, including inflated Type I error rates. Scientific and clinical risk associated with Type I error within the context of this study include identifying and placing undue emphasis on pre-treatment factors that are not true predictors of the quality of alliance, and falsely predicting the extent to which therapeutic alliance predicts treatment change. While failing to correct for multiple comparisons may increase the likelihood of identifying false positive findings, the small sample size, limited research in the area and an interest in balancing the examination of significance in terms of the probability of identifying a true effect serve as rationale for no adjustments being made (Feise, 2002). All variables included in the analyses were selected on the basis of clinical relevance and previously derived empirical evidence. Power analyses using G*Power 3.1 (Faul, Erdfelder, Buchner, & Lang, 2009) indicated that moderate to large effects could be detected using multiple linear regression analyses with a sample of 48 children, depending on the number of predictors included.

Pre-treatment child characteristics and therapeutic alliance

As shown in Table 1, several pre-treatment child characteristics were associated with TPOCS-A ratings, both early and late in treatment. Child age and autism symptom severity (SCQ and SRS) were not related to any early or late alliance measure. Child IQ was associated with late task-collaboration, but no other indicator of alliance. Early and late bond were both positively associated with child report of emotional inhibition (CEMS-Inhibition). Early task-collaboration was positively related to child-reported coping (CEMS-Coping) and negatively associated with child report of emotional dysregulation (CEMS-Dysregulation). Late task-collaboration was positively associated with child report of inhibition (CEMS-Inhibition) and negatively related to child report of emotional dysregulation (CEMS-Dysregulation), parent report of child emotional lability (ERC Lability/Negativity), child externalizing symptoms (BASC-2 Externalizing), child behavioural symptoms (BASC-2 BSI) and clinician rated overall severity of psychopathology (ADIS-P Overall Severity).

OPEN IN VIEWER

Table 1.

Spearman-rho correlations between pre-treatment child characteristics and TPOCS-A ratings (N = 48).

Characteristic	Early TPOCS-A		Late TPOCS-A
	Bond	Task-collaboration	Bond	Task-collaboration
Age	−0.15	0.16	−0.04	0.04
FSIQ-2 a	0.20	0.07	0.29 †	0.34*
SRS-2	0.03	0.03	0.05	−0.25 †
SCQ	0.22	0.14	0.13	0.19
Readiness to participate a	0.17	0.25 †	0.02	−0.05
CEMS b
Inhibition	0.30*	0.27 †	0.36*	0.30*
Dysregulation	−0.24	−0.38*	−0.27 †	−0.32*
Coping	0.09	0.36*	0.24	0.28 †
ERC
Lability/negativity	−0.16	−0.27 †	−0.07	−0.39**
Emotion regulation	0.11	0.17	−0.02	0.14
BASC-2
Externalizing problems	−0.12	−0.22	−0.14	−0.37**
Internalizing problems	0.04	−0.09	0.05	−0.26 †
BSI	−0.06	−0.11	−0.02	−0.36*
ADIS-P overall severity	−0.17	−0.14	−0.04	−0.30*

TPOCS-A = Therapist Process Observational Coding System–Alliance Scale; FSIQ-2 = Full-Scale IQ–2 Subscales; SRS-2 = Social Responsiveness Scale–Second Edition, Total T Score; SCQ = Social Communication Questionnaire; CEMS = Children’s Emotion Management Scale; ERC = Emotion Regulation Checklist; BASC-2 = Behavioural Assessment System for Children–Second Edition; BSI = Behavioural Symptoms Index; ADIS-P = Anxiety Disorder Interview Schedule–Parent Version.

a

n = 47.

b

n = 45.

†

p < 0.10; *p < 0.05; **p < 0.01.

Multiple regression analyses were not calculated for early or late bond, as only a single bivariate correlate emerged as significant. Results of multiple regressions revealed that child pre-treatment characteristics did not account for a significant portion of the variance in early task-collaboration and no unique predictors emerged. Child pre-treatment characteristics accounted for a moderate portion of the variance in late task-collaboration, R² = 0.40, F(7, 34) = 3.20, p = 0.01; however, there were no significant unique predictors.

Therapeutic alliance and primary treatment outcome

A series of hierarchical linear regressions were conducted to determine if alliance variables predicted treatment outcomes. Early bond and task-collaboration were not predictive of treatment change for any outcome variables (Supplemental Table 1). In contrast, significant patterns emerged when measures of late alliance were included as predictors, as shown in Table 2. Specifically, late ratings of alliance accounted for a significant portion of variance in treatment change in parent reports of child emotional lability (ERC Lability/Negativity), ΔR² = 0.07, p = 0.02, with late task-collaboration emerging as a unique predictor (β = –0.36, p = 0.02). Late task-collaboration emerged as a unique predictor of improvements in child report of emotional dysregulation (CEMS-Dysregulation), β = –0.45, p = 0.04, even though late alliance as a whole was not a significant predictor (ΔR² = 0.12, p = 0.07). Late alliance ratings were not predictive of change in CEMS-Inhibition or Coping and ERC Emotion Regulation (all ps > 0.05).

OPEN IN VIEWER

Table 2.

Linear regression results for late therapeutic alliance predicting treatment outcome.

Variable	B	SE B	β
CEMS-inhibition a
Baseline score	0.78	0.17	0.66**
Days in treatment	<0.01	<0.01	0.06
FSIQ-2	<–0.01	0.01	−0.07
TPOCS-A bond	0.03	0.19	0.02
TPOCS-A task	−0.15	0.13	−0.20
CEMS-dysregulation a
Baseline score	0.15	0.12	0.19
Days in treatment	<–0.01	<0.01	−0.05
FSIQ-2	0.01	<0.01	0.27 †
TPOCS-A bond	0.05	0.13	−0.08
TPOCS-A task	−0.19	0.09	−0.45*
CEMS-coping a
Baseline score	0.42	0.12	0.48**
Days in treatment	<0.01	<0.01	0.17
FSIQ-2	<–0.01	<0.01	<0.01
TPOCS-A bond	−0.07	0.13	−0.09
TPOCS-A task	0.17	0.09	0.34 †
ERC lability/negativity a
Baseline score	0.72	0.11	0.68**
Days in treatment	<0.01	<0.01	−0.01
FSIQ-2	0.01	<0.01	0.16
TPOCS-A bond	0.10	0.12	0.11
TPOCS-A task	−0.22	0.09	−0.36*
ERC emotion regulation a
Baseline score	0.57	0.08	0.75**
Days in treatment	<0.01	<0.01	0.03
FSIQ-2	<0.01	<0.01	0.02
TPOCS-A bond	−0.05	0.10	−0.07
TPOCS-A task	0.10	0.07	0.20

CEM = Children’s Emotion Management Scale; FSIQ-2 = Full-Scale IQ–2 Subscales; TPOCS-A = Therapist Process Observational Coding System–Alliance Scale; ERC = Emotion Regulation Checklist.

a

Outcome variable.

†

p < 0.10; *p < 0.05; **p < 0.01.

Pre-treatment child characteristics and therapeutic alliance

Consistent with the initial hypothesis, emotion regulation was significantly related to the quality of therapeutic alliance at different points of treatment. In particular, child self-reported tendencies to inhibit emotional responses were associated with a stronger therapeutic bond and better task-collaboration both early and late in treatment. In the context of therapy with children with autism, emotional inhibition may benefit the therapeutic process, in contrast to the negative effects that suppressing one’s emotions may have in day-to-day life (Aldao, Nolen-Hoeksema, & Schweizer, 2010). In comparison to common overt emotional dysregulation that require significant management of behaviours from parents and individuals working with children with autism (Mazefsky, 2011), children who are better able to inhibit highly emotional displays may allow for therapists to more easily establish rapport. It is also likely that those children who are more emotionally inhibited are less likely to demonstrate negative behaviours and affect that would impede on the quality of the bond as operationalized on the TPOCS-A bond subscale. Several other pre-treatment indicators of emotion regulation were associated with better task-collaboration at early and late sessions. This included child report of greater coping skills and less emotional dysregulation and parent report of greater child emotion regulation. These findings suggest that both parent and child reports of emotional regulation ability and the presentation of emotional dysregulation are pertinent to consider when working to establish therapeutic rapport with children in treatment, especially since emotion regulation is a common challenge for children with autism (Mazefsky & White, 2014).

Parent reports of child externalizing problems, and clinician judgements of overall psychopathology, were associated with poorer task-collaboration in later sessions, but not to early therapeutic alliance or to late-session ratings of therapeutic bond. In the only other study of pre-treatment characteristics and alliance for children with autism, demographic factors and pre-treatment clinical levels of internalizing and externalizing symptoms were not related to post-treatment retrospective ratings of alliance, as reported by child, parent or therapist (Kerns et al., 2018). Beyond autism, associations of child factors and alliance are known to occur as a result of differences in source of reporting (e.g. child report vs therapist report vs independent observer) and the timing of reporting (e.g. at early or late sessions, or retrospectively following treatment completion; McLeod, 2011; McLeod et al., 2017), and thus may contribute to why our findings differed.

Contrary to expectations, child readiness to participate in therapy was not significantly related to the quality of therapeutic alliance, though there was a trending association with early task-collaboration. Although children varied in their degree of readiness to participate, treatment was sought by the child’s primary caregiver, and in this context, children may fail to recognize personal emotional and behavioural challenges (Shirk & Russell, 1998, as cited in Shirk & Karver, 2003). From a process and developmental perspective, this lack of insight may serve as a unique challenge when trying to establish alliance with younger clients (Shirk & Karver, 2003). An additional unexpected finding was that in this sample of children with at least average estimated intellectual functioning, IQ was related to late task-collaboration and had a trending association with bond. Given the heterogeneity in intellectual functioning for children with autism, future research should continue to examine the role of cognitive functioning and variable profiles within the context of CBT, as this treatment orientation is so focused on active task-collaboration.

Although not a focus of this study, exploratory analyses indicated no significant association between autism symptom severity and observer-rated therapeutic bond or task-collaboration. To our knowledge, only one study to date has compared therapeutic alliance in children with and without autism. Klebanoff (2015) found that in CBT targeting anxiety in children, therapist ratings of alliance were significantly lower for children with autism aged 10–11 years, compared with those without autism. However, there was no significant difference between ratings for children below 10 years of age. This age by group finding certainly warrants further study of potentially autism-specific dynamics, and the TPOCS may be a useful tool given its application in both clients with and without autism.

Child pre-treatment characteristics did not predict either aspect of the therapeutic alliance early in treatment, or the therapeutic bond later in treatment, when entered into regressions. Furthermore, only a moderate portion of variance of late task-collaboration was accounted for by the pre-treatment characteristic regression model, with no significant unique predictors. This lack of significant regression results may be a result of the low power to account for small to moderate effects in models where the predictor variables, though not at the level of multicollinearity, continue to show correlations with each other.

Therapeutic alliance and primary treatment outcome

The initial hypothesis around therapeutic alliance and treatment outcome was partially supported, in that only a particular aspect of late alliance was associated with child change in therapy. More specifically, late task-collaboration was predictive of improvements in child emotion dysregulation. Although this study was the first to examine alliance as it relates to emotion regulation as a treatment outcome, these findings align with those from studies involving children without autism, where alliance-outcome effect sizes tend to be greater when considering changes in externalized symptom presentation, compared with internalized presentations (McLeod, 2011). Whereas emotion regulation involves internalized processes (Mazefsky et al., 2013), the current results reflect changes in emotion dysregulation; the child’s negative externalized behavioural response. The measures of emotion regulation that were not significantly predicted by alliance (e.g. ERC Emotion Regulation subscale) focused more on the internalized aspects of emotion regulation. Considerable literature also favours late measures of alliance versus early ones in being related to treatment outcome in therapy for children (Karver, Handelsman, Fields, & Bickman, 2006; McLeod, 2011; Shirk & Karver, 2011). Although ratings of later alliance may be biased by symptom improvement when the reporter has been actively involved or invested in the treatment process (McLeod & Weisz, 2005; Shirk & Karver, 2011), this study used independent observational coding, which may have helped to reduce the perception of treatment progress as a confound.

Limitations

There are several limitations with this study that should be considered. First, results should be interpreted with the consideration that no alpha-level adjustments were made to correct for multiple comparisons. The sample only included participants who completed treatment, and for whom post-treatment data were available. Premature termination is recognized as an important methodological issue for intervention research because it limits generalizability of results, introduces sampling bias and, depending on sample size available, reduces statistical power (Nock & Ferriter, 2005). In addition, the sample included was predominately male, limiting how results generalize to females with autism. Although previous research has not demonstrated a gender-effect in the association between alliance-outcome (Kerns et al., 2018; McLeod, 2011), future studies should aim to include a larger proportion of females to determine if this finding holds within the autism population. Coders could also not be completely blind to session number. Although session order was randomized to reduce coding bias, the session content is manualized for each session and the client or therapist may also have mentioned the session number in video recordings, making it difficult for coders to be blind to the session. Notably, this study only assessed therapeutic alliance using independent observer ratings. Given that alliance-outcome associations tend to vary as a result of informant source, future research should consider assessing the convergence and predictive validity of multi-informant ratings in therapy for children with autism. Finally, this study was part of a larger efficacy trial that in part looked to evaluate therapist fidelity to a manualized protocol, potentially affecting the generalizability of findings to the real-world context.