Abstract
Childhood is a developmental period that involves continual maturation of behaviors. Challenging behavior in young children may be a normal and even a common part of a repertoire of behaviors that will dissipate as a child develops. However, challenging behavior exhibited by older children or children with disabilities can be problematic for families and schools and typically persists across middle childhood (Emerson, Einfeld, & Stancliffe, 2011) or to early adulthood (Einfeld et al., 2006). Challenging behaviors can interfere with the functioning or daily occupations of the people witnessing or experiencing the behavior, particularly the child’s or young person’s parents (Baker et al., 2003; Cheng, Palta, Poehlmann-Tynan, & Witt, 2015).
Within the field of mental health and disability studies, substantial research has been dedicated to atypical or challenging behavior, driven in part by the repercussions of the duress experienced by the person displaying the behavior and the care provider. Parenting programs can be effective at shaping more appropriate behaviors and arresting the development and severity of challenging behaviors exhibited by children with disabilities (Sanders, Markie-Dadds, Tully, & Bor, 2000) as well as those without disabilities (Niccols, 2009). Effective early identification and intervention for children who display increasing problem behavior or children who are at risk for emotional or behavioral difficulties have been identified as crucial (Emerson & Einfeld, 2010; Emerson et al., 2011). Hence, tools that can identify families who are in need of immediate assistance from professionals, or who might be prioritized for service provision, are of the utmost importance.
The Child’s Challenging Behaviour Scale, Version 2 (CCBS–2; Bourke-Taylor, Pallant, & Law, 2014) is a short and psychometrically sound scale that was designed to measure a mother’s rating of the prevalence of challenging behaviors displayed by her child with a disability. The tool was developed to identify core behaviors that mothers recognized as compromising caregiving capacity and self-reported mental health (Bourke-Taylor, Howie, & Law, 2010; Bourke-Taylor, Law, Howie, & Pallant, 2010). The CCBS–2 is freely available and has been translated into three languages. The 9-item scale provides statements about behaviors and includes a 4-item response set (ranging from strongly agree to strongly disagree). Two items are reverse scored before summation of item responses. Higher scores on the scale indicate increased challenging behavior.
The items measured by the scale include behaviors such as aggression, violence, oppositional behavior, uncooperative behavior, resistance to routine, reliance on routine that includes refusal to deviate from routine, rejection of other care providers, and persistent unhappiness or discontented state. The CCBS–2 is an instrument that permits a parent to rate behavior that he or she finds upsetting and difficult to manage, or behavior that is compromising some aspect of the child’s care and parental well-being. The CCBS–2 has clinical and research utility because it can be used to identify families in which a child’s behavior and parental experience of the behavior are problematic for the family; therefore, the behavior needs to be identified to enable professional services to be initiated. Preliminary investigations revealed strong psychometric properties (Bourke-Taylor et al., 2014).
The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) provided a framework to organize measurement properties pertaining to the psychometric evaluation of instruments used in clinical practice and research (Mokkink et al., 2010). The COSMIN criteria provided a framework against which the psychometric properties of the CCBS (initial form) and CCBS–2 (current form) were evaluated. Validity of the initial 11-item CCBS was investigated in several ways. Content and face validity was addressed during development through rigorous application of a mixed-methods instrument design model that included a qualitative study (Bourke-Taylor, Howie, & Law, 2010) to generate items and a subsequent quantitative study to evaluate the properties of the CCBS (Bourke-Taylor, Law, et al., 2010). Previous publications have described how redundant items (only two items) were removed after Rasch analysis and reliability testing (internal consistency), and the scale was retitled the Child’s Challenging Behaviour Scale, Version 2 (CCBS–2) as previously described (Bourke-Taylor et al., 2014).
Construct validity was measured with known groups testing in the initial sample of 152 mothers of children with a disability (Bourke-Taylor, Law, et al., 2010). CCBS scores of women with and without diagnosed mental health conditions were compared, and CCBS–2 scores were found to be significantly higher in the group of women with mental health conditions. Moreover, CCBS–2 scores were associated with family cohesion, maternal empowerment and capacity to manage challenges related to their child’s disability, and maternal self-rated mental health. In construct validity studies, researchers also measured the related child construct of challenging behavior through correlation with psychosocial problems that the child exhibited and how uncooperative the child was when taking medication, as well as using known groups validity to compare groups of children with and without autism spectrum disorder (ASD) and with and without childhood psychiatric diagnoses. Collectively, psychometric evaluation to date supports the reliability and validity of the CCBS–2 as a sound scale that measures a complex construct in a simple way.
Although evaluation of the CCBS–2 supports application for research and potential clinical use, ongoing evaluation of the scale’s psychometric properties is required. Further psychometric evaluation of a scale requires application to different populations (DeVillis, 2012; Streiner, Norman, & Cairney, 2014). As described in COSMIN, construct validity was further examined through hypothesis testing. Hypothesis testing, more traditionally referred to as known groups validity, is a process in which differences among groups are hypothesized to be present (or not). Differences in known groups were hypothesized within the typically developing (TD) sample on the basis of gender, age, and school entry as well as between the TD sample and an age-matched sample of children with disabilities from previously collected data (Bourke-Taylor et al., 2014). To further test application with mothers of TD young children, we selected children ages 3–9 yr. As defined with COSMIN, the questions addressed in this article were described within specific reliability and construct validity concepts. We used Research Question 1 to evaluate reliability (internal consistency), and we applied hypothesis testing with Hypotheses 1–3 to examine construct validity using known groups validity:
Research Question 1. How internally consistent (reliable) is the CCBS–2 when tested with a population of mothers and their TD children?
Hypothesis 1. Significant differences will be detected among groups of children scored on the CCBS–2 on the basis of age groups, as well as before and after school entry, but not on the basis of gender.
Hypothesis 2. Significant differences will be detected among groups of mothers of TD children who reported differences in mental health (depression, anxiety, stress).
Hypothesis 3. Significant differences will be detected between the CCBS–2 scores of age-matched children with and without disabilities.
Method
A cross-sectional mail survey design was used to recruit mothers of young children throughout Australia. The study was approved by the Monash University Human Ethics Committee during data collection and by the Australian Catholic University Human Ethics Committee during analysis and write up.
Participants and Recruitment
Mothers self-selected participation in response to a notice about “behaviours of young Australian children” publicized in websites and child-related magazines; snowball sampling was used. Inclusion criteria required that the mother was the primary care provider of a TD child age 3 to age 9 yr, 11 mo; a resident of Australia; and able to complete the survey in English. Mothers contacted the first author and were sent a survey after verbal consent was received. All mothers provided written informed consent with the returned survey, although names were not recorded after the questionnaires were received and data were entered. The survey included the Depression Anxiety Stress Scales (DASS; Lovibond & Lovibond, 1995), the CCBS–2, and the Parents’ Evaluation of Developmental Status (PEDS; Glascoe, 2003) to identify children with potential developmental differences in the sample.
Instruments
Specific demographic questions were compiled for the purpose of this research project (see Table 1 for demographic details). In this research, we measured maternal subjective mental health using the DASS. The PEDS was used as a developmental screening tool to rule out young children with undiagnosed developmental delay or an unrecognized diagnosis.
Characteristics of the Mother–Child TD Australian Sample (N = 337)
Note. Percentages may not total 100 because of rounding. CCBS–2 = Child’s Challenging Behaviour Scale, Version 2; DASS = Depression Anxiety Stress Scales; M = mean; PEDS = Parents’ Evaluation of Developmental Status; SD = standard deviation; TD = typically developing.
Child’s Challenging Behaviour Scale, Version 2.
In the previous evaluation of the CCBS–2, we used factor analysis to confirm unidimensionality and to investigate internal consistency (Cronbach’s α coefficient = .84) in the initial sample of mothers of children with disabilities (Bourke-Taylor, Howie, & Law, 2010; Bourke-Taylor, Law, et al., 2010; Bourke-Taylor et al., 2014). Rasch analysis determined that a 9-item scale with 4-item responses created the soundest instrument (Bourke-Taylor et al., 2014).
Depression Anxiety Stress Scales.
The DASS measures symptoms of stress, anxiety, and depression, and severity classifications may be calculated according to gender (ranging from normal to extremely severe; Lovibond & Lovibond, 1995). Responders rate 21 statements using a 4-point Likert scale ranging from 0 (did not apply) to 3 (applies very much or most of the time). The DASS is a psychometrically sound instrument, with established internal consistency (reliability) with a population of Australian women (Crawford, Cayley, Lovibond, Wilson, & Hartley, 2011; Lovibond & Lovibond, 1995; Cronbach’s α: depression = .91, anxiety = .84, stress = .90). Construct validity has been supported in large studies (Crawford & Henry, 2003; Lovibond & Lovibond, 1995). Researchers have determined strong sensitivity and specificity for diagnostic groups (differences among people with conditions such as depression, obsessive–compulsive disorder, and anxiety disorders) as well as convergent and discriminant validity relating the scale to other measures of psychological distress (Antony, Bieling, Cox, Enns, & Swinson, 1998). In other studies, researchers have confirmed the factor structure and internal consistency of the scale across gender (Gomez, 2013) and people of different ages (Szabó, 2010).
Parents’ Evaluation of Developmental Status.
The PEDS (Glascoe, 2003) is a 10-item tool that identifies children at risk for developmental issues using parent rating of the child’s development. It is a widely used tool with excellent sensitivity and specificity for parental identification of developmental concerns in children ages 8 yr and younger (Glascoe, 1999, 2003; Gustawan & Machfudz, 2010). Response options for questions include no, yes, and a little, with the option of an open-ended response. Items address fine and gross motor abilities, receptive and expressive language, socialization, self-care, learning, and behavior. Parental scoring has excellent predictive validity and can be used to identify the child as having high, moderate, low, or no risk for developmental problems. In a systematic review of the prevalence of parental concerns as measured by the PEDS, 37 studies were identified (N = 210,242 parental ratings), and children with biological or psychosocial adversity were identified by parents: About 14% of parents had concerns that their child was at high developmental risk, and 20% of parents had concerns that their child was at moderate developmental risk (Woolfenden et al., 2014). Australian parents reported concerns at similar levels for children concurrently identified as having an adverse profile, suggesting higher risk for developmental issues (Coghlan, Kiing, & Wake, 2003).
Data Management and Analysis
IBM SPSS Statistics (Version 22; IBM Corp., Armonk, NY) was used for all data entry and management. Descriptive statistics were generated for all scale items and for scale scores. All tools were scored according to published directions, including the severity classifications for the DASS. We calculated internal consistency (reliability) using the Cronbach’s α coefficient. A Bonferroni adjustment was not applied during any data analysis because we aimed to detect any small differences and to minimize risk for a Type II error. Hypotheses related to construct validity were analyzed as follows:
Hypothesis 1.
To investigate differences in CCBS–2 scores between groups of children, we used analysis of variance to investigate age groups, and we used independent-groups t tests to investigate gender as well as before and after school entry. It was expected that CCBS–2 total scores would significantly differ across the age range (3-, 4-, 5-, 6-, 7-, and 8- to 9-yr-olds), with older age groups recording lower scores. It was expected that school-age children (6- to 9-yr-olds) would have significantly lower scores. No gender differences were predicted.
Hypothesis 2.
To investigate differences in CCBS–2 scores between women classified according to severity of subjective mental health concerns (depression, anxiety, and stress symptoms), we used a Kruskal–Wallis test because data were not normally distributed. Although five categories of severity classifications were calculated with the DASS, equivalency was sought between groups, and three groups were configured: normal, mild to moderate, and severe to extremely severe. Post hoc tests were used to determine which differences between classification groups were significant. It was expected that CCBS–2 total scores would differ across the classifications of severity, with women experiencing more severe stress, anxiety, or depression symptoms recording higher scores.
Hypothesis 3.
To investigate differences in CCBS–2 scores between TD children and age-matched children with disabilities, we used independent-samples t tests. CCBS–2 scores were compared between the TD sample collected in this study and a previously reported sample of children with disabilities (Bourke-Taylor, Howie, & Law, 2010; Bourke-Taylor, Law, et al., 2010). Only school-age children ages 6–9 yr were included in this comparison sample. The sample included 76 mothers of children with disabilities (mean [M] age = 41 yr, standard deviation [SD] = 4.7 yr) who were mostly partnered (n = 66; 86.84%) and who described their children ages 6–9 yr (n = 76) as having the following primary diagnoses: autism (n = 37; 48.68%), Asperger syndrome (n = 9; 11.84%), cerebral palsy (n = 6; 7.89%), Down syndrome (n = 6; 7.89%), intellectual disability (n = 6; 7.89%), developmental delay (n = 3; 3.95%), and other childhood disability (n = 9; 11.84%). Scores from the previous study, which used the original version of the CCBS, were adjusted to be consistent with the revised scoring protocol used for the CCBS–2. The middle category (neither agree nor disagree) was recoded as 2.5, with scores for each item ranging from 1 to 4.
Results
Data were collected from every Australian state. Initially, 361 mothers returned the survey; however, it was necessary to remove 24 surveys because mothers recorded that their child had a disability. The final sample was 337 mother–child dyads (see Table 1). Mothers were largely university educated (72%) and from two-parent homes (93%); 61% of the mothers worked for pay. The average age of children was 5 yr, 2 mo (SD = 21 mo). The frequencies of responses to the 9 items of the CCBS–2 are described in Table 2. Scores on the CCBS–2 ranged from 9 to 29 (M = 17.97, SD = 3.76). The distribution of scores approximated the normal curve with no floor or ceiling effects.
Items and Frequencies of Responses to the Nine-Item CCBS–2
Note. N = 337. Percentages may not total 100 because of rounding. CCBS–2 = Child’s Challenging Behaviour Scale, Version 2.
Values were reverse scored before the final score was summed.
Mothers in the current sample had few concerns about their child’s development as measured by the eight items of the PEDS (see Table 1). Very few mothers were concerned about their child’s hand use: 1 mother (0.3%) had a concern, and 3% of mothers were a little concerned. Similarly, few mothers were concerned about how their child was learning to do things for himself or herself: 2% of mothers were concerned, and 8% of mothers were a little concerned. The area of greatest concern to mothers was their child’s behavior because 30% were a little concerned, and 7% were concerned in this area. This incidental finding was investigated further to compare CCBS–2 scores of mothers identifying behavior as a concern on the PEDS and those mothers who did not.
A Mann–Whitney U test was conducted to compare mothers’ scores on the CCBS–2 for those mothers who scored their child’s behavior on the PEDS as “no concerns” (n = 207) versus “yes or some concerns” (n = 123). There were statistically significant differences between these two groups (“some concerns” Mdn = 27 vs. “no concerns” Mdn = 20; z = −7.26; p ≤ .001; r = .4; medium to large effect size). These results showed that mothers reporting some concern about their child’s behavior also scored higher on the CCBS–2. A Mann–Whitney U test was conducted to compare the DASS scores of the two groups of mothers on the basis of responses to concerns about their child’s behavior. The only significant result was for DASS Stress subscale scores (“some concerns” Mdn = 14 vs. “no concerns” Mdn = 10; z = −2.348; p = .019; r = .13; small effect size).
Research Question 1
The internal consistency of the CCBS–2 with this cohort was adequate (Cronbach’s α = .77).
Hypothesis 1
As predicted, CCBS–2 scores did vary across age groups (p = .008; see Table 3), with younger children (3 yr) recording higher scores than the older age groups (8–9 yr; see Figure 1). An independent-samples t test revealed no statistically significant gender differences in CCBS–2 scores (see Table 3). As a group, school-age children (6–9 yr) recorded significantly lower CCBS–2 scores than children who were not school age (p = .018; see Table 3).
Comparison of CCBS–2 Scores of 6- to 9-yr-Old Children Across Gender, Age, Age Group, and School-Age Sample
Note. CCBS–2 = Child’s Challenging Behaviour Scale, Version 2; df = degrees of freedom; M = mean; SD = standard deviation; TD = typically developing.
We calculated effect size using η2 = .01 (small effect), η2 = .06 (moderate effect), η2 = .14 (large effect).
(A) Line graph of Child’s Challenging Behaviour Scale, Version 2 (CCBS–2) scores across age groups. (B) Box plot of CCBS–2 scores for typically developing children compared with children with disabilities.
Hypothesis 2
Most women reported their depression, anxiety, and stress symptoms within “normal limits,” with the following proportions of women reporting mild or more severe symptoms for depression (20%), anxiety (15%), and stress (34%; see Table 1). Kruskal–Wallis tests revealed significant differences (p < .001) in CCBS–2 scores among women in the three severity groups for all DASS subscales. CCBS–2 scores were higher in the group of women with higher depression scores (Group 1 [Gp1] normal, n = 265, median [Mdn] = 18; Group 2 [Gp2] mild to moderate, n = 55, Mdn = 19; Group 3 [Gp3] severe to extremely severe, n = 11, Mdn = 20), χ 2(2, N = 331) = 11.53, p < .001. CCBS–2 scores were higher in the group of women with higher anxiety scores (Gp1 normal, n = 278, Mdn = 18; Gp2 mild to moderate, n = 36, Mdn = 18; Gp3 severe to extremely severe, n = 15, Mdn = 22), χ 2(2, N = 329) = 9.67, p < .001. CCBS–2 scores were higher in the group of women with higher stress scores (Gp1 normal, n = 218, Mdn = 18; Gp2 mild to moderate, n = 82, Mdn = 19; Gp3 severe to extremely severe, n = 29, Mdn = 20), χ 2(2, N = 329) = 11.69, p < .001. Post hoc tests revealed small to moderate effect sizes for these differences(rs = .16–.38).
Hypothesis 3
Independent-samples t tests were used to compare CCBS–2 scores for school-age children from this sample of TD children with scores obtained in a previous study that focused on children with disabilities. As expected, the sample of children with disabilities had a significantly higher (p < .001) mean score (M = 23.45) than the TD children (M = 17.18; see Table 3). The box plot shown in Figure 2 demonstrates a clear difference in the distribution of CCBS–2 scores for the samples of TD children and the children with disabilities. For the TD children, 50% of the sample scored between 15 and 20, compared with the sample of children with disabilities, 50% of whom were scored between 20 and 27 by their mothers (see Figure 2).
When responses of mothers of TD children and children with disabilities were compared, differences were apparent. Four items that were identified and observed by mothers of children with disabilities were not identified as problematic by mothers of TD children. The items (4, 6, 8, 9) related to whether the mother strongly agreed that the following issues were challenging: being left with another adult when the mother went out (TD children = 2%; children with disabilities = 9%), whether mothers managed most challenging behaviors effectively at home alone (TD children = 1%; children with disabilities = 14%), following the family routine (TD children = 0%; children with disabilities = 10%), and coping with disruptions to the family routine (TD children = 2%; children with disabilities = 24%).
Discussion
In this study, we evaluated internal consistency and tested hypotheses that supported the construct validity of the CCBS–2 as a psychometrically sound scale for use with Australian mothers and their TD children ages 3–9 yr. The CCBS–2 showed acceptable internal consistency with this sample. The results supported all the hypotheses that were tested. CCBS–2 scores were significantly different among children grouped according to age, school entry, and whether they had a disability. CCBS–2 scores did not differ on the basis of gender. As expected, higher scores were recorded by the mothers of the youngest children and children with disabilities. Children identified by their mother as having concerning behavior on the PEDS were also identified with the CCBS–2. Moreover, this study confirmed that women with differences in severity of symptoms for depression, stress, or anxiety also recorded significantly different CCBS–2 scores. Mothers with poorer subjective mental health reported significantly more concerns with challenging behaviors exhibited by their children.
In previous studies, researchers have estimated that the incidence of maternal mental health issues associated with child-related challenging behaviors ranged from 8% to 13% (Baker, Blacher, Crnic, & Edelbrock, 2002; Bayer et al., 2012; Sawyer et al., 2001). Difficult and challenging behaviors can be persistent over time without intervention (Baker et al., 2003) and can be associated with later antisocial behavior resulting in social exclusion, educational underachievement, and a cascade of adjustment issues associated with poor life satisfaction. Moreover, with one in every five adolescents at risk for experiencing a mental health issue in any given year (Patel, Flisher, Hetrick, & McGorry, 2007), it seems pertinent to target early detection and treatment much earlier in life. The CCBS–2 is easy to complete with minimal burden to care providers, and it can be used by clinicians to screen families.
Our study demonstrated a reduction in challenging behavior across the school transition period. Past research has suggested that ongoing challenging behaviors (mainly externalizing) into the school years are influenced by the quality of teacher–child relationships, the presence of challenging behaviors before school entry, and the family’s socioeconomic status (Silver, Measelle, Armstrong, & Essex, 2005).
Implications for Occupational Therapy Practice
The findings of this study have the following implications for occupational therapy practice:
The CCBS–2 is a unique tool that was specifically designed to measure the mother’s perspective, thereby identifying families that may benefit from programs that not only reduce the child’s challenging behaviors but also address maternal perceptions and mental health.
Strong evidence indicates that a disrupted or strained mother–child relationship or maternal mental health issue can have a substantial influence on the child’s later development, social–emotional well-being, and mental health (Baker et al., 2002, 2003; Bayer et al., 2012; Hemphill, Heerde, Herrenkohl, & Farrington, 2015; Patel et al., 2007). The findings from this study corroborate past research that a mother’s report of poor mental health is associated with antisocial, challenging behavior in her child (Bolton et al., 2003; Kim-Cohen, Moffitt, Taylor, Pawlby, & Caspi, 2005; Totsika, Hastings, Emerson, Lancaster, & Berridge, 2011).
The CCBS–2 may be used clinically by occupational therapists and other clinicians as a goal-setting tool. A simple way to use the tool is to ask a parent or caregiver to complete the tool and to identify behaviors that interventions may target. For example, if aggression and violence are evident, a targeted intervention for behavior management might be called for. If the child refuses other care providers, goals may be set around extending appropriate care providers. If the child is unhappy and discontented, reworking the house organization or the family routine for greater independence in safe play might be called for. If the child’s reliance on and capacity to manage routine changes are a problem, then strategies may be put in place to assist the child’s anticipation or understanding of routine issues.
The scale is self-rated and person centered. Although to date, development of the scale has been conducted with mothers, in clinical practice, there may be scope for use with any primary care provider of a child exhibiting challenging behaviors.
Limitations
This study was conducted with a relatively small number of self-selecting Australian women; therefore, much larger, broader, and targeted samples are required to allow the development of norms for clinical use. Further application to other countries (such as the United States) will establish generalizability to other cultures. The CCBS–2 has been translated to Farsi, Spanish, and Serbian.
Conclusion and Future Direction for Research
The CCBS–2 demonstrated adequate internal consistency and construct validity in a sample of mothers and their TD children (ages 3–9 yr). Specifically, the CCBS–2 can be used to differentiate among children according to age; pre– and post–school entry; children with and without a disability; and mothers who self-reported symptoms of depression, anxiety, or stress. Moreover, the CCBS–2 can be used to identify families who would potentially benefit from services in behavior management and mental health.
Screening is often the first step in determining who is eligible for further assessment, and it can be used to identify higher risk families who are likely to benefit from immediate interventions (Babor, Sciamanna, & Pronk, 2004; Fleming, 2002). As a screening measure, the CCBS–2 needs to be able to correctly classify and accurately predict grouping. Screening measures need to have sound sensitivity, specificity, positive predictive value, and negative predictive value (Attia, 2003). Sensitivity and specificity are the most commonly documented indices of screening efficiency. Further study of agreement among caregivers (e.g., mothers and fathers); reliability and intrarater reliability; and sensitivity, specificity, and criterion-related validity are important for further development of the CCBS–2. Further research is needed among populations of TD children and children with disabilities to investigate relationships with existing gold standard assessments, such as the Child Behavior Checklist (Achenbach & Ruffle, 2000).
To date, the CCBS–2 has been validated on mother–child dyads of children with disabilities and TD children. As such, there is a need to continue validating the CCBS–2 in other clinical population groups, such as children with ASD, attention deficit hyperactivity disorder, and other behavior disorders. Even though the CCBS–2 was designed to be a screening measure, it is possible that it could be used as an outcome measure in studies aimed at reducing difficult and challenging behaviors. Current evidence supports the utility of the CCBS–2 for clinical and research purposes and suggests that further psychometric evaluation is warranted.
Footnotes
Acknowledgments
This research was funded by a Monash University Early Career grant. We thank the occupational therapists Aislinn Lalor, Rebecca Stephan, and Sarah Grzegorczyn for assistance with this project.