Specificity of the CAARS in Discriminating ADHD Symptoms in Adults From Other Axis I Symptoms

Abstract

Objective: In this study, we examined the sensitivity and specificity of the Conners’ Adult ADHD Rating Scale–Self-Report: Long Version (CAARS) in 113 adult clinical archival records. Method: Forty-five clients had requested evaluation for ADHD, suggesting problems with attention, and 68 requested other services. To examine the CAARS’ ability to differentiate ADHD symptoms from other Axis I symptoms, it was compared with the Personality Assessment Inventory. Results: The two groups differed significantly on the weighted linear combination of the eight subscales of the CAARS, Wilks’s Lambda = .565, F(7, 105) = 11.56, p < .0001, with higher mean scores found among those requesting evaluation of ADHD. z-tests revealed the eight CAARS subscales were more highly correlated with each other, based on the average intercorrelation, than the nine selected clinical scales of the Personality Assessment Inventory (PAI; Somatic Complaints [SOM], Mania [MAN], Paranoia [PAR], Schizophrenia [SCZ], Anxiety [ANX], Anxiety-Related Disorders [ARD], Depression [DEP], Borderline Features [BOR], and Antisocial Features [ANT]). Some unexpectedly high correlations were found between the CAARS and PAI clinical scales (MAN and SCZ). Conclusion: The results of the present study were mixed, with some analyses yielding positive results with respect to the CAARS’ sensitivity and others suggesting poor specificity.

Keywords

assessment adult ADHD sensitivity specificity CAARS

Taylor and Miller (1997) indicate that recent media coverage has increased the number of self-referrals for ADHD assessment among adults. The difficulties in diagnosing this disorder in adults have been widely discussed elsewhere, including problems in differential diagnosis (see, for example, Conners & Jett, 2001; Faigel, 1995; Katz, Wood, Goldstein, Auchenbach, & Geckle, 1998) and the presence of comorbid conditions (see, for example, Barkley, 1998; Downey, Stelson, Pomerleau, & Giordani, 1997). Although research on the assessment of ADHD in adults is increasing, most of the literature has centered around the comparison of those with ADHD with a control group (e.g., Heiligenstein, Guenther, Levy, Savino, & Fulwiler, 1999; O’Donnell, McCann, & Pluth, 2001; Richards, Rosen, & Ramirez, 1999; Roy-Byrne et al., 1997), but not with those with diagnoses that present similar symptoms, such as depressive or anxiety disorders. It is important to make these contrasts to evaluate the methods used in the assessment of ADHD in adults.

Corbett and Stanczak (1999) state that the most sensitive and useful tool in adult ADHD assessment is a well-constructed interview, whereas Schoechlin and Engel (2005) report that informant and self-report rating scales are the pivotal instruments used in diagnosing ADHD. Others have identified a battery approach, using neuropsychological instruments, as preferred in enhancing a clinician’s ability to detect ADHD in adults (Woods, Lovejoy, Stutts, Ball, & Fals-Stewart, 2002). Reflecting the lack of consensus regarding a single approach, using multiple methods to gather information has been championed as an effective way to diagnose someone with this disorder accurately (Weyandt, Linterman, & Rice, 1995). In support of the multimethod approach, Murphy and Adler (2004) indicate that the evaluation of ADHD in adults should include a thorough clinical interview, self-report rating scales, and informant-rating scales, as well as objective supporting evidence, such as behavioral observations. They emphasize that self-report symptom assessment scales cannot be used as stand-alone agents for the diagnosis of ADHD because of the difficulty in determining childhood onset and in ruling out other psychiatric disorders accounting for the symptoms. Nevertheless, self-report instruments are a clinical staple, providing a good deal of information in an efficient manner, and may be an important part of a multimodal approach to assessment.

The Conners’ Adult ADHD Rating Scale was developed to aid in the assessment and diagnosis of adult ADHD (Conners, Erhardt, & Sparrow, 1999), with self-report and informant-rating forms. The CAARS–Self-Report: Long Form (referred to in the remainder of this manuscript as CAARS) produces four factor analysis-derived subscales, three Diagnostic and Statistical Manual of Mental Disorders (4th ed., DSM-IV; American Psychiatric Association, 1994) symptom subscales, an ADHD Index (intended to identify people who are “likely ADHD”), and an Inconsistency Index. However, a potential problem with the CAARS is that its subscales may be sensitive to not only ADHD symptoms but also other Axis I disorders that include inattention as a prominent symptom as well. Although there is support for the convergent validity of the CAARS, further research is needed that examines its ability to discriminate between symptoms of ADHD and similar features of other disorders (i.e., anxiety and depressive disorders; Erhardt, Epstein, Conners, Parker, & Sitarenios, 1999).

The goal of the present study was to investigate the validity of the CAARS in a mixed clinical population. In particular, the pattern of correlations between the CAARS and another measure of clinical symptoms, the Personality Assessment Inventory (PAI; Morey, 1991), was examined. It was anticipated that the CAARS subscales would correlate more strongly with one another than with PAI scales. It was also expected that CAARS subscales would correlate more strongly with PAI scales that share features of inattention (e.g., Depression [DEP] and Anxiety [ANX] scales) than with scales with less overlap in symptoms (e.g., Paranoia [PAR] and Schizophrenia [SCZ] scales). Investigation of which CAARS subscales best differentiated between clients complaining of ADHD-like symptoms and requesting evaluation for ADHD from those presenting with other difficulties was also undertaken. Although it was hypothesized that CAARS scores would be higher in the former group, no specific hypotheses were made regarding the ability to differentiate between the two groups based on specific CAARS subscales.

Method

Participants

Participant data came from client records at an outpatient mental health facility in the Midwest. Client data from a 12-month period were collected, yielding 113 archival records. All adult clients from this period who requested services were asked to complete the PAI and CAARS as a part of the standard paperwork prior to receiving services. This sample consisted of records for 72 females and 41 males, whose ages ranged from 18 to 53 with a mean age of 27.09 years (SD = 9.37).

Demographic data, presenting problem(s) and discharge diagnoses, were also retrieved from client files. Participant data were divided into two groups. The first group consisted of data from those clients who requested an ADHD evaluation, suggesting that they were initially experiencing problems with attention (n = 45). In all, 6 of these clients were eventually diagnosed with ADHD, 15 did not complete the full evaluation, and the remaining 24 received other diagnoses. The second group consisted of data from those who requested other services such as evaluations for learning disorders, other psychodiagnostic evaluations, and/or therapeutic services, suggesting that they were experiencing other difficulties (n = 68). None of those in the latter group requested evaluation of ADHD, despite it being a listed option on the paperwork forms completed prior to receiving services. The groups neither differed significantly by age, F(1, 111) = 0.268, p > .05, gender, χ²(1) = 1.14, p > .05, and race, χ²(2) = 0.97, p > .91, nor by marital status, χ²(2) = 1.56, p > .46.

Materials

CAARS

The present study examined the self-report, long version of the CAARS. The long version of the CAARS self-report form contains 66 items, scored on a 5-point Likert-type scale, which make up the eight clinical subscales. Four of the clinical subscales have demonstrated empirical support through factor analysis (Conners, Erhardt, & Sparrow, 1999) and are as follows: CAARS-A (Inattention/Memory Problems), CAARS-B (Hyperactivity/Restlessness), CAARS-C (Impulsivity/Emotional Lability), and CAARS-D (Problems With Self-Concept). The three DSM-IV derived scales include CAARS-E (DSM-IV Inattentive Symptoms), CAARS-F (DSM-IV Hyperactive–Impulsive Symptoms), and CAARS-G (Symptoms Total). The CAARS-H ADHD Index is an additional subscale intended to identify people who are “likely ADHD.” Conners, Erhardt, Epstein, et al. (1999) indicate that a person with a T-score above 70 on the ADHD Index is likely to have significant levels of ADHD symptoms that may meet diagnostic criteria. The CAARS also contains an Inconsistency Index to aid in identifying random or careless responding.

Previous studies have yielded internal consistency coefficients of .89 to .92 and a median test–retest reliability coefficient of .89 over the period of 1 month (Erhardt et al., 1999). Correlations between the CAARS and other self-report measures of ADHD have been found to be moderate (Erhardt et al., 1999). An overall classification rate of 85%, based on accurately differentiating those with ADHD from those without ADHD, has been reported by these authors.

PAI

The PAI is an objective, self-report measure intended to assess a number of clinical as well as personality variables in adults. Participants are presented with 344 items, each scored on a 4-point scale. Items on the PAI form 22 non-overlapping scales. Included among these are 4 validity scales, 11 clinical scales, 2 interpersonal styles scales, and 5 treatment-relevant scales. Internal consistency coefficients for the PAI scales range from .81 to .86 for normative, clinical, and college samples (Morey, 1991). A median test–retest reliability correlation has been reported as .83 (Morey, 1991).

Results

Zero-order correlations among the eight CAARS subscales (see Table 1) and between the eight CAARS subscales and selected scales from the PAI (see Table 2) were calculated for the total sample (N =113). The selected scales from the PAI included nine clinical scales (Somatic Complaints [SOM], Mania [MAN], PAR, SCZ, ANX, Anxiety-Related Disorders [ARD], DEP, Borderline Features [BOR], and Antisocial Features [ANT]).

Table 1.

Intercorrelations Among the CAARS Subscales (N = 113)

Subscales	A	B	C	D	E	F	G	H
A	__	.59**	.68**	.43*	.76***	.50**	.68***	.77***
B		__	.60**	.28	.67***	.76***	.75***	.72***
C			__	.47*	.65***	.61***	.66***	.80***
D				__	.39	.23	.38	.63***
E					__	.69***	.89***	.80***
F						__	.85***	.64***
G							__	.79***
H								__

p < .05. **p < .01. ***p < .0001.

Table 2.

Correlations Between CAARS Subscales and PAI Scales (N = 113)

Subscales	A	B	C	D	E	F	G	H	Average correlation
ANX	.44*	.31	.48*	.55**	.31	.26	.31	.50*	.41
ARD	.22	.20	.39	.47*	.16	.14	.18	.37	.29
DEP	.43*	.22	.53*	.68***	.34	.16	.28	.53*	.43
BOR	.48*	.43*	.67***	.55**	.48*	.43*	.43*	.60**	.52
ANT	.26	.36	.48*	.05	.44*	.57**	.45*	.33*	.40
SOM	.17	.08	.26	.40*	.17	.06	.10	.26	.22
MAN	.34	.51*	.49*	.13	.49*	.64**	.56*	.47*	.48
PAR	.22	.13	.50*	.32	.24	.21	.21	.34	.29
SCZ	.47*	.31	.57**	.55**	.50*	.36	.46*	.56**	.48

Note. CAARS = Conners Adult ADHD Rating Scale; CAARS-A = Inattention/Memory Problems; CAARS-B = Hyperactivity/Restlessness; CAARS-C = Impulsivity/Emotional Lability; CAARS-D = Problems With Self-Concept; CAARS-E = DSM-IV Inattentive Symptoms; CAARS-F = DSM-IV Hyperactive Symptoms; CAARS-G = Symptoms Total; CAARS-H = ADHD Index; DSM-IV = Diagnostic and Statistical Manual of Mental Disorders (4th ed.); ANX = Anxiety; ARD = Anxiety-Related Disorders; DEP = Depression; BOR = Borderline Features; ANT = Antisocial Features; SOM = Somatic Complaints; MAN = Mania; PAR = Paranoia; SCZ = Schizophrenia. Significance levels were not tested on average correlations.

p < .05. **p < .01. ***p < .0001.

Intercorrelations of CAARS subscales ranged from .23 to .89, with a median correlation of .67. The six lowest intercorrelations involved the CAARS-D subscale, Problems With Self-Concept. The six highest intercorrelations involved the CAARS-H subscale, ADHD Index. The mean intercorrelation of the eight CAARS subscales was r = .65. This value was used to compare the CAARS intercorrelations with the selected PAI scales. Using Fisher’s r to z transformation, a series of z tests revealed the eight CAARS subscales correlated significantly higher with each other, using a Bonferroni-adjusted α = .0056 (.05/9) and based on the average intercorrelation, than they correlated with the selected clinical scales of the PAI. This was true for those scales reflecting symptoms/syndromes commonly overlapping with ADHD (such as DEP and ANX) and scales assessing symptoms/syndromes not typically associated with ADHD (such as SCZ and PAR).

The CAARS subscales of those who initially sought an ADHD evaluation were compared with those of clients who requested other services (i.e., learning disability evaluation and/or therapeutic services). A multivariate analysis of variance was used to compare the two groups on the weighted linear combination of the eight subscales of the CAARS, achieving an overall significance at the p < .0001 level, Wilks’s Lambda = .565, F(7, 105) = 11.56. Means and standard deviations for the CAARS subscales can be found in Table 3.

Table 3.

Means and Standard Deviations for the CAARS Subscales

	Other requested services (n = 68)		ADHD evaluation (n = 45)
	M	SD	M	SD
CAARS-A	56.63	12.04	67.97	12.88
CAARS-B	50.97	9.69	60.51	9.79
CAARS-C	52.65	12.10	57.78	12.40
CAARS-D	59.96	12.27	61.09	12.02
CAARS-E	60.60	14.26	80.38	10.98
CAARS-F	50.10	10.98	62.04	14.13
CAARS-G	56.66	14.49	75.31	12.73
CAARS-H	55.88	11.59	66.67	10.17

A follow-up logistic multiple regression analysis (with the dependent variable as group and the CAARS subscales as predictor variables) was conducted, to determine which subscales contributed the greatest proportion of unique variance in distinguishing between the two groups. This analysis indicated that the CAARS-E (DSM-IV Inattentive Symptoms) contributed the greatest proportion of variance (R² = .359), followed by CAARS-C (DSM-IV Impulsivity/Emotional Lability), adding an additional 9.7%, and CAARS-D (Problems With Self-Concept), which accounted for an additional 3.3% of the variance.

Discussion

The goal of this study was to investigate the validity of the CAARS in a mixed clinical population. Evidence that the CAARS can discriminate between ADHD and other Axis I disorders that share similar symptoms as ADHD (i.e., inattention, hyperactivity, etc.) would assist clinicians in differential diagnosis. While previous research suggests that the CAARS may be sensitive to ADHD-like symptoms, the question posed in this investigation was whether the CAARS can detect symptoms specific to ADHD.

In the present research, those requesting an ADHD evaluation and those requesting other services differed significantly on the eight subscales of the CAARS. Further analysis indicated that the CAARS subscale E (DSM-IV Inattentive Symptoms) was the best predictor for distinguishing between the two groups. This is not surprising, given that the hallmark symptom of ADHD in adulthood is inattention. The CAARS subscales C (Impulsivity and Emotional Lability) and D (Problems With Self-Concept) were found to contribute to the distinction between the groups. That these two subscales were the only others to contribute to the prediction of group membership may be due to the high degree of overlap of subscale E and other adult ADHD constructs, and hence, their subscales. On all three subscales, the group requesting ADHD evaluations scored higher, suggesting a greater severity of self-reported symptoms. The finding of a significant difference between the two groups in the expected direction provides support for the measure’s sensitivity to ADHD-like symptoms. A limitation to this interpretation is that client records were placed into groups based on client self-selection, that is, whether they initially sought services for attention problems. Therefore, all measures and the grouping item used in this study were based on client self-report; there was no clinician judgment used to guide the selection of the groups. Therefore, the finding that those complaining of inattention at intake reported greater symptomatology on the CAARS may not be remarkable.

Within the full sample of those complaining of attention difficulties and those reporting other primary clinical concerns, correlations among the eight CAARS subscales were significantly greater than those between the CAARS and the clinical scales from the PAI. This indicates that clients reporting some symptoms of ADHD are more likely to report other ADHD-related symptoms than they are to report features of other disorders. This suggests that inattention, a hallmark feature of ADHD but a prominent symptom of many other Axis I disorders, was more likely to be reported in conjunction with other ADHD symptoms.

Correlations between the CAARS subscales and scales from the PAI that share features or overlapping symptoms with ADHD, that is, ANX and DEP, again in looking at the self-reports of the entire sample, were moderate. These PAI scales are associated with anxiety problems and depressive symptoms in adults, two diagnostic groupings commonly occurring comorbidly with ADHD or that share features such as impaired concentration. Several other PAI scales were also significantly correlated with many of the subscales of the CAARS: BOR, MAN, and SCZ. Clients who elevated the BOR scale of the PAI as well as subscales of the CAARS may have been reporting impulsivity and emotional instability, which are features of both syndromes. That the MAN and SCZ scales were significantly correlated with six of the eight CAARS subscales was not anticipated by the current investigation as these syndromes are not commonly associated with ADHD and may be found in psychotic disorders or syndromes. Although this finding may indicate problems with the CAARS, it may be partially understood by examining the subscales that compose those PAI scales. The SCZ scale contains a Thought Disorder subscale, the items of which may have been interpreted by the clients as consistent with inattention or that may actually be tapping into inattentive symptoms. Similarly, the MAN scale includes an Activity Level subscale that may have been interpreted by some clients as reflective of features of hyperactivity or may, in fact, be measuring something similar to hyperactivity. Examining these subscales in the context of the differential diagnosis of ADHD in adults is an area for further study.

As the number of adults requesting assessment for ADHD is increasing, it is important to investigate the measures that are used in helping diagnose this disorder. The results of the present study were mixed, with some analyses yielding positive results with respect to the CAARS’ sensitivity and others suggesting poor specificity. As is a frequent recommendation in assessment research, these results again support the use of a multimethod evaluation that includes careful history-taking in the context of a clinical interview, review of records or documentation, multiple measures of ADHD and other clinical syndromes, and/or neurocognitive measures.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Biographies

Annie Stewart earned her MA in clinical psychology at Murray State University. Since then, she has worked as a therapist, behavior analyst, and most recently as clinical manager at Neurorestorative, a facility for individuals with brain injuries.

Laura Liljequist is a professor of psychology at Murray State University where she coordinates the master’s degree program in clinical psychology. She practices and researches in various areas of assessment.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Barkley

R. A.

(1998). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed.). New York, NY: Guilford.

Conners

C. K.

Erhardt

Epstein

J. N.

Parker

J. D. A.

Sitarenios

Sparrow

(1999). Self-ratings of ADHD symptoms in adults I: Factor structure and normative data. Journal of Attention Disorders, 3, 141-151.

Conners

C. K.

Erhardt

Sparrow

(1999). Conners’ adult ADHD rating scales: Technical manual. North Tonawanda, NY: Multi-Health Systems.

Conners

C. K.

Jett

J. L.

(2001). Attention-deficit hyperactivity disorder (in adults and children): The latest assessment and treatment strategies. Kansas City, MO: Compact Clinicals.

Corbett

Stanczak

D. E.

(1999). Neuropsychological performance of adults evidencing attention-deficit hyperactivity disorder. Archives of Neuropsychology, 14, 373-387.

Downey

K. K.

Stelson

F. W.

Pomerleau

O. F.

Giordani

(1997). Adult attention-deficit hyperactivity disorder: Psychological test profiles in a clinical population. Journal of Nervous and Mental Diseases, 185, 32-38.

Erhardt

Epstein

J. N.

Conners

C. K.

Parker

J. D. A.

Sitarenios

(1999). Self-ratings of ADHD symptoms in adults II: Reliability, validity, and diagnostic sensitivity. Journal of Attention Disorders, 3, 153-158.

Faigel

H. C.

(1995). Attention-deficit disorder in college students: Facts, fallacies, and treatment. Journal of American College Health, 43, 147-155.

10.

Heiligenstein

Guenther

Levy

Savino

Fulwiler

(1999). Psychological and academic functioning in college students with attention-deficit hyperactivity disorder. Journal of American College Health, 47, 181-185.

11.

Katz

L. J.

Wood

D. S.

Goldstein

Auchenbach

R. C.

Geckle

(1998). the utility of neuropsychological tests in evaluation of attention-deficit/hyperactivity disorders (ADHD) versus depression in adults. Assessment, 5, 45-51.

12.

Morey

L. C.

(1991). Personality Assessment Inventory: Professional manual. Odessa, FL: Psychological Assessment Resources.

13.

Murphy

K. R.

Adler

L. A.

(2004). Assessing attention-deficit/hyperactivity disorder in adults: Focus on rating scales. Journal of Clinical Psychiatry, 65, 12-17.

14.

O’Donnell

J. P.

McCann

K. K.

Pluth

(2001). Assessing adult ADHD using a self-report symptom checklist. Psychological Reports, 88, 871-881.

15.

Richards

T. L.

Rosen

L. A.

Ramirez

C. A.

(1999). Psychological functioning differences among college students with confirmed ADHD, ADHD by self-report only, and without ADHD. Journal of College Student Development, 40, 299-304.

16.

Roy-Byrne

Sheele

Brinkley

Ward

Wiatrak

Russo

Variey

(1997). Adult attention deficit hyperactivity disorder: Assessment guidelines based on clinical presentation to a specialty clinic. Comprehensive Psychiatry, 38, 133-140.

17.

Schoechlin

Engel

R. R.

(2005). Neuropsychological performance in adult attention-deficit hyperactivity disorder: Meta-analysis of empirical data. Clinical Neuropsychology, 20, 727-744.

18.

Taylor

C. J.

Miller

D. C.

(1997). Neuropsychological assessment of attention in ADHD adults. Journal of Attention Disorders, 2, 77-88.

19.

Weyandt

L. L.

Linterman

Rice

J. A.

(1995). Reported prevalence of attentional difficulties in a general sample of college students. Journal of Psychopathology and Behavioral Assessment, 17, 293-304.

20.

Woods

S. P.

Lovejoy

D. W.

Stutts

M. L.

Ball

J. D.

Fals-Stewart

(2002). Comparative efficiency of a discrepancy analysis for the classification of attention-deficit/hyperactivity disorder in adults. Clinical Neuropsychology, 17, 351-369.