Abstract
A major issue in psychology involves the relative validity of self-reports of psychological characteristics versus reports made by others. Given that modest correlations between self- and other-reports of various traits and characteristics are the norm rather than the exception (Achenbach, Krukowski, Dumenci, & Ivanova, 2005), clinicians often are faced with how best to reconcile diverse reports of a person’s characteristics or behaviors. Confronted with diverging reports, the relative validity of self- versus other-reports is an important consideration.
Within the social psychology literature, Wilson and Dunn (2004) argued that biases in self-perceptions render accurate self-knowledge difficult to obtain. Supporting this argument, studies assessing a range of characteristics and outcomes in community samples have found that other-reports are just as valid as, if not more valid than, self-reports. For example, Vazire and Mehl (2008) found that other-reports were at least as accurate as self-reports for predictions of daily behavior. In addition, John and Robins (1994) demonstrated that when compared with rankings made by trained assessors, individuals were more accurate in evaluating their peers than themselves on a managerial group-discussion task. Similarly, Kolar, Funder, and Colvin (1996) found that other-reports of personality judgments were more strongly correlated with a behavioral criterion than self-reports. Moreover, studies that test the associations of self- and other-reports to objective life outcomes suggest that other-reports of personal characteristics have greater predictive validity than self-reports. For instance, other-reports of hostility and antagonism are more highly associated with the severity of coronary artery calcification, an indicator of coronary artery disease (Smith et al., 2007).
Although this plethora of research exists examining the validity of self- versus other-ratings of normative personality and behavioral attributes, fewer studies have investigated the issue of self- versus other-rater validity with respect to characteristics associated with psychopathologies. One such study, by Fiedler, Oltmanns, and Turkheimer (2004), replicated the results from community samples and found that other-reports of personality disorder traits have greater predictive validity in terms of military job outcome. In this study, we extend this question to the validity of self- and other-reports of symptoms of ADHD. The high prevalence of adult ADHD (Faraone & Biederman, 2005), where self-reports play a central role in assessment, makes this question particularly relevant. In addition, it is known that individuals with high levels of ADHD symptoms tend to have a positive illusory bias (PIB) in that they see themselves in a much more positive light than how others see them (Knouse, Bagwell, Barkley, & Murphy, 2005; Owens, Goldfine, Evangelista, Hoza, & Kaiser, 2007). This bias again casts doubt on the accuracy of self-reports of individuals with symptoms of ADHD and suggest that others’ reports may present a less biased perspective.
Existing studies examining self- versus other-reports of ADHD symptoms have yielded conflicting results. Some have compared self-reports of ADHD symptoms to “gold standard” assessments involving diagnostic interviews. For example, Magnusson et al. (2006) and Kooij et al. (2005), across youth and adults, found that self-reports of ADHD symptoms correlated more highly with diagnostic interviews than did reports by other raters. In contrast, when comparing self- and other-reports of ADHD symptoms for college students who presented with academic problems, Zucker, Morris, Ingram, Morris, and Bakeman (2002) found no differential relationships between diagnostic interviews and self- versus other-reports. As another example, Belendiuk, Clarke, Chronis, and Raggi (2007) found that in mothers of children with ADHD, self-reported ADHD symptoms were significantly associated with diagnostic interviews conducted with the mothers but were less associated with interview information gathered from other reporters. In addition to concerns with shared rater variance, these studies are limited by a narrow focus on the relationship of reports of ADHD symptoms to actual diagnosis.
Other studies comparing the validity of self- versus other-reports of ADHD symptoms have examined the independent relationships of these reports to functioning or impairment. Controlling for shared rater variance, Barkley, Fischer, Smallish, and Fletcher (2002) followed adults who had been diagnosed with ADHD in childhood and found that the level of ADHD symptoms in adulthood varied drastically depending on whether the adults themselves or their parents provided the ratings. Self-reports were significantly correlated with only two of eight outcome measures, whereas parent-reports were significantly associated with all eight outcome measures (years of education, high school grades, high school class ranking, employer-rated ADHD symptoms, employer-rated work performance, number of jobs fired from, number of friends, and number of arrests). This study provides support for the validity of other-reports of ADHD symptoms of adults when these symptoms are used as predictors of life functioning.
Given the limited number of studies and relatively restricted range of criteria assessed, there appears to be no conclusive answer as to whether self- or other-reports of ADHD symptoms are more valid, although one might argue for the preponderance of evidence supporting the greater validity of other-reports. This study extends this line of research and examines the relative validity of self- versus other-reports of ADHD symptoms in adults as predictors of competence in a variety of important life domains. Competence is an important criterion variable because of its direct implications for functional impairment, an essential component in the diagnosis of ADHD. It was predicted that other-reports would be significantly related to more domains of competence than self-reports of ADHD symptoms.
In addition, this study examines whether adults with high levels of ADHD symptoms are more likely to show a PIB in their evaluations of competence by measuring whether self- and other-reports of ADHD symptoms are related to how positively the adults assess themselves in comparison to evaluations by others who know them well. The PIB has been well established in children with ADHD (Owens et al., 2007), and preliminary evidence supports the continuation of the PIB into adulthood for individuals with ADHD. For instance, Knouse et al. (2005) found that adults with ADHD provided overly positive self-assessments of driving competence in comparison with their relatively unsafe current and past driving behaviors. It was therefore hypothesized that higher ADHD symptoms, measured either by self- or other-report, would be associated with women self-reporting their competence more positively compared with the reports of others.
Finally, we examined two characteristics that might influence the validity of self-reports of ADHD symptoms. First, given that depressive symptoms are associated with both ADHD symptoms (Kessler et al., 2006) and more negative self-views (Pyszczynski & Greenberg, 1987), we predicted that depressive symptoms would attenuate the relationship between self-reported ADHD symptoms and competence ratings. In addition, consistent with studies examining the PIB among children with ADHD (Hoza et al., 2004), it was hypothesized that depressive symptoms would attenuate the relationship between self-reported ADHD symptoms and the PIB. Therefore, we controlled for depressive symptoms in secondary analyses to assess whether these might account for the relationships between self-reported ADHD symptoms and competence ratings or the PIB. The second potentially confounding variable that we examined was response style, specifically self-deceptive enhancement and impression management tendencies. Given that a general tendency to “fake good” or overenhance one’s accomplishment may contribute to self-ratings of competence, we considered whether ADHD symptoms would be related to competence above and beyond these presentation styles.
Method
Participants
Ninety-one pairs of women and other raters were used in the analyses. A recruitment procedure that oversampled from those likely to have high levels of ADHD symptoms was implemented. Recruitment occurred via notices in the community and at an ADHD Program of a local Children’s Hospital, at ADHD support groups, in a lab newsletter, and by contacts with parents in a volunteer registry. Each woman had to be able to speak and understand English, have access to the Internet, have at least one child between 3 and 16 years of age, 1 and have a person close to her who was willing to serve as the other rater. Descriptive information for the women and other raters is shown in Table 1.
Descriptive Statistics for Women and Other Raters
Note: SES = socioeconomic status on Hollingshead’s (1975) Four-Factor Index of Social Status; CAARS = Conners’ Adult ADHD Rating Scale; ASPP = adult self-perception profile; CES-D = Center for Epidemiologic Studies Depression Scale; PDS = Paulhus Deception Scales. Sample sizes varied between 79 and 90 across participant types and variables.
Ratings from 1 to 5 (1 = highest, 5 = lowest).
1 = not at all close; 7 = extremely close.
1 = not at all positive; 7 = extremely positive.
1 = 1-2 days per week; 2 = 3-4 days per week; 3 = 5-6 days per week; 4 = 7 days per week.
The majority (78%) of women were married. In all, 74% identified themselves as Euro-Canadian, 18% identified themselves as Asian Canadian, and 8% identified themselves as of other ethnicities. For the other raters, 59% were male and 41% were female. A total of 73% of other raters identified themselves as Euro-Canadian, 19% categorized themselves as Asian Canadian, and 8% were of other ethnicities. With respect to the relationships between the women and their other raters, 51% of the other raters were partners, 25% were relatives, and 24% were friends. Women and other raters rated the closeness and positiveness of their relationship as well as their frequency of contact with each other. Both the women and other raters generally indicated close, positive relationships with each other. However, there was one case in which the other rater rated their relationship with the woman as not at all close. Analyses were repeated with and without this case, with results that did not differ.
Procedure
The study was approved by our university’s ethics review board. On calling the lab, women were informed of the details of the study by a research assistant, and if they agreed to participate, were asked to designate a close relative or friend as the other rater. Women were then given a password and a link to access questionnaires online. They were mailed a consent form, which they signed and returned, providing the contact information of the other rater. Other raters were contacted by phone and were provided with a password and link to access questionnaires online. Women and other raters were informed and consented that neither would see each other’s responses. For women and other raters, the first page of the online questionnaires provided a description of the study and a check box to indicate consent to participate. Questionnaire order was counterbalanced across pairs of women and other raters but was the same within each pair. Women were mailed an honorarium of CAD$15 for participation in the study.
Measures
Primary Measures
Conners’ Adult ADHD Rating Scale: Screening Version (CAARS:SV)
Level of ADHD symptoms in the women was assessed using the self- and observer-reports of the CAARS:SV (Conners, Erhardt, & Sparrow, 1999). 2 The self-report version of the CAARS:SV is composed of 30 items assessing ADHD symptoms in adults, with a 4-point Likert-type scale of 0 (not at all, never) to 3 (very much, very frequently). The observer-report version of the CAARS:SV has items that are parallel to the self-report version but modified to refer to the individual being rated. The inattentive and hyperactive/impulsive subscales were summed to produce a total ADHD symptoms score. The CAARS:SV has been found to be psychometrically reliable and valid (Conners et al., 1999). The internal consistency for the measure in this study was .92 for the women and .87 for the other raters.
Adult Self-Perception Profile (ASPP)
The ASPP (Messer & Harter, 1986) was completed by the women and other raters to gather perceptions of the women’s competence. The ASPP is composed of 50 items with a 4-point scale designed to capture aspects of adult competence in 11 domains. To capture achievement and social competence, the domains that have been emphasized in studies of competence in children with ADHD, we used six of the domains: (a) sociability, (b) intimate relationships, (c) job competence, (d) household management, (e) intelligence, and (f) nurturance. For each item, two statements reflecting extremes of the type of competence (e.g., some adults are not very productive in their work BUT other adults are very productive in their work) are provided, and the women/other raters pick which of the two statements is like them/the woman and to what degree (Messer & Harter, 1986). Each item is scored on a scale of 1 to 4, with higher numbers reflecting greater perceived competence.
As reported by Messer and Harter (1986), the internal consistencies of the ASPP ranged from .65 for job competence to .82 for household management. In this study, the internal consistencies ranged from .66 to .85 across domains and across the women and other raters. Data suggestive of the validity of the ASPP includes evidence that full-time homemakers and unemployed individuals score lower on job competence than working individuals (Messer & Harter, 1986; Seitzman et al., 2004), and college-educated women score higher on the intelligence subscale than women with a high school education (Messer & Harter, 1986).
To yield information regarding the women’s competence that incorporates the perspectives of the self and the other individual, an average score for each ASPP domain was created on the basis of the ratings by the women and other raters. To test whether self- and other-reports of ADHD symptoms are related to how positively the women view themselves compared with how the other raters view them, standardized discrepancy scores between the women’s and other raters’ scores were calculated, as recommended by the literature (Owens et al., 2007), for each ASPP domain by first standardizing the self- and other-reported scores, and then subtracting other-reported scores from self-reported scores.
Covariate Measures
Center for Epidemiologic Studies Depression Scale (CES-D)
The CES-D (Radloff, 1977) assessed the women’s depressive symptoms. This self-report questionnaire consists of 20 items on a 4-point Likert-type scale ranging from 0 (rarely or none of the time) to 3 (most or all of the time) and has demonstrated reliability and validity (Radloff, 1977). The internal consistency of the measure in this study was .92.
Paulhus Deception Scales (PDS)
The PDS (Paulhus, 1999) measured self-deception and impression management tendencies. This questionnaire consists of 40 items divided into the Self-Deceptive Enhancement (SDE) Scale and the Impression Management (IM) Scale. Responses to each item are made on a 5-point Likert-type scale, ranging from 1 (not true) to 5 (very true). The SDE Scale refers to an unconscious bias to favor oneself and the IM Scale measures the tendency to falsely inflate one’s self-descriptions. Evidence suggests that the PDS are reliable and valid. The internal consistencies in this study were .74 and .79 for the SDE and IM Scales, respectively.
Results
Missing Data
Data missing for the primary variables ranged from 2.20% to 13.19% of cases. The t tests demonstrated that the data were missing at random, and variable-specific mean substitution was employed. Outliers and most instances of nonnormality were dealt with by variable transformation.
Descriptive Analyses
Descriptive information for all variables is presented in Table 1. To allow for ease of interpretation, these statistics are based on the data that have not undergone mean substitution or transformation. On the CAARS, both self- and other-reports indicated levels of ADHD symptoms within the average range (Conners et al., 1999). Also, CAARS self- and other-reports of ADHD symptoms were significantly correlated, r(89) = .40, p < .001. This correlation is highly similar to the interrater correlations between adults and other raters on externalizing disorders reported in a recent meta-analysis (Achenbach et al., 2005). As well, self- and other-reports in the ASPP domains were comparable to those reported by most homemakers and working women (Messer & Harter, 1986). Furthermore, the women scored within one standard deviation of the mean for a community sample on the CES-D (Radloff, 1977). On the SDE and IM Scales of the PDS, the women in this study scored slightly above average compared with the general population (Paulhus, 1999), suggesting some self-deception and impression management.
Do Self- and Other-Reports of ADHD Symptoms Relate to Competence Scores?
Bivariate correlations indicated that for self- and other-ratings, CAARS ratings were generally negatively associated with average ASPP scores (see Table 2), although this was particularly the case for the domains of job competence and household management. Nonsignificant correlations between the women’s age, socioeconomic status (SES), and ASPP scores (ps > .10) indicated no need to control for these demographic variables in subsequent regressions.
Correlations Between Reports of ADHD Symptoms and Average and Difference ASPP Scores
Note: CAARS = Conners’ Adult ADHD Rating Scale; ASPP = adult self-perception profile.
p < .05. **p < .01. ***p < .001.
Six regression models were constructed to determine the relationship of self- and other-reported CAARS scores to the ASPP scores in each domain of competence (see Table 3). The data were screened and assumptions checked for the models. The regression models were significant for the domains of intimate relationships, job competence, and household management. Only other-reports of the women’s CAARS scores were uniquely related to intimate relationships and job competence, whereas self- and other-reports of the women’s CAARS scores were significantly associated with competence in household management.
Regressions Examining the Relationships of Self- and Other-Reported ADHD Symptoms to Domains of Competence
Note: ASPP = adult self-perception profile; CAARS = Conners’ Adult ADHD Rating Scale.
To consider whether the relationship between CAARS and ASPP scores might be accounted for by the women’s depressive symptoms or response styles, the regressions were rerun controlling for response styles at the first step, with CAARS reports entered at the second step. These models yielded comparable results to the regressions conducted without controlling for response styles suggesting that self-deception and impression management did not account for the relationship between ADHD symptoms and competence. Second, regressions were conducted controlling for CES-D scores, and these again yielded parallel results to the main analyses, except that CAARS self-reports were no longer significantly related to competence in household management, p > .05. In sum, the women’s depressive symptoms appear to confound the only relationship between competence and the women’s CAARS self-reports of ADHD symptoms.
Do Self- and Other-Reports of ADHD Symptoms Relate to Competence Score Differences?
Bivariate correlations among self- and other-reported CAARS scores, and difference scores in ASPP domains were examined (see Table 2). CAARS self-reports were negatively correlated only with the difference score in Household Management, whereas other-reported CAARS scores were positively related to difference scores in five of the six ASPP domains. That is, although it was women with self-reported lower levels of ADHD symptoms who were most positively discrepant from other-raters in their assessments of their competence, other-raters’ reports of the women’s ADHD symptoms indicated that it was women with higher levels of ADHD symptoms who were more likely to overestimate their competence in comparison to others’ views of them. Again, nonsignificant correlations between the women’s age, SES, and ASPP difference scores indicated no need to control for these demographic variables.
Six regression models were constructed to determine the independent relationship of self- and other-reported CAARS scores to the ASPP difference scores in each of the competence domains (see Table 4). As before, the data were screened and assumptions checked. Results showed that both CAARS self- and other-reports were significantly related to difference scores in the domains of competence of intimate relationships, household management, intelligence, and nurturance, albeit in opposite directions. CAARS other-reports also were positively associated with sociability and job competence. Such results are actually consistent with a PIB in the reports of competence made by women with higher levels of ADHD symptoms.
Regressions Examining the Relationships of Self- and Other-Reported ADHD Symptoms to Difference Scores in Domains of Competence
Note: ASPP = adult self-perception profile; CAARS = Conners’ Adult ADHD Rating Scale.
To assess the possible influence of the women’s depressive symptoms or response styles on these predictions, regressions were rerun with response style scores entered in the first step and CAARS scores at the second step. Results were the same as the regressions reported in Table 4, except that the intelligence difference score was no longer significantly related to CAARS self-reports when SDE was controlled. Regressions conducted with IM scores controlled yielded results comparable with the regressions in Table 4. Finally, regressions controlling for CES-D scores yielded parallel results to those in Table 4, except that CAARS self-reports were no longer significantly predictive of the difference score in intelligence. In sum, depressive symptoms and self-deceptive response styles seem to confound the relationship between self-reports of ADHD symptoms and difference scores of competence.
Discussion
This study investigated the relative relationship between self- and other-reports of ADHD symptoms to competence in various important life domains among adult women. To assess competence, composite scores of self- and other-reports of the women’s competence for six major life domains were created. Consistent with predictions and prior literature (Barkley et al., 2002; Fiedler et al., 2004; John & Robins, 1994; Kolar et al., 1996; Smith et al., 2007; Vazire & Mehl, 2008), other-reports of ADHD symptoms were significantly negatively associated with competence in more life domains than self-reports. That is, the women whom other raters saw as having more ADHD symptoms had lower levels of competence. What’s more, the sole relationship between self-reported ADHD symptoms and competence was nonsignificant when the women’s depressive symptoms were controlled. Controlling for response styles did not attenuate the relationship between self-reported ADHD symptoms and household management competence.
In addition, the associations of self- and other-reports of ADHD symptoms with interrater differences in competence reports were examined. Difference scores between self- and other-reports of the women’s competence were created to index how positively the women viewed themselves in these life domains compared with how positively the other raters viewed these women. Surprisingly, self-reported ADHD symptoms were negatively related to how positively these women saw themselves compared with how others viewed them in the four life domains. In contrast, other-reported ADHD symptoms were positively related to differences between self- and other-ratings of competence in all six life domains. That is, consistent with previous studies with children and adults (Knouse et al., 2005; Owens et al., 2007), those women whom the other person rated as having higher ADHD symptoms rated their own competence more positively compared with the evaluation from the other rater. Controlling for depression and self-deceptive response style decreased the number of domains in which self-reports of ADHD symptoms were associated with interrater differences in competence scores.
As expected, our results suggest that other-reports of ADHD symptoms are more valid correlates of competence than self-reports of ADHD symptoms. Self-reports of ADHD symptoms were only related to competence in one life domain, and this relationship was nonsignificant when depressive levels were controlled. This result suggests that the commonalities between depression and ADHD symptoms may account for the relationship between self-reports of ADHD symptoms and competence.
The domains that were significantly associated with other-reports of ADHD symptoms were intimate relationships, job competence, and household management. These areas are domains in which ADHD-related impairments are well established (Eakin et al., 2004; Harpin, 2005; Murphy & Barkley, 1996). The areas of competence not related to other-reports of ADHD symptoms were sociability, intelligence, and nurturance. We speculate that this may be because these domains are less clearly affected by symptoms of ADHD. For example, although adults with ADHD may be perceived by others as lacking in social skills (Weiss & Murray, 2003), three of the four sociability items on the ASPP focus on ease and comfort in social interactions, rather than skill per se. Similarly, two of the four nurturance items on the ASPP ask whether the individual partakes in and enjoys caretaking, rather than specifically assessing competence. The lack of association between ADHD symptoms and competence in the intelligence domain is puzzling and may reflect the high level of social desirability of these items (e.g., this adult is just as smart as other adults) and the hesitance of other raters to offer negative evaluations on such items.
Self- and other-reports of ADHD symptoms were differentially related to difference scores in competence, which is in fact consistent with a PIB among adults with ADHD symptoms. Specifically, the women whom the other raters saw as having more severe ADHD symptoms rated themselves as higher in competence in all of the life domains assessed compared with the reports of other raters for these same domains. If other-reports of ADHD symptoms are taken to be more valid, this result suggests that women with higher ADHD symptoms possess a PIB in that they see themselves more positively in life competencies than others see them. It is more difficult to explain the unexpected negative associations between women’s self-reports of ADHD symptoms and the difference scores in competence. We speculate that the PIB may influence not only women’s self-reports of competence but also their reports of ADHD symptoms such that when women overestimate their competence, they are also likely to underestimate, or underreport, their level of ADHD symptoms. The weak power of self-reports of ADHD symptoms to relate to competence measures is further established when depression and response style were controlled and self-reports were associated with even fewer difference scores of competence.
Conclusion
Overall, the results suggest that greater weight should be given to other-reports of ADHD symptoms relative to self-reports of ADHD symptoms in adult women, especially if validity is pragmatically defined as the ability to predict clinically relevant behavior. The unique validity of self-reports, over and above the information offered by other-reports, is weak and decreases when variables such as depressive symptoms and response styles are taken into consideration. In addition, if other-reports are viewed as more valid than self-reports, our results suggest the existence of a PIB among adult women with high levels of ADHD symptoms.
Limitations and Future Directions
This study is not without limitations. First, although we assessed competence using composite and difference scores that considered self- and other-reports and thus reduced the shared rater variance with reports of ADHD symptoms, we did not use more objective measures of competence, such as actual records and observations of competence or employing more than one other rater. A multirater, multimethod study with more objective competence measures would permit stronger conclusions. Another limitation concerns the fact that one obviously cannot infer causation from the correlations found between ADHD symptoms and competence. Although the women’s depression and response styles were controlled, other possible third-variable explanations were not addressed, and the direction of influence between ADHD symptoms and competence cannot be inferred with this cross-sectional research design. Future studies will benefit from longitudinal designs. A further limitation rests with the nonclinical nature of the sample. Only seven of the women in this sample had actually been diagnosed with ADHD, although 59 had scores above the 65th percentile on the CAARS as measured by either self- or other-report. However, evidence suggests that ADHD is a dimensional construct, and there is as yet no consensus regarding the precise cutoff point that should be used to separate adults with ADHD from adults without the disorder (Levy, Hay, McStephen, Wood, & Waldman, 1997). Regardless, we acknowledge that there may be differences between adults diagnosed with ADHD versus adults with high ADHD symptoms and that these results may not extend to clinical samples. Future studies could investigate the relationship between self- and other-reports to competence in a sample of adults who have been diagnosed with ADHD.
Footnotes
Acknowledgements
The authors wish to thank the participants and the research assistants, Kelly McHale and Anisa Sudarsono, for their helpful efforts.
This research was part of the MA thesis of the first author.
The authors declared no potential conflicts of interests with respect to the authorship and/or publication of this article.
The article was funded by the Canadian Institutes of Health Research, the Michael Smith Foundation for Health Research, and the Social Sciences and Humanities Research Council of Canada.