Diagnosing ADHD in Adults: An Examination of the Discriminative Validity of Neuropsychological Tests and Diagnostic Assessment Instruments

Abstract

Objective: The objective of this study is to investigate the discriminative validity of neuropsychological tests and diagnostic assessment instruments in diagnosing adult ADHD in a clinical psychiatric population. Method: Of 108 patients, 60 were diagnosed with ADHD. The Diagnostic Interview for ADHD in adults (DIVA 2.0) and Adult ADHD Self-Report Scale (ASRS) v.1.1 together with eight neuropsychological tests were investigated. Results: All instruments showed poor discriminative ability except for the DIVA, which showed a relatively good ability to discriminate between the groups (sensitivity = 90.0; specificity = 72.9). A logistic regression analysis model with the DIVA and measures of inattention, impulsivity, and activity from continuous performance tests (CPTs) showed a sensitivity of 90.0 and a specificity of 83.3. Conclusion: Neuropsychological tests have a poor ability to discriminate between patients diagnosed with ADHD and patients not diagnosed with ADHD, but variables from CPT tests can contribute to increasing the specificity by 10% if used in combination with the DIVA.

Keywords

adult ADHD diagnostic validity sensitivity specificity

Introduction

Prevalence studies show that 2% to 7% of the adult population meet the criteria for ADHD (Fayyad et al., 2007; Kessler et al., 2006; Simon, Czobor, Balint, Meszaros, & Bitter, 2009). In Sweden, the number of adult patients diagnosed with ADHD has increased in recent years (Giacobini, Medin, Ahnemark, Russo, & Carlqvist, 2014). In clinical practice, patients who are assessed for ADHD generally display a complex mix of symptoms, such as problems in attention, concentration, restlessness, and impulsiveness. Because there can be many reasons for having these symptoms, it is crucial to investigate whether the difficulties are truly neurodevelopmental, and the assessment therefore needs to be comprehensive. Guidelines for the assessment of ADHD vary between countries, but a review of 13 different national guidelines shows a general consensus for the importance of collecting data from different sources, such as clinical psychiatric interview with the patient and an informant that knew the patient as a child, screening for comorbidity, the use of rating scales and questionnaires, and a physical examination (Seixas, Weiss, & Muller, 2012). All guidelines in the review indicated that the diagnosis should be based on the full clinical interview, including assessment of mental state, functional impairment, developmental comorbidity, and family history, and that the clinical interview remains the “gold standard” of assessment of ADHD. According to the review, there was variation among the guidelines regarding the importance of including a neuropsychological evaluation to establish diagnosis. As far as we know, there is no current support for any single neuropsychological test or battery of neuropsychological tests that can satisfactorily distinguish between individuals with ADHD and those with other psychiatric diagnoses. Although research in this area is still limited, several meta-analyses have shown that adults with ADHD generally do not perform as well as healthy controls on neuropsychological tests, particularly those related to sustained and focused attention, inhibition, verbal fluency, and verbal memory (Balint et al., 2009; Boonstra, Oosterlaan, Sergeant, & Buitelaar, 2005; Frazier, Demaree, & Youngstrom, 2004; Hervey, Epstein, & Curry, 2004; Schoechlin & Engel, 2005). When adults with ADHD are compared with other psychiatric patients, the results are more inconsistent (Abramovitch, Dar, Hermesh, & Schweiger, 2012; Epstein, Johnson, Varia, & Conners, 2001; Torralva et al., 2011), and few studies have reported measures of sensitivity and specificity. Results of the few available studies suggest that the discriminative validity of neuropsychological tests is poor when individuals with ADHD are compared with clinical controls, and most suggest diminished specificity (Edebol, Helldin, Holmberg, Gustafsson, & Norlander, 2011; Edebol, Helldin, & Norlander, 2012; Holst & Thorell, 2013; Katz, Wood, Goldstein, Auchenbach, & Geckle, 1998; Walker, Shores, Trollor, Lee, & Sachdev, 2000), with the exception of Söderström and colleagues, who found high specificity for the variable Inattention from the QBTest Plus (Söderström, Pettersson, & Nilsson, 2014).

Continuous performance tests (CPTs) belong to the most common type of neuropsychological tests used in both clinical settings and research for the evaluation of ADHD. There are many versions of CPT tests, but they are generally computer-based vigilance tests aiming at assessing functions such as sustained and selective attention and behavioral inhibition. One of the most widely used and most frequently studied CPT tests in adult ADHD is Conners’ CPT. According to five meta-analyses, Conners’ CPT shows one of the largest test effect sizes in comparisons of adults with ADHD and normal adults (Balint et al., 2009; Boonstra et al., 2005; Frazier et al., 2004; Hervey et al., 2004; Schoechlin & Engel, 2005). The findings are less consistent between patients with ADHD and those with other psychiatric disorders; several studies that have examined Conners’ CPT indicate that differences in performance between patients with ADHD and those with other psychiatric disorders are either small or nonexistent (Advokat, Martino, Hill, & Gouvier, 2007; Solanto, Etefia, & Marks, 2004; Suhr, Sullivan, & Rodriguez, 2011). The Conners’ CPT is also one of the few neuropsychological tests to be labeled a “test of ADHD” because it has an expressed purpose of measuring cognitive impairments related to ADHD, and an indication of the probability of an ADHD diagnosis—an ADHD Index—is given in the result report. Another such “test of ADHD” is the QBTest Plus. It is a computer-based test that combines a CPT with the measurement of motor activity, with the aim of covering all three core areas (hyperactivity, impulsivity, and inattention) in ADHD. So far, only a few studies have examined how adults with ADHD perform on the QBTest Plus in relation to healthy controls and/or clinical controls. The studies that we have found show that adults with ADHD perform poorly compared with healthy controls on several parameters of the QBTest Plus (Edebol et al., 2012; Edebol, Helldin, & Norlander, 2013; Lis et al., 2010), and one study that considered levels of sensitivity and specificity reported relatively good discriminative ability (Edebol et al., 2012). In contrast, studies that have made comparisons with clinical controls show small differences between adults with ADHD and controls, and poor discriminative ability (Edebol et al., 2012; Söderström et al., 2014).

As previously mentioned, the review of national guidelines for assessment of ADHD indicated that there seems to be a consensus that the clinical interview is the “gold standard” of assessment of ADHD (Seixas et al., 2012). The clinical interview uses, and is supported by, various assessment instruments, such as assessment instruments for psychiatric disorders (i.e., the Structured Clinical Interview for DSM-IV-Axis I Disorders [SCID-I]) and a structured diagnostic interview for ADHD. To our knowledge, there are only two structured diagnostic instruments for adult ADHD that are based on Diagnostic and Statistical Manual of Mental Disorders(4th ed.; DSM-IV; American Psychiatric Association, 1994) criteria: the Conners’ Adult ADHD Diagnostic Interview for DSM-IV (CAADID; Epstein, Johnson, & Conners, 1999) and the Diagnostic Interview for ADHD in adults (DIVA 2.0; Kooij, 2012). The CAADID has been widely used in research (Ramos-Quiroga et al., 2012) and is one of the recommended assessment instruments in the National Institute for Health and Clinical Excellence (NICE) guideline for the assessment of ADHD in adults (Seixas et al., 2012). However, no official and commercial version of the CAADID is available in Swedish. The DIVA was first developed in Dutch by J. J. S. Kooij and M. H. Francken in 2007, and revised in October 2010 (DIVA 2.0). The DIVA 2.0 is now available in many languages, including Swedish. To our knowledge, it is frequently used in Sweden, and unlike CAADID, can be used for free of charge, but at the time of this article (March 2015), the DIVA 2.0 had not been validated (DIVA Foundation, 2015).

The term “gold standard” also implies that it is a diagnostic procedure that all other methods/instruments should be tested against. However, as most national guidelines for the assessment of ADHD emphasize the importance of collecting data from different sources and modalities, we think that the “gold standard” should instead be clinical expert consensus where all sources of information and clinical expertise and experience are considered, at least until there is substantial support in the research for a specific instrument or method.

The primary aim of the present study was to investigate the discriminative validity, in terms of sensitivity and specificity, of neuropsychological tests and diagnostic assessment instruments in diagnosing ADHD in an adult psychiatric clinical population with clinical expert consensus used as an external criterion. A secondary aim was to explore, through logistic regression analyses, how different combinations of neuropsychological tests and diagnostic assessment instruments could discriminate between patients diagnosed with ADHD and patients not diagnosed with ADHD.

Method

Participants

Participants were recruited from a consecutive series of patients referred from outpatient psychiatric clinics in the county of Västmanland, Sweden, for assessment of ADHD at the Neuropsychological Clinic in Västerås, Sweden, between January 2013 and June 2014. During this period, a total of 114 patients were assessed for ADHD. Inclusion criteria were a referral for assessment of ADHD, age 18 and older, and an informant, who knew the patient as a child (hereinafter referred to as collateral historian), available and willing to participate in a clinical interview regarding childhood symptoms. Exclusion criteria were treatment with medications (stimulants or nonstimulants) targeting ADHD, IQ score ≤ 70, and the presence of substance-related disorders. Full-scale IQ was estimated from the Wechsler Adult Intelligence Scale–4th edition (WAIS-IV) subtests of Block Design, Similarities, Matrix Reasoning, Vocabulary, and Coding using the Tellegen and Briggs procedure as described by Sattler and Ryan (2009). Six patients were excluded (five patients had an estimated full-scale IQ score ≤ 70, and one patient was excluded because of Klinefelter Syndrome), leaving a total of 108 patients included in the study. Of these, 60 (55.6%) patients met the criteria for an ADHD diagnosis. Of patients diagnosed with ADHD, 46 met the criteria for ADHD combined type (ADHD-C), 13 met the criteria for ADHD predominantly inattentive type (ADHD-IA), and one met the criteria for ADHD predominantly hyperactive/impulsive type (ADHD-HI). The three subtypes were combined to form the “ADHD group,” to ensure adequate statistical power for analyses (Cohen, 1992). Forty-eight (44.4%) patients who did not meet the criteria for an ADHD diagnosis constituted the “non-ADHD group.” Sample characteristics are shown in Table 1. The groups differed significantly in age, estimated full-scale IQ, self-rated ADHD symptoms (Adult ADHD Self-Report Scale [ASRS] v.1.1), self-rated depression (Beck Depression Inventory [BDI-II]) and anxiety (Beck Anxiety Inventory [BAI]), and the presence of other clinical psychiatric disorders (Axis I and/or Axis II). Both groups scored well above the normative mean on measures related to ADHD symptoms, depression, and anxiety. The normative mean value for the total score of the ASRS v.1.1. from the subsample of the U.S. National Comorbidity Survey Replication (NCS-R) is 17.19, and a score of 33 reaches the 95th percentile (National Comorbidity Survey, 2015). In our sample, both groups reached a mean value over the 95th percentile. The ADHD group on average scored in the range of mild depression and mild anxiety according to the BDI and BAI, and the non-ADHD group scored in the range of moderate depression and anxiety.

Table 1.

Sample Characteristics.

	ADHD	non-ADHD	F or χ²	p
n	60 (55.6)	48 (44.4)	—	—
Age (years)	28.18 (9.09)	32.75 (10.61)	5.801	.018
Male	32 (53.3)	25 (52.1)	0.017	.897
Education (years)	11.72 (1.85)	12.32 (1.60)	3.134	.080
Estimated IQ	91.52 (12.31)	98.96 (13.74)	8.787	.004
ASRS Screener	18.15 (3.12)	14.88 (3.50)	26.343	.001
ASRS Total Score	50.45 (9.48)	40.13 (9.11)	32.725	.001
BDI	17.25 (12.70)	23.83 (12.87)	7.084	.009
BAI	11.70 (10.29)	17.96 (11.98)	8.522	.004
COPM	2.79 (1.21)	3.22 (0.97)	3.746	.056
Employment			5.680	.128
Full-time work/studying	34 (56.7)	20 (41.7)
Part-time work/studying	9 (15.0)	11 (22.9)
Unemployment/vocational training	13 (21.7)	8 (16.7)
Long-term sick leave/disability pension	4 (6.7)	9 (18.8)
ADHD subtype			—	—
ADHD-C	46 (76.7)	—
ADHD-IA	13 (21.7)	—
ADHD-HI	1 (1.7)	—
Axis I diagnosis (one or more)	30 (50.0)	40 (83.3)	12.992	.001
Axis II diagnosis (one or more)	10 (16.7)	22 (45.8)	10.880	.001
Distribution of Axis I and II diagnoses
Mood disorder	15 (25.0)	21 (43.8)	4.219	.040
Anxiety disorder	26 (43.3)	33 (68.8)	6.950	.008
Other Axis I disorder	10 (16.7)	23 (47.9)	12.273	.001
Axis II Cluster A disorder	3 (5.0)	6 (12.5)	1.964	.161
Axis II Cluster B disorder	5 (8.3)	4 (8.3)	0.000	1.000
Axis II Cluster C disorder	6 (10.0)	15 (31.2)	7.688	.006

Note. ASRS = Adult ADHD Self-Report Scale (ASRS v.1.1); BDI = Beck Depression Inventory; BAI = Beck Anxiety Inventory; COPM = Canadian Occupational Performance Measure; ADHD-C = ADHD combined type; ADHD-IA = ADHD predominantly inattentive type; ADHD-HI = ADHD predominantly hyperactive/impulsive type. Values in the table represent the number of patients (%) except for age, years of education, estimated IQ, ASRS Screener, ASRS Total Score, BDI, BAI, and COPM, which represent means and standard deviations (SD). “—” indicates clear and obvious differences between groups.

Procedures and Measures

All patients underwent the same assessment protocol. They first met a clinical psychologist, who specialized in neuropsychology, who conducted a clinical interview including assessment of family situation, socioeconomic situation, physical and psychiatric status, medication, and family history concerning ADHD. The psychologist also interviewed all patients using the DIVA 2.0, and interviewed each patient’s collateral historian regarding childhood symptoms using the same diagnostic instrument. At the first appointment, the patients were provided with information about the study and data collection, and signed a document in which they agreed to the collected data being stored in a database and used for statistical processing. The study was approved by the Regional Ethical Review Board in Uppsala, Sweden. The patients were also provided with additional appointment times with the other members of the assessment team: a licensed psychologist who administered the neuropsychological tests, a licensed occupational therapist who performed a semistructured interview on the activities of daily living using the Canadian Occupational Performance Measure (COPM), and an MD, specializing in psychiatry, who made a general psychiatric assessment using the semistructured interviews SCID-I and SCID-II (Structured Clinical Interview for DSM-IV Axis II Personality Disorders). Finally, the assessment team discussed the results from the different parts of the assessment. All members in the assessment team were blind to each other’s assessment results prior to the consensus conference. When all the results were open to all members of the team, consensus on the clinical diagnosis was reached, and the patient was invited back for feedback and discussion, and then received a written referral of the diagnosis and treatment suggestions.

Diagnostic Interview

Diagnostic Interview for ADHD in adults (DIVA 2.0)

The DIVA 2.0 is based on the DSM-IV criteria for ADHD in adults (Kooij, 2012). The interview consists of three parts that each covers symptoms in childhood and adulthood: criteria for inattention, hyperactivity/impulsiveness, and age of onset plus areas of dysfunction due to ADHD. If six or more criteria are met for either inattention and/or hyperactivity/impulsivity in childhood and adulthood as reported by the patient and collateral historian, there is an indication that the requirements for a clinical diagnosis of ADHD may be met. In our study, the collateral historian (usually a parent) was asked to describe the patient’s behavior only in childhood, not as an adult. If there was lack of agreement between the patient and the collateral historian, we generally adhered to the rule of thumb recommended in the manual: to give priority to the statements of the patient. The interview was used to determine whether the patient fulfilled the criteria for ADHD, and if so, to which subgroup the patient belonged: combined type, predominantly inattentive type or predominantly hyperactive/impulsive type. If the subgroups differed with regard to childhood versus adulthood symptoms, the current subgroup in adulthood was accepted as the current diagnosis. Because of the small sample size, different subtypes of ADHD were collapsed into one diagnostic group for the statistical analyses.

Neuropsychological Assessment

The battery of neuropsychological tests used in the present study was selected based on a review of the research related to neuropsychological tests and adult ADHD, and where results had indicated that a particular test showed promising results in being able to discriminate between ADHD and healthy controls or between ADHD and other psychiatric samples (Balint et al., 2009; Boonstra et al., 2005; Edebol et al., 2013; Frazier et al., 2004; Hervey et al., 2004; Lis et al., 2010; Schoechlin & Engel, 2005). The majority of the tests chosen for inclusion are commonly used for neuropsychological assessment in clinical practice and particularly in the assessment of functions of attention (Rabin, Barr, & Burton, 2005).

Digit Span Backward, WAIS-IV

The Digit Span Backward is a working memory subtest where the patient is asked to repeat verbally presented digits that have no logical relationship to one another, in reversed order. It contains series ranging from two to eight digits. Apart from working memory, it provides a measure of attention, concentration, ability to self-monitor, and ability to use encoding strategies and rehearsal strategies (Sattler & Ryan, 2009).

Digit Symbol—Coding, WAIS-IV

The Digit Symbol—Coding test is a processing speed subtest that requires copying symbols with numbers, involving the discrimination between, and rote memory of, visual–number combinations. It assesses visual–motor coordination, scanning ability, visual short-term memory, attention, and concentration (Sattler & Ryan, 2009).

Rey Auditory Verbal Learning Test (RAVLT)

Patients are read a list of 15 unrelated words repeated over five different trials and are asked to repeat the words after each trial. Another list of 15 unrelated words is read, and the patient must again repeat the original list of 15 words, and then repeat them again after 30 min (Lezak, 2004). The RAVLT evaluates a wide diversity of functions: short-term auditory–verbal memory, rate of learning, learning strategies, retroactive and proactive interference, presence of confabulation or confusion in memory processes, retention of information, and differences between learning and retrieval.

Verbal Fluency, Delis Kaplan Executive System (D-KEFS)

The letter fluency condition, also known as the Controlled Oral Word Association Test (COWAT) or the phonemic test, requires patients to name as many words beginning with a single letter as they can in 1 min. Standard administration provides three letters: F, A, and S (Delis, Kaplan, Kramer, & Järvå, 2005). The test is sensitive to deficits in mental flexibility, inhibition, attention, and speed of processing.

Trail Making Test Part B (TMT B)

The Trail Making Test (TMT) is a test of divided attention. It is one of the most frequently used neuropsychological tests and is included in many test batteries (Reitan & Wolfson, 1985; Strauss, Sherman, & Spreen, 2006). The TMT provides information on visual search, scanning, speed of processing, mental flexibility, and executive functions. The TMT consists of two parts. TMT Part A requires an individual to draw lines sequentially connecting 25 encircled numbers distributed on a sheet of paper. Task requirements are similar for TMT Part B, except the person must alternate between numbers and letters (e.g., 1, A, 2, B, 3, C, etc.). The score on each part represents the amount of time in seconds required to complete the task. An analysis of the difference score (B − A) or the ratio between Part B and Part A (B / A) has been introduced to control for general speed of processing and is thought to be a more valid measure of the more complex divided attention required in Part B (Strauss et al., 2006; Tombaugh, 2004).

Paced Auditory Serial Addition Test (PASAT)

The PASAT is an auditory serial addition test for assessing working memory, sustained attention, and speed of information processing. The Gronwall, 61-item-per-trial, version was used in this study. Random numbers from 1 to 9 are presented at a rate of one digit per 2.4 s, and the patient is instructed to add each number to the one that immediately preceded it (Gronwall, 1977). In addition to the total number of correct responses, which is the most commonly used metric for PASAT performance, additional scores have been derived that are presumed to have higher sensitivity to impairment. One of these is the dyad score. A dyad is scored when two consecutive correct answers are given by the patient, in contrast to responding to every alternate item, which is called chunking. Because responding to alternate items significantly reduces the working memory requirements of the task, an inaccurate estimate of information processing speed is therefore obtained if only the number of correct responses is considered (Tombaugh, 2006).

The Quantified Behavior Test (QBTest Plus)

The QBTest Plus is a computer-based test that combines a CPT with the measurement of motor activity. The test duration is 20 min, and during this time, the patient is presented with 600 stimuli that differ in shape and color. Patients are instructed to press a handheld button when the stimulus is identical (both in shape and color) to the immediately preceding stimulus, defined as a target, and not to respond otherwise (nontarget). During the test period, the patient’s motor activity is recorded by means of an infrared camera following a reflective marker attached to a headband. The QBTest Plus gives many parameters; in this study, we chose a priori to examine the three cardinal variables (QBActivity, QBImpulsivity, and QBInattention) and the variables Omission errors, Commission errors, and Reaction Time Variance. The cutoff score, indicating an ADHD diagnosis, chosen for the included variables was 1.5 SD scores (Q-score = 1.5) above the age-related normative mean from the normative data integrated in the QBTest Plus software (Knagenhjelm & Ulberstad, 2010). According to the clinical documentation for the QBTest Plus, the normative database consists of 1,307 individuals between 6 and 60 years old with an even age and gender distribution, and the data have been gathered from several different cohorts (Qbtech, 2015).

Conners’ Continuous Performance Test II (CPT II)

The CPT II is a computerized test designed to measure attention problems and vigilance (Conners & Staff, 2000). Letters of the alphabet are presented on a screen one at a time, and the respondent is instructed to press the space bar on the keyboard or to click the mouse button for every letter except the letter X. Letters appear on the screen with different time intervals between each one, and the test takes approximately 14 min to complete. Different response patterns provide an indication of extent and cause of attention difficulties, impulsivity, or problems maintaining concentration. The test yields 12 scores, and in this study, we chose a priori to examine four scores: Omission errors, Commission errors, Hit Reaction Time Standard Error (SE), and Variability of SE. The normative data integrated in the software are based on a standardization sample of 1,920 individuals, with an age span of 6 to 55+ years, from two separate data collections, which constitutes the nonclinical sample (Strauss et al., 2006). Normative data are also obtained from two clinical samples: an ADHD group (n = 378) and a neurologically impaired group (n = 223). Data from the ADHD sample are used to derive an ADHD Index. The ADHD Index reflects how deviant an individual’s score pattern is from a nonclinical sample and whether it matches an ADHD sample. The ADHD Index was not used in this study. The cutoff score, indicating an ADHD diagnosis, chosen for the variables included in this study was 1.5 SD scores (T-score = 65) above the age-related normative mean from the normative group nonclinical cases.

Assessment of Occupational Performance

The Canadian Occupational Performance Measure (COPM)

The COPM was used to capture the patient’s perspective about his or her occupational performance. The COPM is a semistructured interview where the therapist asks the patient to identify and rate issues in areas of self-care, productivity, and leisure, using a scale from 1 to 10 (Carswell et al., 2004). Two scores, performance and satisfaction with performance, are obtained. In the present study, only the information regarding performance was collected as part of the overall information in the assessment of clinical diagnosis.

Self-Rating Instruments

Adult ADHD Self-Report Scale (ASRS v.1.1)

The ASRS is a self-report symptom checklist developed by the World Health Organization (WHO) that includes 18 questions of recent DSM-IV Criterion A symptoms of adult ADHD (Kessler et al., 2005). Patients are asked how often a symptom has occurred over the past 6 months on a scale ranging from 0 to 4 (never, rarely, sometimes, often, very often). A dichotomous six-question short-form screener (ASRS Screener), consisting of the first six questions of the form, has also been developed and validated. For the ASRS Screener, we used the dichotomized 0 to 24 scoring approach (0-13 vs. 14-26), as it has been shown to be the most robust (Kessler et al., 2007).

Beck Depression Inventory (BDI-II)

The BDI-II is a 21-question multiple-choice self-report inventory for measuring the severity of depression. Patients are asked to rate on a scale from 0 to 3 how they have been feeling for the past 2 weeks. Standard cutoffs are 0 to 9, minimal depression; 10 to 18, mild depression; 19 to 29, moderate depression; and 30 to 63, severe depression (Beck, Steer, Brown, & Lindfors, 2006).

Beck Anxiety Inventory (BAI)

The BAI is a 21-item multiple-choice self-report inventory that measures the severity of anxiety in adults and adolescents. It was developed to minimize the overlap between depression and anxiety states. Patients are asked how they have been feeling in the last week on a scale from 0 to 3, with a maximum total score of 63. Standard cutoffs are 0 to 7, minimal level of anxiety; 8 to 15, mild anxiety; 16 to 25, moderate anxiety; and 26 to 63, severe anxiety (Beck, Steer, & Järvå, 2012).

Psychiatric Assessment

The psychiatric assessment was made by an MD who specialized in psychiatry/neuropsychiatry. The assessment was performed using the SCID-I and SCID-II.

Statistical Analyses

Differences in measures of sample characteristics and neuropsychological functioning were analyzed by univariate general linear model (GLM) for continuous variables and by chi-square for categorical and nominal data. Group differences in neuropsychological measures were further analyzed by GLM, controlling for estimated IQ and age. Effect sizes were calculated using partial eta squared (η²). The clinical ADHD diagnosis obtained by clinical expert consensus was used as the external criterion for the calculation of the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUC) for each dependent variable that showed a significant group difference and where the effect size was moderate or stronger. To determine how well a combination of neuropsychological tests and diagnostic assessment instruments could classify the patients into the correct group, a set of binary logistic regression analyses was performed. All statistical analyses were conducted using IBM SPSS Version 20.

Results

Group Differences

Distributions and mean group differences of the neuropsychological tests are presented in Table 2. In measures that are based on normative data from nonclinical samples (QBTest measures, Q-score; CPT II measures, T-score), our data show that both the ADHD group and the non-ADHD group performed above the normative mean on all QBTest variables in the range of 0.84 to 2.17 SD for the ADHD group and 0.43 to 1.45 SD for the non-ADHD group. On the CPT II variables, only the ADHD group performed poorly in relation to the normative mean in the range of 58 to 60 T-scores. When comparing group means, the univariate GLM showed significant differences between the groups on the majority of variables, with the ADHD group performing more poorly than the non-ADHD group. The size of the effects was small to moderate except for the QBTest Omission errors and CPT II Commission errors, which were high. The ADHD group had a significantly lower estimated full-scale IQ than the non-ADHD group, and when controlling for IQ, the analyses showed significant differences for only the RAVLT VI, the PASAT Total Score, and the variables related to the two continuous performance tests, QBTest and CPT II. When controlling for both IQ and age, the number of significant variables was decreased further, leaving significant differences between the groups for the RAVLT VI, QBTest cardinal variable Inattention, QBTest cardinal variable Activity, QBTest Omission errors, QBTest Reaction time variance, CPT II Commission errors, CPT II Hit reaction time SE, and CPT II Variability of SE with small to moderate effect sizes.

Table 2.

Means and Standard Deviations for Neuropsychological Tests and Results of Univariate GLM, and GLM Controlling for IQ and IQ Plus Age.

	ADHD (n = 60)	non-ADHD (n = 48)	GLM	GLMIQ	GLMIQ + age
	M (SD)	M (SD)	F (η²)	F (η²)	F (η²)
Digit Span Backward from WAIS-IV (raw score)	6.05 (1.62)	6.52 (2.19)	1.645^a (.015)	0.005 (.001)	0.005 (.001)
Digit Symbol—Coding from WAIS-IV (scaled score)	8.40 (1.77)	9.21 (2.48)	3.894^a (.035)	0.589^a (.006)	0.212^a (.002)
RAVLT tot I-V (raw score)	49.65 (9.45)	54.44 (11.24)	5.780 (.052)*	2.036^b (.019)	3.398^b (.032)
RAVLT VI (raw score)	10.42 (2.83)	12.29 (2.78)	11.880 (.101)***	6.933^b (.062)**	9.040^b (.080)**
RAVLT VII (raw score)	10.57 (3.03)	11.92 (3.16)	5.099 (.046)*	2.002^b (.019)	3.233^b (.030)
Verbal Fluency–D-KEFS (raw score)	34.18 (9.24)	38.71 (9.70)	6.118 (.055)*	1.687 (.016)	1.192 (.011)
TMT B (seconds to complete)	72.52 (17.54)	73.38^c (25.45)	0.043 (.001)	2.195^b (.021)	2.801^b (.026)
TMT B / A (%)	236.83 (71.34)	244.51^c (70.57)	0.308 (.003)	0.057 (.001)	0.318 (.003)
TMT B − A (s)	39.43 (15.12)	42.47^c (20.89)	0.759 (.007)	2.387^b (.022)	3.608^b (.034)
PASAT tot correct (raw score)	31.32 (9.83)	37.64^c (11.17)	9.664 (.084)**	3.941 (.037)*	2.187 (.021)
PASAT tot dyads (raw score)	18.18 (11.36)	24.57^c (15.55)	6.030^a (.054)*	1.928^b (.018)	0.751^b (.007)
PASAT tot chunking (raw score)	11.75 (3.69)	11.19^c (6.22)	0.334^a (.003)	0.011^a,b (.001)	0.215^a,b (.002)
QBTest Ina (Q-score)	1.56 (0.85)	0.88 (1.17)	12.121^a (.103)***	9.807^a (.085)**	12.009^a (.104)***
QBTest Act (Q-score)	2.17 (1.03)	1.45 (1.29)	10.334 (.089)**	10.970 (.095)***	10.360 (.091)**
QBTest Imp (Q-score)	0.92 (1.31)	0.43 (1.04)	4.474^a (.040)*	2.958^a (.027)	2.491^a (.023)
QBTest Omi (Q-score)	1.87 (0.91)	1.03 (1.14)	18.255 (.147)***	10.905^a (.094)***	10.175^a (.089)**
QBTest Com (Q-score)	0.84 (1.29)	0.43 (0.94)	3.383^a (.031)	2.546^a (.024)	2.536^a (.024)
QBTest RT Var (Q-score)	1.32 (0.90)	0.68 (1.17)	10.414 (.089)***	9.920 (.086)***	10.922 (.095)***
CPT II Omi (T-score)	59.29 (21.83)	52.77 (17.73)	2.803 (.026)	1.448 (.014)	1.211 (.012)
CPT II Com (T-score)	60.30 (11.26)	50.78 (10.98)	19.472 (.155)***	13.818 (.116)***	11.371 (.099)***
CPT II Hit RT SE (T-score)	58.24 (10.08)	51.88 (12.84)	8.322 (.073)**	6.952 (.062)**	6.786 (.061)*
CPT II Var SE (T-score)	58.64 (11.17)	50.73 (11.92)	12.582 (.106)***	10.424 (.090)**	9.161 (.081)**

Note. GLM = General Linear Modeling; WAIS-IV = Wechsler Adult Intelligence Scale–4th edition; RAVLT = Rey Auditory Verbal Learning Test (tot I-V = total summed score trial 1-5; VI = Trial 6; VII = Trial 7); D-KEFS = Delis Kaplan Executive System; TMT B = Trail Making Test B; PASAT = Paced Auditory Serial Addition Test; QBTest = QBTest Plus (Ina = cardinal variable Inattention; Act = cardinal variable Activity; Imp = cardinal variable Impulsivity; Omi = Omission errors; RT Var = Reaction time variance); CPT II = Conners’ Continuous Performance Test II (Omi = Omission errors; Com = Commission errors; Hit RT SE = Hit reaction time standard error; Var = Variability of standard error); η² = partial eta squared effect size.

Analysis does not fulfill the assumption of equality of error variances.

Analysis does not fulfill the assumption of homogeneity of regression slopes.

Data are missing for one participant in the non-ADHD group.

p < .05. **p < .01. ***p < .001.

Sensitivity and Specificity

The discriminant ability of the neuropsychological tests that showed significant differences between the groups, with effect sizes that were at least moderate, was investigated by calculations of sensitivity and specificity. We chose not to include the variable RAVLT VI, despite the significant difference between the groups, because we did not consider it to be clinically relevant; the difference in raw score was small, and both groups performed within the normal level in relation to normative data. The diagnostic instruments DIVA and ASRS were also included in this analysis. Table 3 specifies the sensitivity, specificity, PPV, NPV, and AUC for each instrument at specified cutoff levels.

Table 3.

Results of Calculations of Sensitivity, Specificity, PPV, NPV, and AUC for Each Instrument in Relation to Clinical ADHD Diagnosis.

	ASRS Screener	DIVA	QBTest Act^a	QBTest Ina^a	QBTest Omi^a	QBTest RT Var^a	PASAT Tot^a	CPT II Com^a	CPT II Var^a
Sensitivity	91.7	90.0	76.7	58.3	73.3	43.3	33.3	33.3	26.7
Specificity	27.1	72.9	43.8	66.7	56.3	75.0	77.1	91.7	85.4
False positives	38.9	19.4	37.0	31.4	32.3	31.6	35.5	16.7	30.4
False negatives	27.8	14.6	40.0	43.9	37.2	48.6	51.9	47.6	51.8
PPV	61.1	80.6	63.0	68.6	67.7	68.4	64.5	83.3	69.6
NPV	72.2	85.4	60.0	56.1	62.8	51.4	48.1	52.4	48.2
Total classification accuracy	63.0	82.4	62.0	62.0	65.7	57.4	52.8	59.3	52.8
AUC	0.759	0.828	0.664	0.673	0.725	0.674	0.663	0.741	0.706

Note. All values in the table represent percentages except AUC. AUC computation was performed on the whole range of values for each instrument except for DIVA, where AUC was computed on the total number of symptom criteria from childhood and adulthood, and for both inattentive and hyperactive/impulsive symptoms. PPV = positive predictive value; NPV = negative predictive value; AUC = area under the receiver operating characteristic curve; ASRS Screener = Adult ADHD Self-Report Scale Screener, dichotomized as 0-13 (non-ADHD) and 14-24 (ADHD); DIVA = Diagnostic Interview for ADHD in Adults, dichotomized as ADHD if six or more symptom criteria in both adulthood and childhood, and in either or both of the domains Attention Deficit and Hyperactivity–Impulsivity, and as non-ADHD if fewer than six symptom criteria; QBTest = QBTest Plus (Ina = cardinal variable Inattention; Act = cardinal variable Activity; Omi = Omission errors; RT Var = Reaction Time Variance); PASAT Tot = Paced Auditory Serial Addition Test Total correct answers; CPT II = Conners’ Continuous Performance Test II (Com = Commission errors; Var = Variability of Standard Error).

Dichotomized as <1.5 SD normative mean (non-ADHD) and ≥1.5 SD normative mean (ADHD).

The DIVA showed the highest total classification accuracy, 82.4%, and also had the most even distribution between sensitivity and specificity, both of which were relatively high. In general, all other instruments showed poor total classification accuracy, where most of the instruments had relatively high specificity but low sensitivity. Exceptions were the QBTest cardinal variable Activity, QBTest Omission errors, and ASRS Screener, which had relatively high sensitivity but low specificity. The highest sensitivity was shown by the ASRS Screener, at 91.7%, and the highest specificity was shown by the CPT II Commissions, at 91.7%.

Logistic Regression Analyses

To investigate how well the diagnostic instruments and the neuropsychological tests could classify the patients into the correct group, a series of binary logistic regression analyses were performed. In the first analysis, only the neuropsychological tests that showed a significant group difference, with effect sizes that were at least moderate (QBTest cardinal variable Inattention, QBTest cardinal variable Activity, CPT II Commission errors, CPT II Variability of SE), were included in the analysis. We chose not to include the variables QBTest Omission errors and QBTest Reaction time variance even though they showed significant differences between the groups, as they are part of the cardinal variable QBTest Inattention and had strong correlations with that variable. This analysis was statistically significant: χ² = 34.430 (4, n = 108), p < .001, Nagelkerke’s R² = .365. The model correctly classified 74.1% of the patients, with sensitivity of 80.0 and specificity of 66.7. Only the variables CPT II Commission errors, QBTest cardinal variable Inattention, and QBTest cardinal variable Activity were significant predictors of clinical diagnosis.

In the second analysis, the neuropsychological tests together with the diagnostic instruments DIVA and ASRS Screener were included. This analysis was also statistically significant: χ² = 75.190 (6, n = 108), p < .001 with df = 6, Nagelkerke’s R² = .671. This model correctly classified 86.1% of the patients, with sensitivity of 90.0 and specificity of 81.3. However, only the variables DIVA and CPT II Commission errors were significant predictors of clinical diagnosis, with trends toward a significant effect in the QBTest cardinal variable Inattention and QBTest cardinal variable Activity.

A third analysis including only the four strongest predictors from the second analyses—DIVA, QBTest cardinal variable Activity, QBTest cardinal variable Inattention, and CPT II Commission errors—was also performed: χ² = 72.488 (4, n = 108), p < .001, Nagelkerke’s R² = .655. All four variables significantly contributed to the model, which correctly classified 87.0% of the patients, with sensitivity of 90.0 and specificity of 83.3. Table 4 shows regression coefficients, Wald statistics, odds ratios, and 95% confidence intervals for odds ratios for each of the four predictor variables.

Table 4.

Logistic Regression Analysis for Assessment Instruments Predicting Diagnosis of ADHD and Non-ADHD (n = 108).

Measure	B	SE	Wald test chi-square	Odds ratio	95% CI for odds ratio
Measure	B	SE	Wald test chi-square	Odds ratio	Lower	Upper
DIVA	3.470	0.674	26.479***	32.133	8.570	120.487
QBTest Ina	0.719	0.293	6.045*	2.053	1.157	3.642
QBTest Act	0.566	0.566	4.333*	1.761	1.034	3.000
CPT II Com	0.071	0.027	6.701**	1.074	1.017	1.133

Note. CI = confidence interval; DIVA = Diagnostic Interview for ADHD in Adults (dummy-variable coded 0 = non-ADHD and 1 = ADHD; QBTest Ina = QBTest Plus cardinal variable Inattention (Q-score); QBTest Act = QBTest Plus cardinal variable Activity (Q-score); CPT II Com = Conners’ Continuous Performance Test II Commission errors (T-score).

p < .05. **p < .01. ***p < .001.

Discussion

The DIVA 2.0 interview proved to be the best predictor of clinical diagnosis, much better than the neuropsychological tests: Digit Span Backward (from WAIS-IV), Digit Symbol—Coding (from WAIS-IV), RAVLT, Verbal Fluency–FAS (from D-KEFS), TMT B, PASAT, QBTest Plus, and CPT II. Why is that? Even if there were significant differences between the groups on several of the neuropsychological tests, with the ADHD group performing poorly in comparison with the non-ADHD group, overall, all tests showed a very poor ability to classify patients into the correct group. Total classification accuracy was 53% to 66%—hardly better than chance. The majority of the tests showed relatively good specificity but very low sensitivity with the exception of the variables QBTest cardinal variable Activity and QBTest Omission errors, which showed the opposite pattern. Furthermore, most of the neuropsychological tests appeared to be related to age and especially to levels of estimated full-scale IQ (e.g., the PASAT test), as many of the differences between the groups disappeared when controlling for these variables. The only tests found to be relatively insensitive to differences in IQ were the two continuous performance tests: QBTest Plus and CPT II.

That being said, in our opinion, it does not rule out the use of neuropsychological tests in the assessment of cognitive function in ADHD for several reasons. One reason is the importance of finding out the intellectual capacity of the patient. Another reason has to do with the potential diagnostic value in a specific test when the effects of age and an IQ level are known to the psychologist conducting the assessment. For instance, if a patient has an IQ level within normal limits or above and is under the age of 40 and performs poorly on the PASAT test, this is an indication of difficulties in information processing speed and working memory that could be related to ADHD. Yet another reason is that the logistic regression analysis model indicated that variables measuring inattention and activity from the QBTest and impulsivity from the CPT II contributed to increasing the specificity, to a greater degree excluding people who did not have ADHD, when used in combination with the DIVA.

Based on the diagnostic criteria for the different subgroups of ADHD, one might expect the cardinal variable Activity from the QBTest to be more relevant and sensitive to the combined and hyperactive/impulsive types of ADHD. To investigate this, we performed an additional post hoc analysis comparing this group (combined plus hyperactive/impulsive type) with predominantly inattentive type ADHD, and the non-ADHD group. One-way analysis of variance showed that there was a significant between-group effect, F(2, 107) = 11.588, p = .001. Pairwise post hoc Least Significant Difference (LSD) comparisons showed that there were significant differences between the ADHD combined plus hyperactive/impulsive group and both of the other groups. No difference was found between the ADHD predominantly inattentive group and the non-ADHD group. The effect size of the difference between the ADHD combined plus hyperactive/impulsive group and the non-ADHD group was large (η² = .16). This result gives an indication that analysis of the discriminative ability of test variables on subgroups of ADHD could give different results than the analysis of a single combined ADHD group. Moreover, the discriminative power for a test variable could be reduced when including a diagnostic subtype—for example, predominantly inattentive—that performs no worse than, in this case, clinical patients without ADHD.

The DIVA interview, with its high sensitivity and relatively high specificity, is a powerful tool, but it must be administered in the way in which it was intended. Based on our experience, we suggest that it must be conducted with patience and insight on the part of the interviewer. It is important that the behavior described by the patient or relative should constitute an established pattern and should be found in many situations and quite frequently; sometimes is not frequently enough. It is also important to stay as long as possible with open-ended questions rather than presenting the example questions (which the patient or the collateral historian can merely answer with a “yes” or “no”) too soon. The point that we are making is that it should be an interview, not a questionnaire or another ASRS. The ASRS, like the DIVA, has high sensitivity—many patients with ADHD get a high score. Unlike patients identified through the DIVA interview, many non-ADHD psychiatric patients also get a high score on the ASRS, so it is not useful as a diagnostic tool. The DIVA interview can be seen as a variant of the research interview where the content is construed in the interview and interaction between two, sometimes three, persons with focus on one of them, the patient, and his or her behavior and experiences now and as a child. The interview thus has the aim of covering both factual events and circumstances and their perceived meaning by the patient and the collateral historian. The interviewer (in this study, a psychologist), having recorded the answers, gave his understanding in a noninterpretive way of what had been said and, if needed, provided the examples in the questionnaire to react to. In a qualitative interview of this kind, it is important for the interviewer to adopt a position of conscious naivety; that is, to be alert for new and unexpected answers, to be inquisitive and tuned in to what is being said (and what is not), as well as being critical to his or her own assumptions and hypothesis during the interview (Kvale, Brinkmann, & Torhell, 2009). The interviewer also has to be open to the possibility that the patient and/or the collateral historian, for whatever reason, may adopt an exaggerated response style to convince the psychologist that he or she does or does not fulfill the criteria for ADHD. With practice in interviewing technique, this usually becomes evident, and the most efficient way to avoid a biased responding style is to reassure the patient or the collateral historian that there is no absolute “truth” in these matters, just different ways of looking at things, and that their voices will be heard and their experiences noted.

Furthermore, as the DIVA is based on the diagnostic criteria from the DSM-IV, six DSM-IV Criterion A symptoms are required to indicate an ADHD diagnosis for both childhood and adulthood symptoms. In this way, the DIVA interview carries with it some absolute criteria that the other assessment instruments do not. For instance, in our project, if it was revealed at the time of the clinical conference that the patient in question did not attain at least six diagnostic criteria for childhood symptoms on the DIVA interview, as judged by the patient or by the collateral historian, in most cases, no clinical diagnosis of ADHD was made. This was the case in all but two instances, in which there was strong clinical evidence of diagnosis, including the COPM assessment of poor functioning in everyday life, despite absent statements to that effect from the patient and the collateral historian. One weakness of the present study is the fact that there was no other information available about childhood symptoms besides the ones reported by the patient and the collateral historian through the DIVA. Because it is common for patients to lack memories of their behavior in childhood, we must be open to the possibility that the most reliable information may be attained from the collateral historian or other sources. However, there is growing evidence that ADHD has a genetic disposition at its roots (Faraone & Mick, 2010; Faraone et al., 2005; Li, Sham, Owen, & He, 2006; Sprich, Biederman, Crawford, Mundy, & Faraone, 2000), so it is possible that neither the patient nor the parent is able to give the full picture of early signs of dysfunction, and thus, a report from an independent source, such as a teacher, would have strengthened the reliability of the DIVA assessment.

Furthermore, the authors of the DIVA note in the last page of the questionnaire that research has indicated that four or more criteria in either the inattention or hyperactivity/impulsivity domains at adult age are sufficient for the diagnosis to be made. To investigate how well this holds true in our clinical sample, we completed additional explorative analyses of sensitivity and specificity when five or four criteria at adult age were used as cutoffs. These analyses showed that the sensitivity increased but the specificity decreased with fewer criteria: 93.3% sensitivity, 58.3% specificity, and 77.8% total classification accuracy for five criteria, and 93.3% sensitivity, 54.2% specificity, and 75.9% total classification accuracy for four criteria, respectively. Our results therefore indicate that six symptom criteria should be used as a cutoff for diagnosis when the DIVA is used in a psychiatric clinical setting with a mixed group of psychiatric patients.

Another weakness is that because of a lack of resources, no diagnostic review process was applied. It would have strengthened the reliability of the clinical diagnoses, and thus the validity of the analyses of the discriminative power of the instruments, if a second group of specialists had reviewed all the assessment data for interrater reliability. Under the given conditions, we tried to reduce bias in the assessment procedure by keeping all members in the assessment team blind to each other’s assessment results prior to the consensus conference. However, despite the blinded procedure and the focus on a consensus approach, it cannot be ruled out that any one of the instruments included in the study may have weighed more heavily in the diagnostic decision. It is not optimal when the instruments intended to be validated constitute the data set that the clinical consensus decision is based upon. A strength of the study is that we used a naturalistic psychiatric sample representative of patients seeking help or evaluation for symptoms of ADHD at psychiatric or ADHD specialty clinics, where the instruments investigated are frequently used, and where the clinician meets patients with a complex picture of symptoms, impairments, and difficulties.

The results from this study support the evidence from other studies that neuropsychological tests cannot stand alone in the process of diagnosing ADHD, even though some of them claim to be “tests of ADHD.” Nevertheless, some of them can provide important information in assessing the core variables in ADHD, especially of attention, activity, and impulsivity. The QBTest seems to have the best potential for assessing problems of sustained attention. Reasons for this could be that it lasts somewhat longer than the CPT II and that it is a bit more demanding in relation to information processing and working memory. The CPT II, however, seems to be more sensitive to impulsivity—probably because of the varying speed of presentation of the target and nontarget stimuli. The QBTest is the only test that measures activity level while the patient is engaged in the continuing performance task on the computer.

The question may be raised as to why tests of sustained attention seem to be most promising in the differential diagnosis of ADHD. This also seems to be the case in children. Tsal, Shalev, and Mevorach (2005) found that deficits in sustained attention were more pronounced than deficits in selective attention, executive attention, and orienting of attention in a group of 27 children with ADHD compared with a control group of children without ADHD. The problem of sustained attention typically arises when the patient is assigned boring, tedious, protracted, or repetitive tasks that lack intrinsic appeal and have no immediate payoff (Barkley & Murphy, 2006). Patients with these difficulties are often classified into the subtype of predominantly inattentive because they find it difficult to activate or arouse themselves to initiate work that must be done, often complain of being unable to stay alert, and frequently seem to be daydreaming or “in a fog.” Twenty-two percent of the patients in our sample were diagnosed with this subtype, and because almost all the rest of the patients with ADHD in our study were diagnosed in the combined category (with both hyperactivity/restlessness and inattention), it stands to reason that the part of the tests that measures aspects of attention stands out from the rest.

There are other long-standing tests of continuous performance in the assessment of behavior regulation and attention in children and adults that were not included in this study. One is the Integrated Visual and Auditory Continuous Performance Test (IVA + Plus) developed by Brain Train, Inc. (Sandford & Turner, 2004). Another widely used test is the Test of Variables of Attention (T.O.V.A.; Greenberg, 1988-2000). Although we have not found any evidence in the research that one CPT test is better than another, our results suggest that they may have dissimilar strengths and may be better or worse at detecting impairments in the different cognitive and motor functions in ADHD. With regard to future research, it would therefore be advantageous to investigate, for example, the four most commonly used CPT tests concerning their ability, separately or combined, to discriminate between individuals with ADHD and healthy and/or clinical controls—for example, in a crossover design trial. Another important issue for future research, based on our results, is to investigate the degree to which clinical experience and different interviewing techniques affect the results of the DIVA 2.0 semistructured diagnostic interview.

In summary, to our knowledge, this is the first report of a study investigating the discriminative validity of both neuropsychological tests and diagnostic assessment instruments and the combination of them in an adult psychiatric clinical population. This is also the first study to validate the DIVA 2.0. The results of this study support previous findings that the discriminative validity of neuropsychological tests, when comparing adults with ADHD with adults with other psychiatric disorders, is poor. Promising preliminary results for the DIVA were found, with relatively good discriminative ability. The results also indicate that variables measuring inattention, activity, and impulsivity from continuous performance tests can contribute to increasing the specificity if used in combination with the DIVA. Based on our results, our suggestion to the clinician regarding the assessment of ADHD is (a) to train his or her interviewing skills in this domain using a semistructured procedure such as the DIVA 2.0, (b) to get an estimate of IQ, (c) to assess aspects of attention/concentration and response inhibition, (d) to work in a team with input from different professionals, and (e) to make an effort to find alternative sources of information regarding the childhood history of the patient.

Footnotes

Acknowledgements

We thank Kerstin Edlund-Söderström, Olof Westin, Maria Bliisi, Andreas Karlsson, and Tuula Wallsten for their participation and invaluable contribution to the implementation of this study.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was financed by the ‘Sjukskrivningsmiljarden,’ an economic fund established by the Swedish government to encourage Swedish county councils to give higher health-care priority to sick leave and to develop processes and methods to reduce its frequency. In addition, research grants for KWN, as the principal investigator, have been received from: FAS/Forte, SRA, the Swedish Brain Foundation, the Uppsala and Orebro Regional Research Council, the Fredrik and Ingrid Thurings Foundation, the County Council of Vastmanland, the Konig-Soderstromska Foundation, the Swedish Psychiatric Foundation, and the Svenska Spel Research Foundation. None of these organizations had a role in the study design, data collection, data analysis, data interpretation, or writing of the report.

Author Biographies

Richard Pettersson, MSc. Psych., received his MSc in psychology from the University of Uppsala, Sweden, in 1998. He is a project director of a five-year study on the assessment and treatment methods of adult ADHD at the Neuropsychological Clinic, County Hospital, Västerås, Sweden.

Staffan Söderström, MSc. Psych., received his MSc in psychology from the University of Gothenburg, Sweden, in 1975. He is the project coordinator of a five-year study on assessment and treatment methods for adult ADHD at the Neuropsychological Clinic, County Hospital, Västerås, Sweden. He is also a licensed specialist in clinical neuropsychology and a clinical supervisor.

Kent W Nilsson, PhD, is a professor in psychiatry at the Centre for Clinical Research, Uppsala University, Sweden. He is the PI for a group of seven PhD students and four postdoctoral students investigating different aspects of mental health, alcohol, and illicit drug abuse as well as comorbidity, mainly from a gene–environment perspective.

References

Abramovitch

Dar

Hermesh

Schweiger

(2012). Comparative neuropsychology of adult obsessive-compulsive disorder and attention deficit/hyperactivity disorder: Implications for a novel executive overload model of OCD. Journal of Neuropsychology, 6, 161-191.

Advokat

Martino

Hill

B. D.

Gouvier

(2007). Continuous Performance Test (CPT) of college students with ADHD, psychiatric disorders, cognitive deficits, or no diagnosis. Journal of Attention Disorders, 10, 253-256.

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

Balint

Czobor

Komlosi

Meszaros

Simon

Bitter

(2009). Attention deficit hyperactivity disorder (ADHD): Gender- and age-related differences in neurocognition. Psychological Medicine, 39, 1337-1345.

Barkley

R. A.

Murphy

K. R.

(2006). Attention-deficit hyperactivity disorder: A clinical workbook. New York, NY: Guilford Press.

Beck

A. T.

Steer

R. A.

Brown

G. K.

Lindfors

(2006). BDI-II: Beck Depression Inventory: Manual, Svensk version. Stockholm, Sweden: Psykologiförlaget.

Beck

A. T.

Steer

R. A.

Järvå

(2012). BAI—Beck Anxiety Inventory: Manual, Svensk version. Stockholm: Pearson Sweden.

Boonstra

A. M.

Oosterlaan

Sergeant

J. A.

Buitelaar

J. K.

(2005). Executive functioning in adult ADHD: A meta-analytic review. Psychological Medicine, 35, 1097-1108.

Carswell

McColl

M. A.

Baptiste

Law

Polatajko

Pollock

(2004). The Canadian Occupational Performance Measure: A research and clinical literature review. Canadian Journal of Occupational Therapy, 71, 210-222.

10.

Cohen

(1992). A power primer. Psychological Bulletin, 112, 155-159.

11.

Conners

C. K.

Staff

M. S. H.

(2000). Conners’ Continuous Performance Test II: Computer Program for windows technical guide and software manual. North Tonawanda, NY: Multi-Health Systems.

12.

Delis

D. C.

Kaplan

Kramer

J. H.

Järvå

(2005). D-KEFS. Svenskt manualsupplement [Swedish Manual Supplement]. Stockholm, Sweden: Psykologiförlaget.

13.

DIVA Foundation. (2015). DIVA Foundation—Validation studies. Retrieved from http://www.divacenter.eu/DIVA.aspx?id=359

14.

Edebol

Helldin

Holmberg

Gustafsson

S.-A.

Norlander

(2011). In search for objective measures of hyperactivity, impulsivity and inattention in adult attention deficit hyperactivity disorder using the Quantified Behavior Test Plus. Europe’s Journal of Psychology, 7, 443-457.

15.

Edebol

Helldin

Norlander

(2012). Objective measures of behavior manifestations in adult ADHD and differentiation from participants with Bipolar II Disorder, Borderline personality disorder, participants with disconfirmed ADHD as well as normative participants. Clinical Practice & Epidemiology in Mental Health, 8, 134-143.

16.

Edebol

Helldin

Norlander

(2013). Measuring adult attention deficit hyperactivity disorder using the Quantified Behavior Test Plus. PsyCh Journal, 2, 48-62.

17.

Epstein

J. N.

Johnson

D. E.

Conners

C. K.

(1999). Conners’ Adult ADHD Diagnostic Interview for DSM-IV. North Tonawanda, NY: Multi-Health Systems.

18.

Epstein

J. N.

Johnson

D. E.

Varia

I. M.

Conners

C. K.

(2001). Neuropsychological assessment of response inhibition in adults with ADHD. Journal of Clinical and Experimental Neuropsychology, 23, 362-371.

19.

Faraone

S. V.

Mick

(2010). Molecular genetics of attention deficit hyperactivity disorder. The Psychiatric Clinics of North America, 33, 159-180.

20.

Faraone

S. V.

Perlis

R. H.

Doyle

A. E.

Smoller

J. W.

Goralnick

J. J.

Holmgren

M. A.

Sklar

(2005). Molecular genetics of attention-deficit/hyperactivity disorder. Biological Psychiatry, 57, 1313-1323.

21.

Fayyad

De Graaf

Kessler

Alonso

Angermeyer

Demyttenaere

. . . Jin

(2007). Cross-national prevalence and correlates of adult attention-deficit hyperactivity disorder. The British Journal of Psychiatry, 190, 402-409.

22.

Frazier

T. W.

Demaree

H. A.

Youngstrom

E. A.

(2004). Meta-analysis of intellectual and neuropsychological test performance in attention-deficit/hyperactivity disorder. Neuropsychology, 18, 543-555.

23.

Giacobini

Medin

Ahnemark

Russo

L. J.

Carlqvist

(2014). Prevalence, patient characteristics, and pharmacological treatment of children, adolescents, and adults diagnosed with ADHD in Sweden. Journal of Attention Disorders. Advance online publication 5 November 2014. doi:10.1177/1087054714554617

24.

Greenberg

L. M.

(1988-2000). T.O.V.A. Continuous Performance Test. Los Alamitos, CA: Universal Attention Disorders.

25.

Gronwall

D. M.

(1977). Paced auditory serial-addition task: A measure of recovery from concussion. Perceptual & Motor Skills, 44, 367-373.

26.

Hervey

A. S.

Epstein

J. N.

Curry

J. F.

(2004). Neuropsychology of adults with attention-deficit/hyperactivity disorder: A meta-analytic review. Neuropsychology, 18, 485-503.

27.

Holst

Thorell

L. B.

(2013). Neuropsychological functioning in adults with ADHD and adults with other psychiatric disorders: The issue of specificity. Journal of Attention Disorders. Advance online publication 17 October 2013. doi:10.1177/1087054713506264

28.

Katz

L. J.

Wood

D. S.

Goldstein

Auchenbach

R. C.

Geckle

(1998). The utility of neuropsychological tests in evaluation of Attention-Deficit/Hyperactivity Disorder (ADHD) versus depression in adults. Assessment, 5, 45-52.

29.

Kessler

R. C.

Adler

Ames

Demler

Faraone

Hiripi

. . . Walters

E. E.

(2005). The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population. Psychological Medicine, 35, 245-256.

30.

Kessler

R. C.

Adler

Barkley

Biederman

Conners

C. K.

Demler

. . . Zaslavsky

A. M.

(2006). The prevalence and correlates of adult ADHD in the United States: Results from the National Comorbidity Survey Replication. The American Journal of Psychiatry, 163, 716-723.

31.

Kessler

R. C.

Adler

L. A.

Gruber

M. J.

Sarawate

C. A.

Spencer

Van Brunt

D. L.

(2007). Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) Screener in a representative sample of health plan members. International Journal of Methods in Psychiatric Research, 16, 52-65.

32.

Knagenhjelm

Ulberstad

(2010). QbTest technical manual. Stockholm, Sweden: Qbtech AB.

33.

Kooij

J. J. S.

(2012). Adult ADHD: Diagnostic assessment and treatment (3rd ed.). London, England: Springer.

34.

Kvale

Brinkmann

Torhell

S.-E.

(2009). Den kvalitativa forskningsintervjun [The Qualitative Research Interview]. Lund, Sweden: Studentlitteratur.

35.

Lezak

M. D.

(2004). Neuropsychological assessment. Oxford, UK: Oxford University Press.

36.

Sham

P. C.

Owen

M. J.

(2006). Meta-analysis shows significant association between dopamine system genes and attention deficit hyperactivity disorder (ADHD). Human Molecular Genetics, 15, 2276-2284.

37.

Lis

Baer

Stein-en-Nosse

Gallhofer

Sammer

Kirsch

(2010). Objective measurement of motor activity during cognitive performance in adults with attention-deficit/hyperactivity disorder. Acta Psychiatrica Scandinavica, 122, 285-294.

38.

National Comorbidity Survey. (2015). ASRS distribution scale in the general population information. Retrieved from http://www.hcp.med.harvard.edu/ncs/ftpdir/adhd/asrs-distribution_scale.pdf

39.

Qbtech. (2015). Clinical documentation for QbTest in patients with ADHD. Retrieved from https://www.qbtech.com/assets/pdf/clinical_documentation_en.pdf

40.

Rabin

L. A.

Barr

W. B.

Burton

L. A.

(2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members. Archives of Clinical Neuropsychology, 20, 33-65.

41.

Ramos-Quiroga

J. A.

Bosch

Richarte

Valero

Gomez-Barros

Nogueira

. . . Casas

(2012). Criterion and concurrent validity of Conners Adult ADHD Diagnostic Interview for DSM-IV (CAADID) Spanish version. Revista de Psiquiatría y Salud Mental, 5, 229-235.

42.

Reitan

R. M.

Wolfson

(1985). The Halstead-Reitan Neuropsychological Test Battery: Theory and clinical interpretation. Tucson, AZ: Neuropsychology Press.

43.

Sandford

J. A.

Turner

(2004). Manual for the integrated visual and Auditory Continuous Performance Test. Richmond, VA: Brain Train.

44.

Sattler

J. M.

Ryan

J. J.

(2009). Assessment with the WAIS-IV. San Diego, CA: Jerome M. Sattler.

45.

Schoechlin

Engel

R. R.

(2005). Neuropsychological performance in adult attention-deficit hyperactivity disorder: Meta-analysis of empirical data. Archives of Clinical Neuropsychology, 20, 727-744.

46.

Seixas

Weiss

Muller

(2012). Systematic review of national and international guidelines on attention-deficit hyperactivity disorder. Journal of Psychopharmacology, 26, 753-765.

47.

Simon

Czobor

Balint

Meszaros

Bitter

(2009). Prevalence and correlates of adult attention-deficit hyperactivity disorder: Meta-analysis. The British Journal of Psychiatry, 194, 204-211.

48.

Söderström

Pettersson

Nilsson

K. W.

(2014). Quantitative and subjective behavioural aspects in the assessment of attention-deficit hyperactivity disorder (ADHD) in adults. Nordic Journal of Psychiatry, 68, 30-37.

49.

Solanto

M. V.

Etefia

Marks

D. J.

(2004). The utility of self-report measures and the continuous performance test in the diagnosis of ADHD in adults. CNS Spectrums, 9, 649-659.

50.

Sprich

Biederman

Crawford

M. H.

Mundy

Faraone

S. V.

(2000). Adoptive and biological families of children and adolescents with ADHD. Journal of the American Academy of Child & Adolescent Psychiatry, 39, 1432-1437.

51.

Strauss

Sherman

E. M. S.

Spreen

(2006). A compendium of neuropsychological tests: Administration, norms, and commentary. Oxford, UK: Oxford University Press.

52.

Suhr

J. A.

Sullivan

B. K.

Rodriguez

J. L.

(2011). The relationship of noncredible performance to continuous performance test scores in adults referred for attention-deficit/hyperactivity disorder evaluation. Archives of Clinical Neuropsychology, 26, 1-7.

53.

Tombaugh

T. N.

(2004). Trail Making Test A and B: Normative data stratified by age and education. Archives of Clinical Neuropsychology, 19, 203-214.

54.

Tombaugh

T. N.

(2006). A comprehensive review of the Paced Auditory Serial Addition Test (PASAT). Archives of Clinical Neuropsychology, 21, 53-76.

55.

Torralva

Gleichgerrcht

Torrente

Roca

Strejilevich

S. A.

Cetkovich

. . . Manes

(2011). Neuropsychological functioning in adult bipolar disorder and ADHD patients: A comparative study. Psychiatry Research, 186, 261-266.

56.

Tsal

Shalev

Mevorach

(2005). The diversity of attention deficits in ADHD: The prevalence of four cognitive factors in ADHD versus controls. Journal of Learning Disabilities, 38, 142-157.

57.

Walker

A. J.

Shores

E. A.

Trollor

J. N.

Lee

Sachdev

P. S.

(2000). Neuropsychological functioning of adults with attention deficit hyperactivity disorder. Journal of Clinical and Experimental Neuropsychology, 22, 115-124.