Abstract
Introduction
Executive functions (EFs) encompass the cognitive processes that underlie goal-directed behavior and are orchestrated by activity within the prefrontal cortex (PFC; for example, Olson & Luciana, 2008; Shimamura, 2000).
ADHD is a neurodevelopmental disorder that emerges during the early stages of development in which deficits at three different levels arise together: attention deficit, difficulty in inhibition (impulsivity), and disorder of motor activity control. These deficits are serious enough that they prohibit the normal performance of daily activities and compromise family and peer relationships (American Psychiatric Association [APA], 2000).
ADHD is frequently associated (50%-70% of patients) with other psychopathological disorders (Biederman, 2005). According to the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; DSM-5; APA, 2013), oppositional defiant disorder (ODD) co-occurs in 50% of children with the combined type of ADHD, whereas conduct disorder (CD) is observed to a lesser extent (25% with the combined type). The DSM-5 does not provide any statistics about the association between ADHD and learning disorders (LDs), but studies have determined association rates that range from 8% to 39% (Bloom, Miller, Garcia, & Hynd, 2005). Different neurocognitive models have been proposed to explain why so many children with ADHD present a comorbid disorder.
Historically, there has been some debate over the specificity of EF deficits in ADHD (Sergeant, Geurts, & Oosterlaan, 2002). However, one meta-analysis (Pennington & Ozonoff, 1996) indicated a clear deficit in the EF domain associated with ADHD, particularly for working memory, planning, and inhibition. These results were confirmed 10 years later by a second meta-analysis conducted by Willcutt, Doyle, Nigg, Faraone, and Pennington (2005).
Regarding comorbidity, De Jong, Oosterlaan, and Sergeant (2006) discussed several studies that test for a double dissociation between ADHD and reading disorder (RD), suggesting three hypotheses of comorbidity (phenocopy, cognitive subtype, and common etiology).
The phenocopy hypothesis proposes that one disorder may lead to a copy of the symptoms of another disorder (Pennington, Groisser, & Welsh, 1993). For example, children with RD could show hyperactivity or inattention at school because of frustration due to reading difficulty rather than because of neurocognitive problems resulting from ADHD (Marzocchi, Ornaghi, & Barboglio, 2009). However, the results of several neuropsychological double dissociation studies suggest that the phenocopy hypothesis regarding the nature of the comorbidity of ADHD and RD has little support.
The cognitive subtype hypothesis suggests that children with both ADHD and RD have a different or more severe form of ADHD or RD than children with either disorder alone (Rucklidge & Tannock, 2002). The cognitive subtype hypothesis was supported by Willcutt et al. (2001). In their study, the ADHD + RD group was more impaired on all measures than the group with ADHD-only, RD-only, and controls. Similar findings have been described by Seidman, Biederman, Monuteaux, Doyle, and Faraone (2001), who compared the performance of children with ADHD, LD (RD and dyscalculia), and both disorders on EF and non-EF tasks. The comorbid group performed poorest on all tasks, suggesting that comorbidity could worsen EF in children with ADHD.
Finally, some other studies provide evidence for the common etiology hypothesis, which suggests that ADHD and RD have common genetic origins. This hypothesis is supported when similar deficits are found in all the three groups. One common underlying deficit is thought to lead to different disorders (Willcutt, Pennington, Olson, Chhabildas, & Hulslander, 2005). Willcutt, Pennington and colleagues (2005) compared the performance of children with ADHD, RD, or ADHD + RD with healthy controls on tasks of language, reading, memory, EF, and speed of processing and described the different profiles characterizing the four groups. In particular, the ADHD + RD group showed a combination of the deficits observed in the ADHD group and in the RD group (reading disabilities and deficits in inhibition and information processing).
With respect to the comorbidity of ADHD and ODD/CD, it is not clear whether these disorders are two independent behavioral syndromes (Schachar, 1991) with different etiological correlates. In fact, ADHD seems to be due to cognitive deficits, and CD appears to be due to environmental deprivation (Schachar & Logan, 1990). According to Pennington and Ozonoff (1996), children with CD do not show an EF deficit but do show a deficit in impulse control. Clark, Prior, and Kinsella (2000) evaluated EF in four groups of adolescents, ADHD-only, CD-only, ADHD + CD, and healthy controls, using the Six Elements Test (SET) and the Hayling Sentence Completion Test. The ADHD-only group performed poorest on both tasks. The ADHD + CD group was impaired in problem solving and behavior monitoring, indicating that an EF deficit is associated with only ADHD and not CD. These findings were also supported by recent studies (Brocki, Nyberg, Thorell, & Bohlin, 2007; Oosterlaan, Scheres, & Sergeant, 2005; Wahlstedt, Thorell, & Bohlin, 2009). Thus, the study of EF could better define the cognitive functioning of children with ADHD and associated disorders, in particular with RD or with ODD.
The difficulty of studying EF in children concerns the limited number of specific tasks available to evaluate the executive domain with ecological validity (Burgess et al., 2006), which would allow researchers to formulate hypotheses regarding the daily functioning of ADHD children. Other studies have confirmed that classical EF tasks have low ecological validity when comparing children’s task performance with the results of Executive Function Questionnaires (EFQs) completed by their parents (Anderson, Anderson, Northam, Jacobs, & Mikiewicz, 2002; Bodnar, Prahme, Cutting, Denckla, & Mahone, 2007; Mahone et al., 2002; Vriezen & Pigott, 2002). These findings have been confirmed by a recent review conducted by Toplak, West, and Stanovich (2013) on the association between performance-based and rating measures of EF. The authors scrutinized 20 studies examining this association and reported that only 24% of correlational comparisons were statistically significant, suggesting that the two types of measures (performance-based vs. ratings) appear to capture different aspects of cognitive functioning.
A growing body of research on the nature and development of EF mechanisms has demonstrated that EF can be separated into distinct processes. Letho, Juujarvi, Kooistra, and Pulkkinen (2003) confirmed the model of EF proposed by Miyake et al. (2000) using a battery of EF tasks with a sample of children aged 8 to 13 years. According to these authors, EFs are composed of three interrelated domains: Inhibition, Working Memory, and Shifting. Wu and colleagues (2011) supported this tripartite model using a different battery of Attention and EF tasks with a sample of children aged 7 to 14 years.
The main limitation of such models is that they consider human goal-directed behaviors as separable into distinct cognitive processes, although the functioning of the mind is more likely to involve a sequential flow of cognitions and behaviors. Burgess, Veitch, Costello, and Shallice (2000) argued that an ecological model of EF could be derived from a multitasking test of consecutive cognitive processes: rule learning, planning, executing or following a plan, retrieving the performance and recalling the rules. Similarly, Zelazo and Frye (1998) proposed a model of EF comprising four sequential steps: representation of the problem, planning, executing, and evaluation of the behavior. Mackinlay, Charman, and Karmiloff-Smith (2006) designed a test of multitasking in children inspired by the Greenwich Test (Burgess et al., 2000) and the SET (Shallice & Burgess, 1991; Siklos & Kerns, 2004) to validate the EF model proposed by Burgess et al. (2000). Other attempts to devise more ecologically valid tasks to assess EF in patients with frontal lobe deficits have been made by Knight, Alderman, and Burgess (2002), who proposed a simpler version of the Multiple Errands Test (Schweiger & Marzocchi, 2008; Sgaramella, Bisiacchi, & Falchero, 1995; Shallice & Burgess, 1991).
The present study has different goals:
We aimed to test a model of EF in which the different components could be interpreted as sequential steps of a complex behavior aimed at achieving a goal because the tripartite models do not explain how inhibition, working memory, and shifting are related to each other.
We analyzed which component/s of EF could be impaired in children with ADHD, using an EF model in which cognitive processes are thought to be sequential and not fractionated.
Finally, we were interested to know whether children with ADHD and comorbid RD or ODD are more impaired than children with ADHD-only, using an EF framework.
Method
Participants
Children with ADHD were diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders (4th ed.; DSM-IV; APA, 1994) criteria using the Parent Interview for Child Symptoms, revised for DSM-IV (PICS-IV; Ickowicz et al., 2006) by a medical doctor specialized in child neuropsychiatry. The PICS-IV requires verification of the presence of at least six inattention and hyperactivity–impulsivity symptoms in at least two contexts (e.g., school and home). The PICS-IV also allows for the exclusion of other psychopathologies such as psychosis or pervasive developmental disorders.
To verify children’s impairment in several contexts, the Conners’ (1997) Rating Scale or the “Sindrome da Deficit di Attenzione” questionnaire (SDA, an Italian version of the ADHD-Rating Scale; Marzocchi & Cornoldi, 2000) was also presented to parents and teachers. The teacher and parent versions of the SDA questionnaire include 18 items based on the DSM-IV criteria for diagnosing ADHD (9 items concerning inattention and 9 concerning hyperactivity–impulsivity) and 8 items relevant to ODD. Marzocchi and Cornoldi (2000) provided a detailed description of the psychometric characteristics of the questionnaires. In short, the interrater reliabilities for inattention and hyperactivity–impulsivity were .80 and .74, respectively. The test–retest reliabilities were .83 and .81 for inattention and hyperactivity–impulsivity, respectively.
Reading skills were assessed using an Italian standardized test, the MT (Memory Training) Test of Text Reading (Cornoldi, Colpo, & Gruppo, 1998). The MT test presents text passages of increasing length according to grade level (from 120 syllables for first graders to 500 syllables to eighth graders) and evaluates reading fluency according to two parameters: reading speed (syllables/seconds) and reading accuracy (number of errors). The diagnostic criterion for RD was a score at least two standard deviations (SDs) below the mean of the children of the same age on the MT Test of Text Reading (for speed or accuracy).
Children in the ADHD-only group had an average score above the 90th percentile on the Inattention and Hyperactivity–Impulsivity subscales, an average score below the 70th percentile on the ODD subscale for both parent and teacher questionnaires, and a score no worse than 1 SD below the mean on the reading test. Children in the ADHD + ODD group had an average score above the 90th percentile on the Inattention and Hyperactivity–Impulsivity subscales, an average score above the 90th percentile on the ODD subscale for both the parent and teacher questionnaires and a score no worse than 1 SD below the mean on the reading test. Children in the ADHD + RD group had an average score above the 90th percentile on the Inattention and Hyperactivity–Impulsivity subscales, an average score below the 70th percentile on the ODD subscale for both the parent and teacher questionnaires and a score worse than 2 SDs below the mean on the reading test. Children in the control group were recruited from primary schools in the same areas as ADHD children. The ADHD groups and normal controls were matched by age. Typically developing children were excluded from the study if their parent or teacher stated that the child had ever received a clinical diagnosis (e.g., a behavioral problem or a learning disability), their full-scale IQ (FSIQ) estimate was below 80 as measured with the short version of the Wechsler Intelligence Scale for Children–III (WISC-III), or their score on one of the behavioral questionnaires of the parent or the teacher exceeded the 75th percentile.
Two subtests of the WISC-III (vocabulary, block design; Wechsler, 2006) were administered to all children. The estimated FSIQ was used to match the four groups. These WISC subtests have a correlation of .93 to .95 with the FSIQ (Groth-Marnat, 1997). Children with an IQ score below 80 were excluded from the study. All children in the clinical groups underwent a comprehensive neurological and medical examination. The exclusion criterion for these groups was the presence of CDs or mood or anxiety disorders. No child had a history of brain damage, epilepsy, psychosis, or a specific language disorder.
According to these criteria, 104 children aged 7 to 12 years were selected to participate in this study and were divided into four groups: 68 normal controls, 11 ADHD-only, 12 ADHD + ODD, and 13 ADHD + RD.
Materials
The EF battery consisted of four tasks and a behavioral questionnaire chosen to investigate several variables within the EF domain: Honk Test (derived from the Change Task by Oosterlaan & Sergeant, 1998), Daily Planning Test (derived by Sgaramella et al., 1995), Sentence Completion Test (derived by Shallice et al., 2002), Battersea Multitask Paradigm (Mackinlay et al., 2006), and Executive Function Questionnaire (EFQ) parent form (Schweiger & Marzocchi, 2008). All tasks were specifically adapted by the authors for the present study from previously established tasks. The EFQ parent form was an adapted and modified version of the Behavior Rating Inventory for Executive Functions (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000). Further details about task adaptation from the original version and scoring information are described below.
Honk Test
The Honk Test is a computerized task adapted from the Change Task paradigm that lasts approximately 15 min (Oosterlaan & Sergeant, 1998). In the present study, the type and number of the stimuli were changed from the original version, but the general paradigm remained the same as that reported by Oosterlaan and Sergeant (1998). This task is composed of three conditions (Go, Stop, and Change) and assesses three basic cognitive variables: vigilance, inhibition, and cognitive flexibility.
In three blocks, each with 160 consecutive trials, a visual stimulus (a red car) was shown on a computer screen for a maximum of 2 s, balanced for side of presentation (left or right). In the Go condition, children had to press a key according to the side where the stimulus appeared. The participants’ response time and accuracy were recorded. In the Stop condition, children had to respond as in the Go condition on 75% of the trials but inhibit their response and refrain from pressing any key when a horn sounded during the visual stimulus during the other 25% of trials. Finally, in the Change condition, children had to respond as in the Go condition on 75% of the trials but use a different response key when a horn sounded in the remaining trials.
In the Stop and Change conditions, the horn tones were presented 50, 200, 350, or 500 ms before the expected response of the child. In the Stop condition, the children had to inhibit their primary response. In the Change condition, the children had to press a different button with the non-dominant hand. The next trial began 1 s after the child’s response. The scoring of this task was fully computerized. The dependent variables of the Honk Test were as follows: (a) median reaction time (RT) for correct responses in the Go condition, (b) SD of RTs for correct responses in the Go condition, (c) number of errors due to lack of inhibition for the Stop Condition, (d) number of commission errors due to lack of inhibition, and (e) SD of the RTs for correct responses in the Change condition.
Daily Planning Test
The Daily Planning Test is a pencil-and-paper task designed to be completed in 20 min with the goal of evaluating the ability to plan daily activities in a simulated environment. This test is derived from the Multiple Errands Test (Shallice & Burgess, 1991) and used in frontal lesioned patients and older adults (Sgaramella et al., 1995). Participants have to carry out 10 errands in the correct order, using logical constraints (e.g., the errand “You should visit your grandmother—she asked you to buy bread” requires the participant to choose “buying bread” as the first errand) and choosing appropriate places to complete each errand (i.e., “at the bakery” for the errand “buying bread”). Moreover, children must draw the correct route on a map of an imaginary town provided by the examiner. The total score comprises the number of errands correctly reported in a logical sequence and the selection of appropriate places (maximum 10).
Sentence Completion Test
The Sentence Completion Test requires oral responses and investigates verbal inhibition, cognitive flexibility, and use of verbal strategies. The task is a modification of the Junior Hayling Sentence Completion Test (Shallice et al., 2002) and consists of 20 sentences in which the final word is missing. Children were presented with 10 sentences of each of two types (A and B). For Type A, children were asked to complete the sentence with a word that fits the phrase, earning a maximum score of 10. Following the procedure of the adult Hayling test (Burgess & Shallice, 1996) for Type B, children were asked to produce a word that made no sense at all in the context of the sentence. The children were told that the word had to be completely unrelated to the words in the sentence. The missing words were matched for frequency and age of acquisition. The sentences were presented alternately (Type A, Type B, Type A, etc.). The scoring of the Type B sentences is as follows:
0. if the child said a word unrelated to the sentence and used a strategy, for example, by naming objects in the room (us-type responses).
1. if the child said a word unrelated to the sentence without using a strategy (u-type responses).
2. if the child said a word related to the sentence or to the related answer (s-type responses).
3. if the child said a word that completed the sentence (c-type responses).
The total score for this task comprises the sum of scores for Type B sentences. Thus, a high score for Type B sentences indicates poorer performance. (The maximum value is 30.)
Battersea Multitask Paradigm
The Battersea Multitask Paradigm requires the child to plan and execute a number of tasks simultaneously to gain as many points as possible given a set of rules. This task is a modified version of a test set up by Mackinlay et al. (2006), inspired by the Greenwich Test (Burgess et al., 2000) and the SET (Shallice & Burgess, 1991; Siklos & Kerns, 2004). The paradigm consists of three independent tasks: bead sorting, counter sorting, and caterpillar sorting. Participants have to achieve the most effective strategy to perform these three tasks simultaneously and score the most points within a 6-min time limit (which is not long enough to complete all three tasks). The task performance is constrained by four simple rules, which are learned before the beginning of the test and used by the experimenter to compute the child’s score in different test stages. The Battersea Multitask Paradigm is divided into a practice game, where the experimenter explains the single tasks, and six stages, which represent the dependent variables of the test: (a) learning the four rules, with free and cued recall; (b) planning, where participants describe an executive plan; (c) performance of the three tasks in 6 min, a multifaceted dimension that relies upon executive strategy formation, cognitive flexibility, and mostly on inhibitory skills; (d) evaluation of coherence between planning and real performance; (e) performance retrieval; and (f) retrieval of rules learned at the beginning of the paradigm. For further details, the reader is referred to Mackinlay et al. (2006).
EFQ
The EFQ parent form is an adapted and modified version of the BRIEF (Gioia et al., 2000) that investigates children’s behavior in a familiar context, providing a heterogeneous profile of executive functioning. The modified version was created according to the following steps: (a) we selected the four most correlated items to the eight subscales (according to the Factor Analysis presented in the original BRIEF manual); (b) we transposed the original items of the BRIEF into a positive form (e.g., “the child is able to”) and translated them into Italian; and (c) a pilot study was carried out to check the quality and the clarity of the items.
The EFQ consists of 32 items describing children’s behavior in many circumstances, divided into eight subscales (4 items per scale): Inhibition, Shifting, Emotional Control, Initiative, Attention, Planning/Organization, Organization of Materials, and Self-Monitoring. Parents score each single item on a 5-point Likert-type scale (from “never” to “often”), and each subscale score is the sum of the single-item scores. A total score can be derived by adding together every subscale score. The maximum score (indicating excellent EF) is 20 for each scale.
For the parent version of the EFQ, Cronbach’s alpha was .92.
Procedure
The EF battery was administered to all participants individually in a single session. The total duration of the testing session varied from 50 to 75 min. The tasks were always administered in the same order: Honk Test, Daily Planning Test, Sentence Completion Test, and Battersea Multitask Paradigm. The parents completed the EFQ. The EF tasks were interesting enough to ensure the full motivation and compliance of all participants, and no oppositional behaviors were observed during task completion across groups.
Statistical analysis
First, a chi-square analysis was conducted to examine group differences in gender distribution, and univariate ANOVA was used to individually examine group differences in age and IQ. When significant differences arose, preliminary analyses (ANOVA) were conducted to examine the role of these factors in dependent measures.
Second, an exploratory factor analysis was conducted on the 13 neuropsychological measures of the EF battery to identify latent dimensions of neurocognitive functioning and to clarify the relations between tasks. Principal-axis extraction and Varimax rotation with Kaiser normalization were used on control group scores to extract factors with eigenvalues greater than 1. The standardization of the measures normalized the variance, and the kurtosis and skewedness values were less than 1.
Next, after checking that the assumptions (multivariate linearity, homogeneity of covariance matrices, independence of observations) were not violated, a MANOVA was carried out to compare the four groups of children (ADHD-only, ADHD + RD, ADHD + ODD, controls) on all factors extracted from the EF battery and all measures of EFQ. The alpha level was set to .05 for all data analyses. Effect sizes for MANOVAs are reported using partial eta squared (
Because this study was exploratory, a priori power analysis was not conducted; however, Cohen’s d effect size for contrasts between groups and 95% confidence intervals for d were calculated to appropriately indicate the statistical power for significant results.
Results
Gender was not balanced across groups, as there was 1 female in the ADHD + ODD group, 1 female in the ADHD + RD group, and 24 females in the normal control group, χ2(3) = 11.365; p = .010. Preliminary analysis using gender as a factor revealed that the difference between boys and girls was significant only on the SD of the RTs for the Honk Test’s Go condition, F(1, 100) = 4.551; p = .036;
Table 1 presents behavioral and cognitive characteristics of the children included in the study.
Participant Characteristics: Age, IQ, ADHD, and ODD Symptoms.
Note. IQ was obtained from the WISC-III (Wechsler, 2006). Inattention, Hyperactivity–impulsivity, and ODD are adjusted values obtained from CRS (Conners, 1997) or the SDA questionnaire (Marzocchi & Cornoldi, 2000). The adjustment was made to report raw values of CRS and SDA on a 0-3 scale (0 = minimum, 3 = maximum). Contrasts are significant at alpha = .05 (Tukey post hoc test). ODD = oppositional defiant disorder; RD = reading disorder; NC = normal controls; A = ADHD-only; A + O = ADHD + ODD; A + R = ADHD + RD; FSIQ = full-scale IQ; SDA = Sindrome da Deficit di Attenzione. WISC-III = Wechsler Intelligence Scale for Children–III; CRS = Conners’ Rating Scale. Contrasts in bold are significant at alpha = .05.
Data Reduction
The results from the exploratory factor analysis on neuropsychological measures revealed five factors (explaining 69% of the variance) with eigenvalues greater than 1; these factors were labeled as Speed of Processing, Inhibition, Planning, Execution, and Retrospective Memory. Measures of general processing speed, namely, the Honk Test Go condition RT, Honk Test Go condition SD of RTs, and Honk Test Change condition SD of RTs, loaded on Speed of Processing, with loadings ranging from 0.76 to 0.85. The number of errors on the Honk Test Stop condition and the number of errors on the Honk Test Change condition loaded on the Inhibition factor, with loadings of 0.82 and 0.80, respectively; additionally, the performance score of the Battersea Multitasking Paradigm loaded primarily on the Inhibition factor, with a loading of 0.46, most likely because this test requires the participant to avoid responding repeatedly to the same task to achieve the most effective performance. The Planning score and Coherence score on the Battersea Multitasking Paradigm loaded on the Planning factor, with loadings of 0.88 to 0.65, respectively. The total scores on the Daily Planning Test and Sentence Completion Test loaded on the Execution factor, with loadings of 0.68 to 0.78, respectively. All measures involved in retrospective memory demands of the Battersea Multitasking Paradigm, namely, the Rule Learning score, Performance Retrieval, and Rule Retrieval, loaded together the Retrospective Memory factor, with loadings ranging from 0.45 to 0.62. Table 2 shows the results of the exploratory factor analysis.
Principal-Axis Factor Analysis of Executive Function Tasks.
Note. A dash (—) indicates a factor loading of less than .20. Loadings in bold type indicate primary factor loadings.
A second exploratory factor analysis was carried out on the eight subscales of the parents’ EFQ. Two factors that explained 73.5% of the total variance were extracted. Five subscales (Initiative, Attention, Plan/Organize, Organization of Materials, and Self-Monitoring) loaded on Factor 1, labeled Metacognition; three subscales (Inhibition, Shifting, and Emotional Control) loaded on Factor 2, labeled Inhibitory Control. Table 3 presents saturation values of the eight subscales of the EFQ on the two factors.
Principal-Axis Factor Analysis of EFQ subscales.
Note. EFQ = Executive Function Questionnaire; RT = Reaction Time. Dash (—) indicates a factor loading less than .20.
Comparison Between Groups
Table 4 presents the unadjusted means of each group on the five factor scores.
Group Means and Standard Deviations for Executive Function Tasks.
Note. Contrasts in bold are significant at alpha = .05. ODD = oppositional defiant disorder; RD = reading disorder; NC = normal controls; A = ADHD-only; A + O = ADHD + ODD; A + R = ADHD + RD. CI = confidence interval.
The group effect was significant for the Speed of Processing factor score because children with ADHD + RD were slower than normal controls. For the Inhibition factor score, the difference between groups did not reach statistical significance. The group effect was significant for the Planning Factor score, with the normal control group outperforming the ADHD-only group. The group effect on the Execution Factor score was significant because all of the ADHD groups performed worse than the normal controls. The group effect on Retrospective Memory factor score was significant, as the ADHD + RD group performed worse than the normal controls.
Moreover, the average group scores on the eight EFQ subscales and two factors were compared. Children in the ADHD-only and ADHD + ODD groups were judged by their parents as having more Inhibitory Control difficulty than were normal controls (p < .001). Moreover, all groups with ADHD (ADHD-only, ADHD + ODD, and ADHD + RD) were rated by their parents as having lower metacognitive skills than their peers in the control group (p < .001). On seven of the eight subscales, all groups with ADHD were rated as more problematic than were controls (all ps < .001). Only for the Organization of Materials subscale did the post hoc analysis find no significant difference between groups. This null result was due to the lower scores of the normal controls on this subscale compared with the other scales (Table 5).
Group Means and Standard Deviations for the EFQ.
Note. Contrasts are significant at alpha = .05 (Tukey post hoc) in bold. EFQ = Executive Function Questionnaire; NC = normal controls; A = ADHD-only; ODD = oppositional defiant disorder; A + O = ADHD + ODD; A + R = ADHD + RD; RD = reading disorder.
Finally, to study the ecological validity of the new EF battery, a Pearson correlation analysis was performed between the EFQ and the five factor scores of the EF measures. The factor of the EF battery labeled Execution significantly correlated with almost all measures of the EFQ (only Organization of Materials did not correlate with Execution), with correlations ranging from .30, p = .01 to .46, p = .001. The Inhibitory Control factor of the EFQ significantly correlated with the Inhibition factor, r = −.29, p = .01, and the Execution factor, r = .36, p = .01, of the EF tasks, whereas the Metacognition factor of the EFQ significantly correlated with the Speed of Processing factor, r = −.22, p = .05, and the Execution factor, r = .33, p = .01, of the EF battery. Overall, 23 (46%) of a total of 50 correlational comparisons between EFQ measures and the five factors extracted by EF performance-based task were statistically significant (p < .05).
Discussion
The purpose of the present study was threefold. Our first aim was to test a model of EF in which the different components could be interpreted as sequential steps of a complex behavior to achieve a goal. Second, we aimed to analyze which components of EF could be impaired in children with ADHD. Finally, we intended to determine whether children with ADHD and comorbidity with ODD or with RD are more impaired than children with ADHD-only, using an EF framework. To achieve these goals, four groups of participants (ADHD-only, ADHD + ODD, ADHD + RD, and typically developing children) were presented with a battery of new EF tasks inspired by neuropsychological models (Burgess et al., 2000; Zelazo & Frye, 1998), and parents rated their everyday life EF skills using the EFQ. Five cognitive factors (Speed of Processing, Inhibition, Planning, Execution, and Retrospective Memory) were extracted from EF tasks via exploratory factor analysis. Similarly, two cognitive factors (Metacognition and Inhibitory Control) were extracted from the EFQ using factor analysis. Two distinct group comparisons were performed on the EF battery and on the questionnaire results to test the discriminative capacity of the new EF measures. Afterward, a Pearson correlation analysis was performed between the EFQ subscales and the five factor scores of the EF tasks to examine the ecological validity of the EF battery.
With regard to the first goal of our study—to test a model of EF comprised of a sequential flow of cognitive processes—the exploratory factor analysis conducted on the neuropsychological measures of the EF battery identified five latent dimensions of neurocognitive functioning that support a unified model of EF in children: Inhibition, Planning, Execution, and Memory of Performance, sustained by Speed of Processing. In fact, our data support a “sequential” EF model, wherein EF could represent consecutive cognitive processes (see, for example, Burgess et al., 2000; Zelazo & Frye, 1998) rather than other neurocognitive models that have proposed a division of EF into distinct and fractionated domains (see, for example, Letho et al., 2003; Wu et al., 2011). More specifically, the present results are consistent with the “ecological” EF model proposed by Burgess et al. (2000). According to this neurocognitive model, performing a goal-directed behavior requires a child to pass through sequential steps: controlling impulsive responses (Inhibition), learning the rules of the activity to be performed (Memory), establishing a working plan (Planning), performing the action scheme (Execution) and, finally, verifying the accomplishment of the goal of the behavior by recalling the rules of the task.
With regard to the other two major aims, the neurocognitive EF battery used in the present study has been shown to be more sensitive to between-group differences than the classic EF measures used in other studies investigating EF in children with ADHD with or without comorbidities (Barnett, Maruff, & Vance, 2009; Klorman et al., 1999; Qian, Shuai, Cao, Chan, & Wang, 2010). In fact, with respect to our examination of between-group differences, the present results yield evidence for the existence of a specific executive profile for each group of children with ADHD. Children with ADHD + RD demonstrated impairment on cognitive factors representing the Speed of Processing and Retrospective Memory factors, which is consistent with previous studies (Shanahan et al., 2006; Willcutt, Pennington, et al., 2005), and on the Executive factor. Children with ADHD + ODD showed a unique deficit in the Executive factor. This finding is consistent with Clark et al. (2000), who used the SET and Hayling Test. Thus, ADHD + ODD children could be described by parents as displaying severe executive dysfunction in everyday activities (e.g., reporting more externalizing symptoms in clinical interviews), even though such a dysfunction may not be evident during performance-based tests (Gioia & Isquith, 2004). Finally, ADHD-only children showed a specific deficit in the Planning factor and in the Executive factor. This finding is consistent with the work of Clark et al. mentioned above and with a previous Italian study (Marzocchi et al., 2008). The present study provides evidence for the existence of a common deficit across all groups with ADHD regarding the Executive factor, the most relevant component in the executive domain. It is worth remembering that the Executive factor was included in the total scores on the Daily Planning Task and the Sentence Completion Test. Moreover, these deficits in performance-based tests were confirmed by parental ratings on an EFQ.
Finally, with regard to the ecological validity of the neuropsychological tests of EF, the literature on the relationship between EFQs and classic EF tests yields conflicting findings. Some studies have reported no significant correlations between the BRIEF and classical performance-based EF tests (Anderson et al., 2002; Bodnar et al., 2007; Mahone et al., 2002; Vriezen & Pigott, 2002), whereas other studies have shown weak to moderate correlations between BRIEF and classical EF tests (McCandless & O’Laughlin, 2007; Toplak, Bucciarelli, Jain, & Tannock, 2009). Furthermore, a recent review by Toplak et al. (2013) on the correlations between performance-based versus ratings of EF reported that only 24% of correlational comparisons were statistically significant, suggesting that the two types of measures could assess different aspects of cognitive functioning. In the present work, however, moderate associations were obtained between performance on the EF measures and EFQ scores. The Inhibitory Control factor of the EFQ significantly correlated with the Inhibition and Executive factors of the EF tasks, and the Metacognitive factor of the EFQ significantly correlated with the Speed of Processing and Executive factors of the EF tasks. Moreover, 46% of the correlational comparisons between EFQ measures (parent ratings of EF) and the five factors extracted by the EF performance-based tasks reached significance in the present study. Our results therefore indicate a relative but significant overlap between EF performance-based tests and executive dysfunction in everyday activities as rated by parents, showing that the EF battery used in the present study demonstrates sufficient ecological validity.
In summary, the present study separates and describes different cognitive profiles of children with ADHD-only and those with ADHD and comorbid disorders using neuropsychological EF measures. These tests show good ecological validity, significantly correlating with parent ratings of EF in everyday activities. Moreover, our work yields meaningful information about EF in children aged 7 to 12 years (i.e., a system of consecutive cognitive processes), supporting the hypothesis that an integrated model of executive functioning, in which different components are interpreted as sequential processes, could be helpful in understanding ADHD and associated disorders.
From a neuroanatomical point of view, it is interesting to note that the components of EF extracted from the EFQ (Metacognition and Inhibitory Control) could reflect the different functional organization within the PFC in children with ADHD that has been described in the literature. Indeed, the Inhibitory Control, or “hot” affective aspect of EF (Zelazo & Müller, 2002), is reported to activate inferior frontal regions (Ardila, 2008; Fuster, 2002), while the Metacognitive, or “cool” cognitive aspects of EF (Zelazo & Müller, 2002), could involve the dorsolateral PFC bilaterally (Ardila, 2008; Fuster, 2002). Future neuropsychological studies of ADHD should document the association of the dorsolateral and inferior frontal cortices more thoroughly.
Limitations and Future Directions
Despite our promising results, some limitations of the present study should be considered. The main limitation is related to the small sample sizes of the clinical groups; the present results therefore require replication in a larger scale study. Moreover, an analysis of concurrent validity and construct validity between the new EF tests (Multitask, Daily Planning Task, Sentence Completion Test) and the classic EF tasks (Tower of London, Wisconsin Card Sorting Test, Trail Making Test, Verbal Fluency) could be useful for clarifying which cognitive dimensions are effectively evaluated by these new EF tests.
Finally, it should be noted that there was a disproportionate gender distribution between the clinical subgroups (one female in the ADHD + ODD group, one female in the ADHD + RD group) and the healthy control group (24 females). Although a Group × Gender effect was not observed for any neuropsychological measure in this study, consistent with previous findings not supporting any effect of gender on impairment or development of EF (for a review, see Rucklidge, 2010), future studies in which the samples are large enough to study gender differences by means of the present EF battery and questionnaire could be fruitful.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
