An Exploratory Study on the Junior Brixton Spatial Rule Attainment Test in 6- to 8-Year-Olds

Abstract

This study explored the Junior Brixton Test (JBT), an executive function (EF) measure for children, in comparison to the Wisconsin Card Sorting Test (WCST) in a sample of 6- to 8-year-olds, all attending the first 2 years of elementary school. Factor analyses indicated two main domains in both measures, namely concept formation and cognitive flexibility. However, within the cognitive flexibility domain of the JBT, perseveration scores reflected qualitatively different perseverative errors. More specifically, perseveration of previous rule and same stimulus scores loaded on the same subcomponent, whereas perseveration of same response loaded on another. The latter score was also negatively correlated both with a measure of general reasoning ability and a memory span task. The authors argue that the JBT is a promising tool to explore individual variations behind seemingly one type of executive function error, namely perseveration.

Keywords

executive function Junior Brixton Test Wisconsin Card Sorting Test (WCST)perseveration

Executive function (EF) refers to a range of cognitive processes necessary for problem solving in novel or demanding situations (Huizinga & van der Molen, 2007; Lezak, 1982; Miyake et al., 2000). It is conceptualized as comprising separate but interrelated components, including working memory (WM), shifting/flexibility, and inhibition (Huizinga & van der Molen, 2007; Miyake et al., 2000). This study particularly focused on the shifting/flexibility component of the construct as flexible adjustment to the changing demands of the environment is regarded as a prerequisite for efficient EF performance (Zelazo, Müller, Frye, & Marcovitch, 2003). Put in different terms, cognitive inflexibility (perseveration) is a hallmark of executive dysfunction (Possin et al., 2005; Robinson, Heaton, Lehman, & Stilson, 1980; Shallice et al., 2002). This is why many EF measures, including the Junior Brixton Test (JBT)—the child version of the Brixton Test, comprise the assessment of this component.

In the present study cognitive inflexibility is investigated via the JBT in a sample of early school-age children. This childhood period is specifically targeted because developmental studies on EF have mostly focused on preschoolers (Best & Miller, 2010; Best, Miller, & Jones, 2009). However, to better understand the developmental trajectory of EF components, more emphasis on the middle childhood period is essential. Best et al. (2009) proposed several reasons why focus of EF research should move beyond the preschool age toward the middle childhood. They stressed the fact that much of the development of EF occurs after age 5. Furthermore, starting school brings into the child’s life several novel experiences (such as, school adjustment and new social environment) and, hence, challenges the child’s executive skills that are still in the process of maturation. Examination of the developmental course of EF components throughout middle childhood may pave the way to a better understanding of this process.

Studies intended to delineate the developmental trajectory of EF in this age period entail the use of EF tasks constructed specifically for these children. The Wisconsin Card Sorting Test (WCST), the most frequently used EF measure, is applicable to individuals aged 5 to 89 years (Strauss, Sherman, & Spreen, 2006). However, its reliability with young children has been questioned. Bujoreanu and Willis (2008), for example, argued that the number category is too abstract for the developmental level of 6-year-olds. This is why most EF tasks use color or shape as relevant categories for preschoolers and older children (Cragg & Chevalier, 2012). These issues point to a need for new EF tasks specifically designed for early school-age children. In this regard, Shallice et al.’s study (2002) on the EF profile of children with ADHD merits special attention. This study provided adaptations of two EF measures to 7- to 12-year-old children, and one of them is the JBT. In Shallice et al.’s study, the JBT successfully differentiated ADHD and non-ADHD groups, appearing as a promising EF instrument for school-age children.

The WCST and the Brixton Test are both presented as quantitative measures of abstract thinking and mental set-shifting (Berg, 1948; Burgess & Shallice, 1996; Heaton, Chelune, Talley, Kay, & Curtis, 1993). Developmental studies on the internal structure of the WCST have confirmed this assertion, indicating that the test comprises these two domains (i.e., Lee, Riccio, & Hynd, 2004; Lin, Chen, Yang, Hsiao, & Tien, 2000; Somsen, van der Molen, Jennings, & van Beek, 2000; Yalcin & Karakas, 2007; Yeniceri & Altan-Atalay, 2011). As for the JBT, to our knowledge, there is yet no research on its internal composition. We believe the test merits further attention for several reasons. First, although it is conceptually similar to the WCST, both the nature of the stimuli and the absence of number concept competence make the JBT more appropriate for the developmental level of young children. Second, results obtained in the study by Shallice et al. (2002) have attested to the validity of the test. As such, it seems to be a good candidate to fulfill the need for new EF tasks suitable for school-age children, including those in the early years of schooling. These reasons have prompted the present research that aimed to explore the factorial structure of the JBT in a sample of 6- to 8-year-olds, all attending the first 2 years of elementary school. Indicated domains were compared with the WCST from which the JBT is conceptually derived.

This study further explored the association of EF tests (WCST and JBT) with WM and fluid intelligence (Gf) tasks. Backward digit span, a measure of WM, was expected to show a moderate degree of correlation with these tests as WM is strongly argued to be one of the major components of EF (Friedman et al., 2006; Miyake et al. 2000). A similar degree of association was also expected with an overall Gf measure since EF has been reported to be significantly related with Gf (Arffa, Lovell, Podell, & Goldberg, 1998; Bielak, Mansueti, Strauss, & Dixon, 2006; Friedman et al., 2006). For instance, Bielak et al. indicated that Gf significantly contributes to performance on the Brixton Test. Arffa et al., in a sample of 9- to 14-year-olds, reported that Full-Scale IQ significantly predicted children’s total, perseverative, and nonperseverative errors and trials to complete first category in the WCST.

Method

Participants

Data of 121 students, from three public (n = 79) and a private school (n = 42) in Istanbul, were used in this study. Four students from the public school sample were excluded from the data due to suspicion of neuropsychological disorders during administrations (i.e., difficulty in understanding instructions, possibility of language deficits, and developmental delay). Participants aged 72 to 107 months (M = 86.66, SD = 8.08). There are 56 boys and 65 girls, and among them 58 children are first and 63 are second graders. None of the parents reported any diagnosis of a psychological or neurological disorder of their children. Two participants’ missing Gf measure (CogAT®) scores were replaced with the mean value.

Instruments

The Junior Brixton Spatial Rule Attainment Test

The test consists of 45 cards. Each card has a 2 x 5 array of turtles (1 green and 9 gray turtles), and each position is numbered from 1 to 10. Cards differ from each other with respect to the position of the stimulus item (“green turtle”), and the participant’s object is to guess the position of the green turtle in the consecutive card. There are five blocks of trials in line with five different rules: The position of item increases by one, decreases by one, switches between Positions 5 and 10 (up/down), stays on the same place, and switches between Positions 4 and 10. The movement rule changes after each nine cards, and the participant is not informed about rules or the rule change.

The test provides five scores: plausible and guessing errors, and three perseveration errors. Plausible error refers to the incorrect nonperseverative responses that are spatially close to the green turtle (i.e., in the above, below, left, right, or cross of it), whereas guessing error is making spatially irrelevant incorrect responses. Finally, three perseveration scores are (a) perseveration of previous rule (PPR) that is the incorrect application of the rule of a previous trial; (b) perseveration of same stimulus (PSS), referring to repeatedly choosing the green turtle seen on the presented card although it is not correct; and (c) perseveration of same response (PSR) that is the repetition of participant’s own incorrect response consecutively.

The Wisconsin Card Sorting Test (WCST)

The computerized version (WCST-CV4) of the WCST-128, with the standard administration procedure (Heaton et al., 1993), was administered. The test includes three sorting categories presented twice (color, form, and number), and change occurs after 10 consecutive correct responses. The test provides 15 scores: trials administered (TA), total errors (TE), percent of total errors (%TE), total correct responses (TC), number of categories completed (CC), perseverative responses (PR), perseverative errors (PE), percent of perseverative errors (%PE), nonperseverative errors (NPE), percent of nonperseverative errors (%NPE), trials to complete first category (TC1st), number of conceptual level responses (CLR), percent of conceptual level responses (%CLR), failure to maintain set (FMS), and learning to learn (LL). Since some of these scores are redundant in the presence of others, following scores are chosen for the statistical analyses: PE, NPE, CLR, CC, T1stC, and FMS.

Backward Digit Span (BDS)

BDS subtest of the Wechsler Intelligence Test for Children-Revised (WISC-R) is a memory span test for 6- to 15-year-olds. Children are given a digit string and then asked to recall them backwards. Standard instructions, as provided by Savasir and Sahin (1995) for the Turkish version, were used in this study.

The Cognitive Abilities Test Form 5—Nonverbal Battery (CogAT®-NB)

CogAT® Form 5 is developed to obtain information about the development of cognitive skills of students from kindergarten to Grade 12 (Thorndike & Hagen, 1994). The test basically measures the general reasoning ability (g) in learning and problem solving. It consists of two batteries, which are developed for different grades: (a) primary battery (Level 1 and 2) for participants from kindergarten to Grade 3, and (2) multilevel battery (Levels A to H) for participants from Grade 4 to 12. Each battery consists of verbal, quantitative, and nonverbal categories. The nonverbal battery (Level 1), which measures general abstract reasoning abilities, is used in the present study. Alp and Diri (2003) examined the internal consistency of the nonverbal battery for Turkish students from kindergarten to Grade 1. They indicated the KR20 value for the entire test as 0.97. They also reported that CogAT® is a strong and reliable predictor of school achievement for Turkish students from kindergarten to Grade 1.

Procedure

A consent form, explaining the aim and procedure of the study, was sent to parents before administrations. Children were individually tested with the JBT and the WCST, given in a counterbalanced order, followed by the BDS. Assessments lasted approximately 40 min with each child. The nonverbal battery of CogAT® was administered to groups of 6 to 15 children in about 30 min. The standard instructions were given at the beginning of each test.

Results

The mean scores, standard deviations, and minimum and maximum values for test variables are presented in Table 1.

Table 1.

Descriptive Statistics: Means, Standard Deviations, Minimum and Maximum Values.

Test scores	M	SD	Min	Max
WCST-PE	30.73	15.33	6	91
WCST-NPE	37.91	20.99	3	91
WCST-CLR	39.28	24.63	0	89
WCST-CC	2.23	1.94	0	6
WCST-TC1st	58.61	46.84	10	129
WCST-FMS	1.04	1.25	0	7
JBT-PPR	7.96	1.25	0	7
JBT-PSS	2.00	1.76	0	10
JBT-PSR	1.43	1.62	0	8
JBT-PLA	5.00	3.34	0	18
JBT-GUE	4.50	5.07	0	26
COGAT®-NB	26.12	7.87	11	39
BDS	3.55	1.40	0	9

Note. WCST = Wisconsin Card Sorting Test; PE = Perseverative Errors; NPE = Nonperseverative Errors; CLR = Conceptual Level Responses; CC = Categories Completed; TC1st = Trials to Complete First Category; FMS = Failure to Maintain Set; JBT = Junior Brixton Test; PPR = Perseveration of Previous Rule; PSS = Perseveration of Same Stimulus; PSR = Perseveration of Same Response; PLA = Plausible Errors; GUE = Guessing Errors; CogAT®-NB = The Nonverbal Battery of the Cognitive Abilities Test; BDS = Backward Digit Span.

Factorial Structures of EF Tests

The factorial structures of the JBT and the WCST are explored and compared with respect to the extracted components in each test. Six scores of the WCST (PE, NPE, CLR, CC, TC1st, and FMS) were subjected to principal component analysis (PCA) with Varimax rotation, but observed extraction value for FMS score was too small (.178), indicating poor fit to the factor solution, so it was excluded from the analysis. Another PCA with Varimax rotation was conducted with the remaining scores. With eigenvalues greater than 1.0, a two-factor solution emerged, explaining 89.1% of the variance. The first factor, accounting for 63.7% of the variance, had loadings from NPE, CLR, CC, and TC1st scores, and the second factor, explaining 25.3% of the variance, consisted only of PE score. The first component reflects conceptual thinking and the second one taps cognitive (in)flexibility (see Table 2).

Table 2.

Factor Loadings of Executive Functions Measures.

Junior Brixton Spatial Rule Attainment Test				Wisconsin Card Sorting Test
Scores	Factor 1	Factor 2	Factor 3	Scores	Factor 1	Factor 2
Guessing errors	.902	.016	.009	Nonperseverative Errors	.818	−.525
Plausible errors	.798	−.069	.169	Conceptual level response	−.929	−.215
Perseveration of previous rule	−.429	.743	.323	Categories completed	−.926	−.209
Perseveration of same stimulus	.176	.873	−.298	Trials to complete 1st category	.840	.137
Perseveration of same response	.155	−.067	.936	Perseverative errors	.216	.923

Regarding the JBT, a PCA with Varimax rotation was carried out with its five scores: plausible, guessing, PPR, PSS, and PSR errors. Three factors, explaining 82.1% of the variance, emerged with eigenvalues greater than 1.0. The first factor, accounting for 33.78% of the variance, embraced plausible and guessing errors. This factor is referred to as difficulty in rule attainment in this study as it reflects conceptualization skills. The second factor, accounting for 26.4% of the variance, included PPR and PSS scores, and the third factor, explaining 21.9% of the variance, only had the PSR score (see Table 2). These two factors both tap the cognitive (in)flexibility domain, yet possibly due to the nonunitary nature of perseverative behaviors, emerged as two distinct components: (a) perseveration of an external or previously learned schema, and (b) perseveration of an internal schema.

Correlations Among EF, WM, and Gf

Regarding the relationship between EF and WM, correlation coefficients revealed that the BDS is significantly associated with the concept formation scores of EF tests: plausible and guessing errors in the JBT, and NPE, CLR, CC, and TC1st scores of the WCST. On the other hand, among the perseveration scores in EF tests, only the PSR score of the JBT indicated a significant negative correlation with the BDS (see Table 3). Following this finding, participants were grouped with respect to their performance on the BDS (M = 3.55, SD = 1.40), such that children whose scores were 1 standard deviation (SD) below the mean constituted the low-span group, whereas those who scored 1 SD above the mean constituted the high-spans (n = 26 in each group). Thus, the former group included those who could recall up to 2 digits in the reverse order, and the latter were the ones who achieved 5 or more digits. When these two groups were compared on the basis of their EF performances, one-way ANOVA indicated that the high-span group made fewer plausible errors (M = 3.88, SD = 2.98) and PSR (M = 2.07, SD = 2.17) on the JBT compared to low-spans (M_plausible = 6.19, SD_plausible = 3.39, and M_PSR =.92, SD_PSR =1.01), F(1, 51) = 6.778, p = .01, and F(1, 51) = 6.002, p = .01, respectively. On the WCST, high spans displayed fewer PE (M = 24.80, SD = 11.69) than low-spans (M = 31.26, SD = 11.07), F(1, 51) = 4.260, p = .04. They also performed better on the CLR and the CC (M_CLR = 48.96, SD_CLR = 25.61, and M_CC = 2.80, SD_CC = 2.05) than low spans (M_CLR = 31.88, SD_CLR = 19.37, and M_CC = 1.65, SD_CC = 1.83), F(1, 51) = 7.353, p = .009, and F(1, 51) = 4.557, p = .03, respectively.

Table 3.

Correlation Matrix for Test Scores of Wisconsin Card Sorting Test, Junior Brixton Spatial Rule Attainment Test, Cognitive Abilities Test, and Backward Digit Span.

Scores	1	2	3	4	5	6	7	8	9	10	11	12
1. WCST-PE	1
2. WCST-NPE	−.292**	1
3. WCST-CLR	−.438**	−.696**	1
4. WCST-CC	−.382**	−.626**	.867**	1
5. WCST-TC1st	.258**	.523**	−.695**	−.767**	1
6. JBT-PPR	.040	−.056	−.008	.019	.030	1
7. JBT-PSS	−.058	.018	.045	.082	−.084	.340**	1
8. JBT-PSR	.137	.083	−.173	−.159	.156	.065	−.210*	1
9. JBT-PLA	.165	.205*	−.310**	−.355**	.205*	−.219*	−.052	.161	1
10. JBT-GUE	.130	.168	−.235**	−.208*	.189*	−.355**	−.137	.165	.522**	1
11. CogAT®-NB	−.088	−.262**	.281**	.303**	−.195*	−.069	−.011	−.241**	−.133	−.172	1
12. BDS	−.162	−.209*	.315**	.260**	−.194*	−.009	.072	−.292**	−.217*	−.235**	.429**	1

Note. WCST = Wisconsin Card Sorting Test; PE = Perseverative Errors; NPE = Nonperseverative Errors; CLR = Conceptual Level Responses; CC = Categories Completed; TC1st = Trials to Complete First Category; JBT = Junior Brixton Test; PPR = Perseveration of Previous Rule; PSS = Perseveration of Same Stimulus; PSR = Perseveration of Same Response; PLA = Plausible Errors; GUE = Guessing Errors; CogAT®-NB = The Nonverbal Battery of the Cognitive Abilities Test; BDS = Backward Digit Span.

p < 0.05 (2-tailed).**p < 0.01 (2-tailed).

As for the association between EF and Gf, CogAT® was indicated to be significantly correlated with the PSR score of the JBT, and NPE, CLR, CC, and TC1st scores of the WCST (see Table 3).

Discussion

The present study first explored the internal structure of the JBT in comparison to the WCST and its relations with WM and Gf in 6- to 8-year-old Turkish children. Concept formation and cognitive (in)flexibility emerged as two main domains of both EF measures. However, on the JBT, cognitive (in)flexibility was represented by two distinct subdomains, reflecting qualitatively different perseveration errors.

The perseveration score of the WCST is traditionally used as a common denominator of EF deficit, namely cognitive inflexibility, that is observed in patients with frontal lesions (Heaton et al., 1993; Robinson et al., 1980) and also in typically developing young children (Chelune & Baer, 1986; Heaton et al., 1993; Huizinga & van der Molen, 2007; Paniak, Miller, Murphy, Patterson, & Keizer, 1996; Shallice et al., 2002; Somsen, 2007; Yeniceri & Altan-Atalay, 2011). This score is defined in the WCST as the perseveration of previously correct but no longer relevant category (Heaton et al., 1993). However, perseverative responses do not necessarily include a single type of behavior. Sandson and Albert (1984), for instance, discussed the variety of perseverative behaviors. They proposed a taxonomy for perseveration including three categories, which the authors argued to be distinct at both clinical and process levels, and quite possibly “at the (neuro)anatomical level as well” (Sandson & Albert, 1984, p. 717). Later, the study of Possin and colleagues (2005) examined the validity of this taxonomy and confirmed the diversity of perseverative behaviors. Recurrent perseveration, described as the repetition of a previously emitted response that “falls within the appropriate task framework” is found to be distinct from stuck-in-set and continuous perseverations, both of which were correlated with a large number of clinical measures (Possin et al., 2005, p. 954). The major difference between the JBT and the WCST regarding perseveration is that the JBT defines different perseverative behaviors, calculated in separate scores. In addition to the incorrect repetition of the previous rule/concept (PPR), two different perseveration types are identified in this test (G. M. Marzocchi, personal communication, June 2, 2008). For instance, when the child repeatedly chooses the green turtle on the presented card (although it is not correct); she or he is argued to be stuck in an external stimulus that leads to perseveration of same stimulus (PSS) responses. The child may also be captured by an internal schema that leads to perseveration of the same response (PSR), which is incorrect for both current and previous trials. The factorial structure of the JBT suggests that perseveration to an internal schema (PSR) is a qualitatively different kind of EF failure compared to the PPR and the PSS. The recurrence of this type of perseveration might represent a preoccupied cognitive style that operates solely on an internal template disregarding external cues. The distinct nature of PSR is further evidenced by its significant negative associations with participants’ memory span and reasoning: none of the perseveration scores of the two EF tests, but the PSR, is negatively correlated with the BDS and the nonverbal CogAT®, suggesting that increased frequency of this type of perseveration is associated with significant WM and Gf deficiencies. Therefore, we argue that considering perseveration as a single entity overshadows such nuances embedded in this score, when in fact it may reflect important variations.

Concept formation is the other indicated domain of both EF tests. This domain in the JBT is composed of plausible and guessing error scores, which refer to errors that are either spatially close to the green turtle or spatially irrelevant to it, respectively. Correlational analyses indicated that only the plausible errors of the JBT, not the guessing errors, was associated with the nonperseverative errors (NPE) score of the WCST. Such association is not surprising since both scores represent a trial-and-error approach in search of the test rule. The guessing response, on the other hand, reflects a different kind of EF failure which, as Shallice et al. (2002) argue, is “more characteristic of frontal errors” (p. 46). In Shallice et al.’s study, this score revealed significant main effects of both age and having ADHD. The present study explored this score in normally developing children and indicated its significant association with the level of conceptualization on the WCST.

Despite this indicated difference between plausible and guessing errors, they were both related with one’s level of conceptualization, number of categories achieved, and capacity of initial conceptualization, as measured by the WCST. Therefore, we may argue that the conceptual thinking factors of the two EF tests are related yet not identical. The converging and the diverging nature of these components were also confirmed by their associations with WM and Gf: The concept formation components of both EF tests were associated with WM, whereas only that of the WCST, but not of the JBT, was related to Gf. This divergence may be due to the fact that the JBT only provides error scores that aim to target specific difficulties in EF (i.e., perseveration, inefficient reasoning, or risk-taking), whereas the WCST provides more profound conclusions in understanding one’s both strengths and weaknesses (i.e., set maintenance). On the other hand, both EF tests seem to be significantly associated with WM, not only regarding their concept formation scores but also perseveration responses: children with high WM span displayed less conceptual and perseverative errors and higher conceptual thinking capacity. As such, current findings regarding EF and WM are in line with previous studies (i.e., Geary, Brown, & Samaranayake, 1991; Lemaire & Lecacheur, 2011).

Conclusions

The most striking finding of the present study, regarding the JBT, is that the test allows us to investigate individual variations behind perseverative errors. Such errors have been treated as a single score on the WCST. However, when the nature of errors leading to this score is examined, one can trace whether the child is distracted and captured by an external stimulus/rule or an internal schema. It needs to be underlined, however, that the present findings are limited to a sample of normally developing children. Future studies including clinical samples might shed more light on the diagnostic implications of this distinction. A second strength of the JBT is that its content is more child-friendly and its administration is easier. The test, on the other hand, lacks scores indicating certain strengths of children, such as their ability to maintain set. This may stand as a weakness since defining strengths is as important as capturing certain difficulties when assessing executive skills. Nevertheless, when this weakness is compensated by additional tasks, the JBT might serve as a good EF measure for young children, especially when perseverative behaviors are of concern.

Footnotes

Acknowledgements

We would like to thank Prof. Tim Shallice and Dr. Gian Marco Marzocchi for providing the instructions and the material of the Junior Brixton Spatial Rule Attainment Test. We are also grateful to all children and families who participated in this study.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

References

Alp

I. E.

Diri

(2003). Bilissel Yetenekler Testi’nin (CogAT®) ana sinifi ve birinci sinif ogrencileri icin kurultu gecerligi calismasi [A validation study of CogAT® for Turkish kindergarten and grade 1 students]. Turk Psikoloji Dergisi [Turkish Journal of Psychology], 18, 19-31.

Arffa

Lovell

Podell

Goldberg

(1998). Wisconsin Card Sorting Test performance in above average and superior school children: Relationship to intelligence and age. Archives of Clinical Neuropsychology, 13, 713-720.

Berg

E.A.

(1948). A simple objective technique for measuring flexibility of thinking. Journal of General Psychology, 39, 15-22.

Best

J. R.

Miller

P. H.

(2010). A developmental perspective on executive function. Child Development, 81, 1641-1660.

Best

J. R.

Miller

P. H.

Jones

L. L.

(2009). Executive functions after age 5: Changes and correlates. Developmental Review, 29, 180-200.

Bielak

A. A. M.

Mansueti

Strauss

Dixon

R. A.

(2006). Performance on the Hayling and Brixton tests in older adults: Norms and correlates. Archives of Clinical Neuropsychology, 21, 141-149.

Bujoreanu

I. S.

Willis

W. G.

(2008). Developmental and neuropsychological perspectives on the Wisconsin Card Sorting Test in children. Developmental Neuropsychology, 33, 584-600.

Burgess

P. W.

Shallice

(1996). Bizarre responses, rule detection and frontal lobe lesions. Cortex, 32, 241-259.

Chelune

G. J.

Baer

R. A.

(1986). Developmental norms for the Wisconsin Card Sorting Test. Journal of Clinical and Experimental Neuropsychology, 8, 219-228.

10.

Cragg

Chevalier

(2012). The processes underlying flexibility in childhood. The Quarterly Journal of Experimental Psychology, 65, 209-232.

11.

Friedman

N. P.

Miyake

Corley

R. P.

Young

S. E.

DeFries

J. C.

Hewitt

J. K.

(2006). Not all executive functions are related to intelligence. Psychological Science, 17, 172-179.

12.

Geary

D. C.

Brown

S. C.

Samaranayake

V. A.

(1991). Cognitive addition: A short longitudinal study of strategy choice and speed-of-processing differences in normal and mathematically disabled children. Developmental Psychology, 27, 787-797.

13.

Heaton

R. K.

Chelune

G. J.

Talley

J. L.

Kay

G. G.

Curtis

(1993). Wisconsin Card Sorting Test manual (revised and expanded). Odessa, FL: Psychological Assessment Resources.

14.

Huizinga

van der Molen

M. W.

(2007). Age-group differences in set-switching and set-maintenance on the Wisconsin Card Sorting Task. Developmental Neuropsychology, 31, 193-215.

15.

Lee

Riccio

C. A.

Hynd

G. W.

(2004). The role of executive functions in attention deficit hyperactivity disorder: Testing predictions from two models. Canadian Journal of School Psychology, 19, 167-189.

16.

Lemaire

Lecacheur

(2011). Age-related changes in children’s executive functions and strategy generation: A study in computational estimation. Cognitive Development, 26, 282-294.

17.

Lezak

M. D.

(1982). The problem of assessing executive functions. International Journal of Psychology, 17, 281-297.

18.

Lin

C. C. H.

Chen

W. J.

Yang

H. J.

Hsiao

C. K.

Tien

A. Y.

(2000). Performance on the Wisconsin Card Sorting Test among adolescents in Taiwan: Norms, factorial structure, and relation to schizotypy. Journal of Clinical and Experimental Neuropsychology, 22, 69-79.

19.

Miyake

Friedman

N. P.

Emerson

M. J.

Witzki

A. H.

Howerter

Wager

T. D.

(2000). The unity and diversity of executive functions and their contributions to complex “frontal lobe” tasks: A latent variable analysis. Cognitive Psychology, 41, 49-100.

20.

Paniak

Miller

H. B.

Murphy

Patterson

Keizer

(1996). Canadian developmental norms for 9- to 14-year-olds on the Wisconsin Card Sorting Test. Canadian Journal of Rehabilitation, 9, 233-237.

21.

Possin

K. L.

Filoteo

J. V.

Roesch

S. C.

Zizak

Rilling

L. M.

Davis

J. D.

(2005). Is a perseveration a perseveration? An evaluation of cognitive error types in patients with subcortical pathology. Journal of Clinical and Experimental Neuropsychology, 27, 953-966.

22.

Robinson

A. L.

Heaton

R. K.

Lehman

R. A. W.

Stilson

D. W.

(1980). The utility of the Wisconsin Card Sorting Test in detecting and localizing frontal lobe lesions. Journal of Consulting and Clinical Psychology, 48, 605-614.

23.

Sandson

Albert

M. L.

(1984). Varieties of perseveration. Neuropsychologia, 22, 715-732.

24.

Savasir

Sahin

(1995). Wechsler Cocuklar icin Zeka Olcegi (WISC-R) El Kitabi [Manual for the Wechsler Intelligence Scale for Children-Revised (WISC-R)]. Ankara, Turkey: Turk Psikologlar Dernegi Yayinlari [Turkish Psychological Association Publications].

25.

Shallice

Marzocchi

G. M.

Coser

Del Savio

Meuter

R. F.

Rumiati

R. I.

(2002). Executive function profile of children with attention deficit hyperactivity disorder. Developmental Neuropsychology, 21, 43-71.

26.

Somsen

R. J. M.

(2007). The development of attention regulation in the Wisconsin Card Sorting Task. Developmental Science, 10, 664-680.

27.

Somsen

R. J. M.

van der Molen

M. W.

Jennings

J. R.

van Beek

(2000). Wisconsin Card Sorting in adolescents: Analysis of performance, response times and heart rate. Acta Psychologica, 104, 227-257.

28.

Strauss

Sherman

E. M. S.

Spreen

(2006). A compendium of neuropsychological tests: Administration, norms, and commentary (3rd Ed.). New York, NY: Oxford University Press.

29.

Thorndike

R. L.

Hagen

E. P.

(1994). Cognitive Abilities Test (CogAT®) Form 5: Interpretive guide for school administrators. Itasca, IL: The Riverside Publishing.

30.

Yalcin

Karakas

(2007). Wisconsin Kart Esleme Testi performansinda gelisimin niceliksel ve niteliksel etkileri [Qualitative and quantitative effects of development in Wisconsin Card Sorting Test performance]. Cocuk ve Genclik Ruh Sagligi Dergisi [Turkish Journal of Child and Adolescent Mental Health], 14, 24-32.

31.

Yeniceri

Altan-Atalay

(2011). Age-related changes in the Wisconsin Card Sorting Test performances of 8- to 11-year-old Turkish children. The Clinical Neuropsychologist, 25, 1179-1192.

32.

Zelazo

P. D.

Müller

Frye

Marcovitch

(2003). The development of executive function: Cognitive complexity and control-revised. Monographs of the Society for Research in Child Development, 68, 93-119.