Validity and Utility of the Test of Creative Thinking Drawing Production for Dutch Adolescents

Creativity Definitions

For decades, researchers have been interested in the definition of creativity (Feldman, 2003; Kaufman & Beghetto, 2009; Plucker et al., 2004; Rhodes, 1961; Runco & Jaeger, 2012). Extensive efforts by Guilford (1950) and Torrance (1984) have stimulated and shaped this research, with a primary emphasis on divergent thinking (i.e., the capacity to generate diverse and numerous ideas as a fundamental principle of creativity). More recently, personality, cognitive, and sociocultural approaches to creativity research were explored (Brandau et al., 2007; Feldman et al., 1994; Glăveanu, 2013; Hennessey & Amabile, 2010; Kaufman & Beghetto, 2009; Selby et al., 2005; Sternberg, 2006, 2012; Sternberg & Lubart, 1999; Treffinger, 2009). Although it remains elusive what creativity exactly entails, there is a growing consensus that creativity comes from the confluence of multiple personal and environmental components (Glăveanu, 2013; Plucker et al., 2004; Sternberg, 2012). The theoretical foundation of the TCT-DP comes from Rogers’s (1962) theory of creativity in which creativity is defined as “the emergence in action of a novel relational product, growing out of the uniqueness of the individual on the one hand, and the materials, events, people, or circumstances of his life on the other” (p. 4). This definition is in alignment with the Plucker et al. (2004) definition of creativity as “the interaction among aptitude, process, and environment by which an individual or group produces a perceptible product that is both novel and useful as defined within a social context” (p. 90). Both definitions pinpoint that (a) creativity is the generation of a product (see also Rhodes, 1961; Sawyer, 2012); (b) creative products should be novel, useful, appropriate, and valuable, which match the Runco and Jaeger (2012) argument that creativity should be judged based on “originality” and “effectiveness” (p. 2; see also D. H. Cropley, 2015, 2017); and (c) creativity originates from complex interactions between individual and social variables (see also Glăveanu, 2013; Sternberg, 2005, 2012). Therefore, the TCT-DP can capture multiple aspects of creativity and creative potential.

Creativity and Academic Achievement

Academic achievement, such as school grades or grade point average (GPA), and creativity are distinct constructs. Academic achievement is a learning outcome typically measured by classroom grades and external achievement tests. Creativity encompasses thinking processes and strategies that are important for knowledge acquisition and learning. Thus, although distinct, they are related in that creativity plays an important role in learning. However, the relationship between these two constructs is complex. Empirical ambiguities regarding the association of creativity and academic achievement vary from r = –.07 (Vijetha & Jangaiah, 2010) to r = .66 (Yeh, 2004; Tan et al., 2013). A meta-analysis of 120 studies conducted by Gajda et al. (2017b) reported that there is significant evidence that creativity and academic achievement show a moderate correlation (r = .22). Also, different effect sizes are found for different types of creativity and academic achievement measures. When standardized achievement tests are used instead of GPA, or verbal creativity tests instead of self-report scales, correlations are stronger (Gajda et al., 2017b). Moreover, in recent studies conducted in the United States, the correlation between creativity and academic achievement is stronger in elementary (r = .54; Wallace & Russ, 2015) and middle school students (r = .35; Dai et al., 2012), than in college students (r = .06; Hunter & Cushenbery, 2015). Furthermore, according to Karwowski et al. (2020), this correlation varies by subject. Karwowski et al. (2020) reported that creativity is a strong predictor of math achievement when solving basic tasks but a constantly significant predictor of language tasks at any complexity level. This finding coincides slightly with Zhang et al. (2018), who found that the correlation between creativity and achievement is stronger for language tasks than math tasks. Zhang et al. (2018) hypothesized that the open-ended tasks in language classes could explain this difference. Furthermore, Kim (2008) asserted that the relationship between academic achievement and creativity is weaker for higher creativity scores, especially when creativity is related to underachievement.

Cultural differences may also play a role when investigating the relationship between creativity and academic achievement. For example, correlations between creativity and academic achievement tests range from r = .07 in Poland (Gralewski & Karwowski, 2012) to r = .66 in England (Tan et al., 2013). Freund and Holling (2008) executed a multilevel analysis of predictors of academic achievement in Germany. They found that a one-unit increase in GPA was associated with a .203 decrease in creativity as measured by the Berlin Structure of Intelligence Test for Youth: Assessment of Talent and Giftedness. Zhang et al. (2018) found that in upper primary students, the relationship between creativity (as measured by the Chinese version of Torrance Test of Creative Thinking [TTCT] Figural Form A) and self-reported school grades ranged between r = .07 and r = .21, indicating small associations. Thus, there is still a great deal of uncertainty regarding the relationship between creativity and academic achievement.

Creativity and Motivation

Under certain circumstances, such as autonomy-supportive environments that encourage risk-taking, creatively gifted students can develop a deep love of and devotion to domains that interest them (Prabhu et al., 2008; Runco, 2005). Considerable evidence has shown that a high level of motivation drives creativity (Adler & Chen, 2011; Amabile, 2018; Hennessey & Amabile, 2010). Intrinsic motivation is conducive to creativity (Amabile, 1996; De Jesus et al., 2013; Hennessey, 2003; Zhou & Shalley, 2010). This type of motivation refers to doing something because it is inherently interesting or enjoyable (Ryan & Deci, 2000). A strong desire to actively engage in optimal-challenging and risk-taking tasks contributes to self-actualized motivation and task-focused motivation, which leads to creativity (Albert, 1990; Amabile, 1996; Heinzen, 1989; Maslow, 1968; Perkins, 1988; Sternberg & Lubart, 1996). Flow experience or the “subjective experience of engaging in just-manageable challenges by tackling a series of goals, continuously processing feedback about progress, and adjusting action based on this feedback” (Nakamura & Csikszentmihalyi, 2014, p. 240) produces task enjoyment and personal interest, which are essential components of intrinsic motivation for creativity.

Extrinsic motivation refers to doing something because it leads to a separable outcome (Ryan & Deci, 2000). Although there is a growing body of research suggesting extrinsic motivation hinders creativity by distracting an individual’s attention (Amabile, 1983; Hennessey, 1989), the positive effects of extrinsic motivation on creativity should not be neglected (Grant & Berry, 2011; Hennessey & Amabile, 2010; Zhou & Hoever, 2014). Extrinsic motivators, such as informative feedback for creativity and recognition of creative accomplishment, contribute to creativity maintenance and growth (Collins & Amabile, 1999; Ryan & Deci, 2000). Furthermore, in a recent meta-analysis of 52 experimental studies and eight nonexperimental studies, Byron and Khazanchi (2012) found that creative-contingent rewards were positively related to increasing creativity, but this did not apply to performance-contingent or completion-contingent rewards. Thus, specific types of extrinsic motivators, such as creative-contingent rewards, could increase creativity.

Creativity and Inquisitiveness

Inquisitiveness refers to the motivation to gain new knowledge (Amabile, 1985; Gajda, 2016). An inquisitive person is characterized as “motivated to engage sincerely in good questioning” (Watson, 2015a, p. 279). Being inquisitive is essential to learning. Watson (2015b) asserted that children have a natural tendency to ask questions, making it easier to build intellectual inquisitiveness at an early age. When students ask self-regulated questions, they are self-motivated to find the answers and improve their epistemic standing (Chouinar et al., 2007; Watson, 2015b). Intellectual inquisitiveness contributes to enhanced motivation and advanced learning. Hence, many researchers agree that inquisitiveness is a distinct characteristic of creative individuals (Litman, 2005; Loewenstein, 1994; MacKinnon, 1978; Sternberg, 1985, 2006).

Although inquisitiveness has been incorporated in creativity measures, such as the Khatena–Torrance Creative Perception Inventory (Khatena & Torrance, 1976), there are mixed findings regarding its relationship with creativity. Some researchers have argued that inquisitiveness and curiosity lead to insights that evolve into creativity (Bull et al., 1995). Lucas (2016) suggested that inquisitiveness contributes to creative thinking because it has people questioning, investigating, and challenging ideas. Wilhelm (2014) argued that composing essential questions requires creativity and that these questions stimulate a person to solve problems creatively. However, Naderi et al. (2010) found a small but significant negative association between female students’ perception of their inquisitiveness and creativity (r = –.008). Additional research is necessary to explore further how creativity and inquisitiveness are related.

Creativity and Intelligence

Many researchers have investigated the relationship between intelligence and creativity. Some researchers (Batey & Furnham, 2006; Nusbaum & Silvia, 2011) found a moderate positive correlation between intelligence and creativity. Others (Simonton, 2009) reported that an extremely high level of intelligence might hinder creativity development. For example, Karwowski et al. (2016) conducted eight studies with participants from elementary, middle, and high schools (n = 12,255) using multiple measures of intelligence (Raven Matrices and Baddeley Grammatical Reasoning Test) and creativity tests (TCT-DP and Test of Creative Imagery Abilities). They found mixed evidence (i.e., correlations ranging from r = −.02 to r =.35) for the relationship between intelligence and creativity. Karwowski et al. (2016) also evaluated the creativity–intelligence relationship through necessary condition analysis (NCA; Dul, 2016) and found evidence (effect sizes ranging from d = .127 to d = .286) that intelligence played the role of being necessary but not sufficient in predicting creativity.

A large number of studies are devoted to the threshold hypothesis (Cho et al., 2010; Gralewski & Karwowski, 2012; Sligh et al., 2005). This hypothesis states that correlations between intelligence and creativity exist below the threshold IQ level of 120 but tends to disappear above this IQ threshold (Fuchs-Beauchamp et al., 1993; Getzels & Jackson, 1963; Yamamoto, 1964). Some research supports this hypothesis (Batey & Furnham, 2006; Carson, 2011; Jauk et al., 2013), whereas others contest it (Karwowski & Gralewski, 2013; Kim, 2005; Preckel et al., 2006). Jauk et al. (2013) conducted an empirical study with 297 adult participants and found that the threshold points varied according to the applied measures of creative potential. When a qualitative measure was used to assess ideational originality, there was a linear correlation between intelligence and creativity when IQ scores were below 104. However, when a quantitative measure was used for ideational fluency, this IQ threshold was around 85. Furthermore, a meta-analysis on the threshold hypothesis in 21 studies published between 1961 and 2004 yielded no statistically significant differences between the correlations at different levels of IQ (Kim, 2005). This finding was replicated by Batey and Furnham (2006) and Cho et al. (2010). Due to the methodological flaws in measuring intelligence, the relationship between intelligence and creativity is inconsistent and unclear.

Creativity Measures

Despite the prevalence of creativity in various definitions of giftedness, few gifted identification protocols incorporate measures of creativity (Luria et al., 2016). Yet, Luria et al. (2016) have argued that adding creativity measures to gifted identification protocols may address some potential bias in gifted identification. For example, creativity measures generally do not show the same ethnic differences as intelligence tests do (Kaufman, 2006; Kaufman et al., 2004). Furthermore, adding creativity to gifted identification protocols allows for a fuller representation of students’ cognitive abilities (Luria et al., 2016). Finally, researchers have found that creativity may negatively affect students’ performance on intelligence tests and standardized achievement tests (Luria et al., 2016). Thus, by adding creativity measures to gifted identification protocols, some of the bias would be balanced out that stem from using solely intelligence or achievement measures in these protocols.

There are a variety of creativity measures that intend to measure characteristics related to creative persons (Creative Personality Scale [CPS]; Gough, 1979), creative products (Consensual Assessment Technique [CAT]; Amabile, 1982), creative processes (TTCT; Torrance, 1966), and creative places or press (Creative Achievement Questionnaire [CAQ]; Carson et al., 2005). Measures of creativity have been criticized for a variety of reasons. For example, the TTCT–Verbal Form has been criticized for linguistic bias (Plucker, 1999). The Remote Associations Test (Mednick, 1962) is limited to measuring verbal creativity (A. J. Cropley, 2000), and researchers have found it difficult for test-takers whose first language is not English (Estrada et al., 1994). The CAT has been criticized for its costly nature due to the irreplaceable expert rating (Kaufman et al., 2008). Urban and Jellen (1996, 2010) created a creativity measure that could withstand these common critiques and called it the Test Zum Schöpferischen Denken – Zeichnerisch or the TCT-DP (Urban & Jellen, 1996, 2010).

Based on Rogers’s definition of creativity, Urban (1995) proposed an interactive components model of creativity to assess creativity, which included six interactive components: (a) divergent thinking and acting, (b) general knowledge base and thinking base, (c) specific knowledge base and area-specific skills, (d) focusing and task commitment, (e) motivation and motives, and (f) openness and tolerance of ambiguity. Urban suggested that these cognitive and personality components work together in the creative process. The TCT-DP is an image production test based on Urban and Jellen’s interactive components model of creativity and applies a holistic and gestalt-oriented approach to assessing creative potential (Urban, 2004).

The Present Study

TCT-DP is considered a promising instrument for creativity research. Researchers have confirmed that the TCT-DP’s results are reliable and valid across cultures (e.g., A. J. Cropley, 2000; Kalis et al., 2014; Rudowicz, 2004; Urban, 2004; Urban & Jellen, 2010). However, its divergent validity and utility within a more extensive test battery for high-ability identification require further investigation. Divergent validity helps establish construct validity (i.e., the degree to which a test measures what it claims to measure) by demonstrating that the construct measured by the TCT-DP can be distinguished from other constructs. Therefore, the present study focused on the utility and divergent validity of TCT-DP data (Urban & Jellen, 1996, 2010) when used within a high-ability identification protocol widely used in the Netherlands and includes measurements for academic achievement, school motivation, intelligence, and inquisitiveness. This high-ability identification protocol was designed to inform teachers about students’ academic strengths and weaknesses, intellectual potential, school motivation, and inquisitiveness. Schools then used that information to decide on appropriate interventions for their students. Highly creative students may not thrive in traditional classrooms that are not stimulating their creativity (Shavinina, 2009), which may influence their achievements and motivation, and well-being in school. Therefore, having information on students’ creativity can help teachers better understand students’ behavior and needs and appropriately intervene when necessary. Thus, because many definitions of giftedness include creativity, we wanted to explore the utility and divergent validity of the TCT-DP as a creativity measure in this identification protocol.

Our review of the literature indicated that there is conflicting evidence regarding three issues. First, researchers have reported inconsistent evidence about the correlations between creativity, academic achievement, intelligence, school motivation, and inquisitiveness. Second, prior research has been inconclusive regarding the predictive value of creativity for academic achievement, particularly when controlling for factors such as intelligence. Finally, the threshold hypothesis has been both accepted and rejected in various research studies. The following research questions were formulated to investigate the issues mentioned above:

Method

Results

Descriptive statistics for all variables are included in Table 1. Concerning our first hypothesis, we found small but statistically significant positive correlations between creativity and school motivation, inquisitiveness, and academic achievement (see Table 2). However, no significant correlation was found between creativity and intelligence.

OPEN IN VIEWER

Table 1.

Number (n), Mean (M), Standard Deviation (SD), and Range of Scores on All Variables.

Variable	n	M	SD	Range
Intelligence	710	93.83	14.94	60–139
School motivation	709	57.49	10.90	8–72
Inquisitiveness	709	22.89	5.80	5–39
Academic achievement	710	6.95	0.70	4.34–8.80
Creativity	710	22.78	8.95	4–52

OPEN IN VIEWER

Table 2.

The Pearson Correlation Coefficients Among All Variables (n = 726).

Variable	Intelligence	School motivation	Inquisitiveness	Academic achievement	Creativity
Intelligence	1
School motivation	−.01	1
Inquisitiveness	.033	.549**	1
Academic achievement	.284**	.260**	.128**	1
Creativity	.049	.173**	.083*	.094*	1

*

p < .05. **p < .01.

Second, a multiple regression model was used to investigate the predictive value of the TCT-DP on academic achievement when controlling for intelligence, school motivation, and inquisitiveness. Results are presented in Table 3. No assumptions underlying multiple regression (i.e., linearity, homoscedasticity, and multicollinearity; Field, 2013) were violated. The percentage of variance explained by the complete model was significant, R² = .15, F(4, 707) = 31.41, p < .000, with medium effect size, Cohen’s f² = .18 (Cohen, 1988). However, concerning the second research question, results showed that creativity (TCT-DP) was not a significant predictor for academic achievement when controlling for the other predictors, unstandardized B = .003, SE(B) = .003, standardized β = .034, p = .34, 95% confidence interval (CI) = [–.003, .008]. Because data were gathered from different cohorts of students between 2009 and 2015, we tested whether this conclusion could be drawn for each cohort separately. Results showed that this was the case: In each cohort, the TCT-DP was not a significant predictor for academic achievement when controlling for the other predictors.

OPEN IN VIEWER

Table 3.

Multiple Regression Model With Predictors of Academic Achievement.

Variable	B	SE B	β	p	95% CI
Intelligence	.013	.002	.286	<.001	[.010, .017]
Inquisitiveness	−.004	.005	−.036	.39	[–.014, .006]
School motivation	.018	.003	.276	<.001	[.012, .023]
Creativity	.003	.003	.034	.34	[–.003, .008]

Note. B = unstandardized regression coefficient; SE = standard error; β = standardized regression coefficient; CI = confidence interval.

Finally, the threshold theory was tested using regression analyses with intelligence as the dependent variable. Scores on creativity were first centered, reducing the problem of multicollinearity (Dalal & Zickar, 2012; Olvera et al., 2019), and then we entered the square of these scores into the model as the independent variable. This allowed us to test whether the relation between intelligence and creativity was curvilinear or not. Results showed that there was no significant curvilinear relationship, unstandardized B = .006, SE(B) = .007, p = .446. This indicated no differences in the correlations between creativity and intelligence at different levels of intelligence.

Discussion

We investigated the utility and validity of a creativity test, the TCT-DP, within a high-ability identification protocol used in the Netherlands. The identification protocol’s primary goal is to provide teachers with information about students’ academic, intellectual, and motivational needs. Based on our results, we concluded that the TCT-DP measures a different construct than intelligence, school motivation, and inquisitiveness, and therefore provides useful additional information on the students’ creativity that can help teachers understand their students better and intervene more appropriately. This conclusion coincides with previous studies that found small correlations among the concepts measured (Amabile, 1985; Gajda, 2016; Gralewski and Karwowski, 2012; Tan et al., 2013). Interestingly, we did not find a linear correlation between creativity and intelligence. Thus, as measured by the TCT-DP, creativity seems unrelated to intelligence as measured by a paper-and-pencil intelligence test such as the Intelligence Structure Test. The game-like setup of the test may explain this. Several researchers have concluded that scores on an intelligence test are independent of the scores on creativity tests when the creativity test is assessed in a game-like, nonevaluative context (A. J. Cropley & Maslany, 1969; Pankove & Kogan, 1968). However, Wallach and Kogan (1965) found this game-like nature necessary to measure “true” creativity. Finally, it should be noted that Runco and Albert (1986) found the relationship between creativity and intelligence to be a function of the measures used. Their results showed no significant relationship between intelligence and creativity when using IQ tests as a measure of intelligence.

Another factor that may have contributed to not finding a relationship between creativity and intelligence is the participants’ age. Kim (2005) pointed out that intelligence and creativity are less related in older groups such as secondary school students than younger groups from primary school. The educational influence on students’ use of cognitive abilities was less for younger students, leading to a stronger correlation between intelligence and creativity (Kim, 2005). Because the present study participants were all in middle school, age may have played a part in obtaining no significant correlation.

We also found that creativity did not significantly affect academic achievement when controlling for intelligence, school motivation, and inquisitiveness, even though small bivariate correlations between creativity and these other variables were found. This is in line with the expectation of a weakened effect (Gajda, 2016; Gralewski and Karwowski, 2012). Both Gajda (2016) and Gralewski & Karwowski (2012) found that intelligence and school motivation play a more significant role in explaining achievement than creativity. Therefore, we conclude that creativity did not seem to predict academic achievement when multiple control variables are taken into account. Researchers have found that the relationship between creativity and achievement may depend on classroom dynamics and teacher behaviors (Gajda et al., 2017a). For example, Gajda et al. (2017a) found that teachers who were more encouraging of creative behaviors had classrooms with a positive association between creativity and academic achievement. Therefore, it could be beneficial for teachers to have information on students’ creativity. It may change how they perceive students’ behavior and increase their encouragement of creative behavior, leading to a positive association between creativity and academic achievement.

We found no curvilinear relationship between intelligence and creativity, even though intelligence was measured with a test that especially discriminated in the high IQ regions. This result indicated no support for the threshold theory, which is in line with Kim’s (2005) and Batey and Furnham’s (2006) meta-analyses.

In conclusion, our findings provide support for the divergent validity of the TCT-DP in a Dutch sample. The TCT-DP indeed measures a different construct than intelligence, school motivation, inquisitiveness, and school achievement. Moreover, when using this creativity test within an identification protocol for high ability that already includes measures of intelligence, school motivation, and inquisitiveness, it has an added value in that it offers additional information regarding students’ creativity. Having information on students’ creativity may allow teachers to understand better and serve these students.