Assessing Classroom Climate in Special Education: A Validation Study of the Special Education Classroom Climate Inventory

Abstract

This study examined the internal structure, convergent validity, and reliability of the student self-report Special Education Classroom Climate Inventory (SECCI) in a sample of 325 students attending special education classes in six (semi) secure residential settings and in two youth prisons in the Netherlands. Both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) provided evidence of a theoretically based four-factor model—with Teacher Support, Positive Student Affiliation, Negative Student Interactions, and Unstructured Classroom Environment as dimensions—showing an adequate fit to the data, providing preliminary support for validity of the SECCI. Limited evidence for convergent validity was found in significant but small associations between classroom climate and academic self-concept. Ordinal Cronbach’s alpha reliability coefficients were good for all factors. The SECCI might be used to assess and subsequently target (problematic) classroom climate in schools for special education for students in (secure) residential youth care and youth prisons, although further development of the SECCI and replication of our study results seem necessary.

Keywords

classroom climate special education residential youth care justice-involved adolescents youth prison

Introduction

If children between the ages of 12 and 18 years cannot live at home or in foster care, mostly due to severe psychiatric and behavioral problems or criminal behavior, they are treated in (secure) residential youth care facilities or youth prisons where they attend special education classes (Carman, Dorta, Kon, Martin, & Zarrilli, 2004). It is assumed that a positive classroom climate can increase learning motivation of juveniles attending (special) education in residential youth care (Anderson, Hamilton, & Hattie, 2004) and improve their academic effort and competence (Maras, Demetre, Moon, & Tolmie, 2012).

Although various measures have been developed for the assessment of classroom climate (Altaf, 2015) and school climate (Ramelow, Currie, & Felder-Puig, 2015; Voight & Hanson, 2012) in regular education, a self-report questionnaire for measuring classroom climate for students in residential schools for special education has yet to be developed, in particular because mental disorders, trauma, and conduct problems in adolescents living in residential institutions (Colins et al., 2010; Van Dam, Nijhof, Scholte, & Veerman, 2010) may lead to different classroom dynamics compared with regular education. The present study therefore describes the development of the Special Education Classroom Climate Inventory (SECCI), which is a student self-report questionnaire designed to assess the climate in the classrooms of residential schools for special education, and examines the internal structure and convergent validity of the SECCI.

Classroom Climate and Its Assessment

Most research in (regular) education has focused on school climate instead of classroom climate (Clifford, Menon, Gangi, Condon, & Hornung, 2012; Steffgen, Recchia, & Viechtbauer, 2013; Thapa, Cohen, Guffey, & Higgins-D’Alessandro, 2013). Clifford et al. (2012) defined school climate as “the quality and the characteristics of school life” (p. 3). In their review, Thapa et al. (2013) defined the essential dimensions of school climate, including safety; teacher–student relationships and relationships among students, teaching, and learning; institutional environment (e.g., physical surrounding and resources); and the school improvement process (the results of school reform programs). These dimensions of school climate may be subsumed under the two main dimensions: Support and Structure (see Stockard & Mayberry, 1992).

Where school climate refers to the quality and character of school life, classroom climate refers to the quality of the students’ proximal social learning environment (McRobbie & Fraser, 1993), in particular with respect to teacher support and structure as well as group atmosphere among students, from the perspective of conditions that facilitate the learning motivation, academic achievement, and cognitive and social development of students (Altaf, 2015; Moos, 1979). In schools for special education, especially in secure residential institutions, it might not be the communication among teachers and students in the school that matters most but more specifically the communication among students (and their teacher) in the classroom (Anderson et al., 2004; Breeman et al., 2014; Weber, Somers, Day, & Baroni, 2016). Moreover, students in special education settings regularly participate in only one class, enter only one classroom, and encounter only one or very few teachers. Hence, the classroom climate is probably more salient compared with the school climate.

Often students in special education have problems not only with cognitive (i.e., executive) functioning but also with social-emotional functioning, affective functioning (i.e., trusting others), and developing an identity (Carman et al., 2004; Quinn, Rutherford, Leone, Osher, & Poirier, 2005). In special education classes, in particular those within residential settings, teachers provide rather individualized instruction, paying increased attention to socioemotional and identity-development of their students (Carman et al., 2004).

Already in 1949, Withall (1949) searched for a way to measure classroom climate, and suggested that meaningful learning of students can only occur in safe, nonthreatening situations, and that knowledge about the psychological atmosphere in the classroom is very important. Research in regular schools shows that the classroom climate is one of the most important factors influencing social-emotional behavior and learning motivation of students (Anderson et al., 2004; Wissink et al., 2014). The dimensions assessed in most research on classroom climate (for an overview, see Altaf, 2015) pertain to three broad domains of classroom experiences: (a) Interpersonal Relationships (involvement, affiliation, and support), (b) Goal Orientation (task orientation and competition), and (c) System Maintenance and Change (order and organization, rule clarity, teacher control, and innovation).

Only limited research has been conducted on classroom climate in special education classes and not, or hardly any, on education in (secure) residential settings and youth prisons for adolescents and young adults with severe behavioral problems. As far as we know, there is only one validated student self-report instrument for classroom climate in special education, namely, the revisited version of the Classroom Environment Scale (CES-SP; Trickett, Leone, Fink, & Braaten, 1993). However, the CES-SP proved to be marginally reliable, with only one scale showing satisfactory reliability (teacher support, Cronbach’s α = .70), but three scales yielding insufficient alphas below .60 (affiliation, task orientation, and teacher control).

Given the rather low reliabilities of the CES-SP and the particular context of special education classrooms in (secure) residential institutions and youth prisons, dealing with justice-involved adolescents showing high levels of psychopathology and often a mild intellectual disability, we decided to develop a brief and simple student self-report instrument assessing classroom climate based on the available literature: the SECCI.

Given the major importance of the interpersonal relationship dimension between teachers and students for educational and behavioral outcomes (Altaf, 2015; Carman et al., 2004; Jellesma, Zee, & Koomen, 2015; MacAulay, 1990; Moos, 1979; Roorda, Koomen, Spilt, & Oort, 2011), we developed two scales that aim to measure student interactions (Positive Student Affiliation and Negative Student Interactions), and one scale that aims to measure support by the teacher. Because lack of order and classroom disorganization are antagonistic to successful goal orientation and system maintenance (Altaf, 2015; MacAulay, 1990), and because students in special education classes find it difficult to concentrate in the classroom, follow the rules, and comply with teacher directives in an unstructured environment (Hocutt, 1996), the SECCI contains a scale that measures unstructured classroom environment.

The goal of the present study is to validate the SECCI for students receiving special education in (secure) residential settings and youth prisons. We used both exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) to examine the internal structure of the SECCI (Goodwin & Leech, 2003; Rios & Wells, 2014). To test convergent validity (Goodwin & Leech, 2003), we examined the associations between classroom climate and student’s academic self-concept in terms of perceived academic effort and academic competence (Maras et al., 2012). This is because in particular academic self-concept arises from social-environmental rather than personal factors and affects both academic achievement and classroom behavior (Guay, Marsh, & Boivin, 2003; Maras et al., 2012). Reliability is tested by computing ordinal Cronbach’s alphas for each subscale of the SECCI (Gadermann, Guhn, & Zumbo, 2012).

Method

Participants

A total of 325 pupils (63% boys, 37% girls; 13% placed under criminal law in youth prisons and 87% under civil law in secure settings) participated in the study; the mean age of respondents was 16.1 years (SD = 2.0, age range = 12-25 years). The study was performed in 2013 in the Netherlands in eight schools for special education in six secure residential settings and two youth prisons. Originally, the sample consisted of 465 students who were asked to fill in the SECCI questionnaire. The nonresponse rate was 30.1%, leaving a sample of 325 students. The main reasons for nonparticipation were acute psychotic problems, going to court that day, or lack of motivation.

Procedure

Schools were selected to represent a wide range of schools in Dutch secure special education to enhance generalizability of results. Some schools were located in the same building as the living groups and other schools were located in separate buildings. All students were asked to participate voluntarily and signed an informed consent declaration, which guaranteed anonymity. The questionnaires were given a number to guarantee anonymity of the participants. Respondents had 1 day available to fill in the questionnaire and handed in the questionnaire in a sealed blank envelope.

Questionnaires

SECCI

The questionnaire (Dutch language) was designed to be used with students showing low cognitive levels or a mild intellectual disability. In a test-trial in a Dutch youth prison in 2011, 26 pupils filled in the 40-item questionnaire and commented on the questionnaire. From this research, we then drew up a final list of 22 items with four scales rated on 3-point response format, ranging from 1 = I do not agree to 2 = I neither agree or disagree and 3 = I agree (see Table 1). Each item belongs to only one of the four scales for classroom climate.

Table 1.

Exploratory Factor Analysis of the SECCI: Standardized Loadings, Communalities (h2), and Uniqueness (u2).

Item		F1	F2	F3	F4	h2	u2
Item		Teacher Support	Negative Student Interactions	Positive Student Affiliation	Unstructured Classroom Environment	h2	u2
3	The teachers help us with problems	.78^a				.62	.38
6	The teachers are having time for us	.77				.58	.42
4	The teachers are helping me when I’m having difficulties with the lessons	.73				.55	.45
5	The teachers tell me when I’m doing fine	.70				.49	.51
2	The teachers are listening to us	.69				.55	.45
1	The teacher is kind	.63				.44	.56
7	The teachers often say “nice job”	.61				.40	.60
11	We are bullying each other in the classroom		.80			.65	.35
12	We are talking mean about each other in the classroom		.80			.63	.37
10	We call each other names in the classroom		.75			.56	.44
9	There is arguing in the classroom		.69			.57	.43
13	We are laughing at each other behind our back in the classroom		.66			.51	.49
8	Sometimes there will be beating and kicking in the classroom		.57			.32	.68
14	I like my classmates			.79		.64	.33
15	I have friends in my class			.65		.43	.57
16	In the classroom they like me			.64		.40	.60
17	My classmates are nice			.61		.40	.60
18	I like to work together with my classmates			.45		.23	.77
22	It is messy in the classroom				.71	.51	.49
21	I cannot pay attention in the classroom				.66	.66	.49
20	The class is never quiet				.58	.58	.35
19	Students make a lot of noise in the classroom				.57	.57	.44
	Eigenvalues	3.60	3.28	2.11	1.76
	Explained variance (%)	16	15	10	8

Note. Correlations between factors range from −.17 to .60. RMSEA = .057. SECCI = Special Education Classroom Climate Inventory; RMSEA = root mean square error of approximation.

Factor loadings less than .20 are not displayed.

The Teacher Support Scale (seven items) assesses the responsiveness of teachers to specific needs of the students. Paying attention to students, taking complaints seriously, respect, and trust are important characteristics of teacher support. An example of a support item is “The teachers are listening to us.” The Negative Student Interactions Scale (six items) assesses negative student interactions in the classroom. An example of a negative student interactions item is “We call each other names in the classroom.” The Positive Student Affiliation Scale (five items) assesses positive student affiliations in the classroom. An example of a positive student affiliation item is “I like to work together with my classmates.” The Unstructured Classroom Environment Scale (four items) assesses the degree to which the classroom environment lacks structure. An example of an unstructured classroom environment item is “The classroom is never quiet.” Higher scale scores on the SECCI scales represent more teacher support, more negative student interactions, more positive student affiliations, and a more unstructured classroom environment.

About Me Questionnaire: Academic Competence and Academic Effort

A simplified and shortened Dutch translation of the About Me Questionnaire was used (Beld, Van der Voort, Van der Helm, Kuiper, & Stams, 2017; Maras et al., 2012), assessing Academic Competence (i.e., how capable the student thinks he is in the area of academic skills) and Academic Effort (i.e., whether the student thinks he puts effort in school and whether he is willing to learn), using a 4-point response format: 1 = completely disagree, 2 = disagree more than agree, 3 = agree more than disagree, 4 = completely agree. An example of an academic competence item was “I get good grades at school”; an example of an academic effort item was “I work hard at school.” Cronbach’s alphas were sufficient for both scales, with .84 and .87 for Academic Competence (four items) and Academic Effort (four items), respectively. Higher scale scores represent more academic competence and effort.

Analyses

To examine the internal structure of the SECCI, we first conducted an EFA using principal axis analysis in R (Revelle, 2016) with oblique (promax) rotation because we expected that the factors would be correlated. Subsequently, we examined both the bifactor (nested or general-specific) model (Wolff & Preising, 2005), using the Schmid–Leiman algorithm (Schmid & Leiman, 1957), and hierarchical (indirect) model (Revelle, 2016) to investigate the likelihood of a general factor, or more precisely, to examine whether several empirically and/or theoretically derived subdimensions of classroom climate (i.e., specific factors), specified each by a subset of indicators, account for unique variance above and beyond the variance accounted for by a general factor specified by all indicators (see Rios & Wells, 2014).

Finally, we analyzed the factor solution derived from the EFA by means of CFA in Mplus (Muthén & Muthén, 1998-2006). A multifactor model was specified—using the maximum likelihood estimator with robust standard errors (MLR) that accounts for violations of normality—in which each item loaded on only one factor, allowing very similarly worded items to correlate. Both the model’s chi-square and fit-indices, which are nonsensitive to sample size (comparative fit index [CFI], Tucker–Lewis index [TLI], standardized root mean square residual [SRMR], and root mean square error of approximation [RMSEA]), were used to evaluate model fit (Kline, 2016). The following fit index cutoff values are indicative of good model fit: CFI >.95, TLI >.95, and SRMR and RMSEA <.07 (Hu & Bentler, 1999; Kline, 2016; Steiger, 2007). Whereas a nonsignificant chi-square indicates exact model fit, a ratio between the chi-square statistic and the degrees of freedom (df) that is lower than 2.5 indicates a close fit to the data (Hu & Bentler, 1999). Subsequently, internal consistency reliabilities were assessed by means of ordinal Cronbach’s alpha because of the ordinal 3-point response format (Gadermann et al., 2012), and correlations were computed between all SECCI scales. Finally, convergent validity was examined by computing correlations between the SECCI scales and Academic Effort and Academic Competence.

Results

Internal Structure and Reliability of the SECCI

EFA (see Table 1) yielded a four-factor structure, showing a satisfactory fit to the data (SRMR = .030; RMSEA = .057) explaining 49% of the variance, with four eigenvalues greater than 1: Factor 1, designated as Teacher Support, with an eigenvalue of 3.60 (16% explained variance); Factor 2, designated as Negative Student Interactions, with an eigenvalue of 3.28 (15% explained variance); Factor 3, designated as Positive Student Affiliation, with an eigenvalue of 2.11 (10% explained variance); and Factor 4, designated as Unstructured Classroom Environment, with an eigenvalue of 1.76 (8% explained variance). Factor loadings ranged between .45 and .80, whereas all cross loadings were below .20, with only five loadings between .10 and .17, and the remaining 51 (cross) loadings between 0 and .10. The four factors showed small to strong (significant) associations, ranging from r = −.18 (between Negative Student Interactions and Positive Student Affiliations) to r = .63 (between Negative Student Interactions and Unstructured Class Environment). Notably, the factor Positive Student Affiliations was moderately associated with the factor Teacher Support, with r = .44.

Analysis of the bifactor model (see Appendix A and Table 2) did not support a single general factor, without the specific factors, explaining only 17% of the variance, with insufficient model fit (SRMR = .180; RMSEA = .151). There were five very small factor loadings between .13 and .20 (not displayed in Appendix A), six small factor loadings between .20 and .27, and 11 factor loadings beyond the conventional cutoff of .30. A brief version of the SECCI, based on general factor scores greater than .30, would primarily represent negative student interactions and unstructured classroom environment, which seems an underrepresentation of classroom climate from a theoretical point of view. In line with the larger general factor loadings for the negative classroom climate dimensions, the specific factors Negative Student Interactions and Unstructured Classroom Environment only marginally account for unique variance above and beyond the variance accounted for by a general factor, although the factor loadings of these specific factors remain satisfactory. Notably, the eigenvalues drop considerably to 1.10 (5% explained variance) and 0.80 (4% explained variance) for factors Negative Student Interactions and Unstructured Classroom Environment, respectively. Nevertheless, as a general factor for classroom climate does not fit the data well, and the four-factor model provides a good fit to the data, the four-factor model is supported in the EFAs.

Table 2.

Bifactor Exploratory Factor Analysis of the SECCI: Standardized Loadings, Communalities (h2), and Uniqueness (u2).

Item		G	F1	F2	F3	F4	h2	u2
Item		G	Teacher Support	Negative Student Interactions	Positive Student Affiliation	Unstructured Classroom Environment	h2	u2
11	We are bullying each other in the classroom	.66^a		.47			.65	.35
12	We are talking mean about each other in the classroom	.65		.47			.64	.36
9	There is arguing in the classroom	.63		.40			.57	.43
10	We call each other names in the classroom	.60		.42			.54	.46
13	We are laughing at each other behind our back in the classroom	.60		.38			.51	.49
21	I cannot pay attention in the classroom	.52				.46	.48	.52
22	It is messy in the classroom	.51				.50	.51	.49
19	Students make a lot of noise in the classroom	.48	.15			.40	.44	.56
8	Sometimes there will be beating and kicking in the classroom	.45		.33			.31	.69
20	The class is never quiet	.42				.40	.35	.65
2	The teachers are listening to us	.36	−.67	.10			.57	.43
14	I like my classmates	.27			−.76		.65	.35
6	The teachers are having time for us	.26	−.73				.58	.42
4	The teachers are helping me when I’m having difficulties with the lessons	.25	−.70				.56	.44
3	The teachers help us with problems	.25	−.75				.63	.37
1	The teacher is kind	.23	−.59				.44	.56
17	My classmates are nice	.23			−.57		.39	.61
7	The teachers often say “nice job”	.20	−.56				.39	.61
16	In the classroom they like me	.18			−.60		.39	.61
18	I like to work together with my classmates	.16			−.42		.23	.77
5	The teachers tell me when I’m doing fine	.15	−.63				.46	.54
15	I have friends in my class	.13			−.63		.44	.56
	Eigenvalues	3.70	3.10	1.10	1.90	0.80
	Explained variance (%)	17	14	5	9	4

Note. General factor RMSEA = .151. SECCI = Special Education Classroom Climate Inventory; RMSEA = root mean square error of approximation.

Factor loadings less than .10 are not displayed.

Appendix B shows results of the hierarchical model, including a general factor accounting for the variance among the four empirically derived factors in EFA. However, this hierarchical model does not seem tenable, because the factor loadings of the general factor on the positive classroom dimensions are too small, with −.32 for Teacher Support and −.28 for Positive Student Affiliation. Possibly, a hierarchical model with two higher order factors, representing the positive and negative dimensions of classroom climate, does better account for the correlations between the four specific factors. We therefore conducted another EFA, which showed that these two, second-order factors provided a better fit to the data, explaining 47% of the variance. Now, factor loadings were satisfactory: with Unstructured Classroom Environment and Negative Student Interactions loading .75 and .70 on negative classroom climate, respectively, and Teacher Support and Positive Student Affiliation loading .76 and .48 on positive classroom environment, respectively.

We subsequently tested the internal structure of the SECCI in CFA, based on a sufficient subject to free parameters and variables ratio approaching 5:1 and 15:1, respectively, for all CFA analyses, and a sufficiently large sample size to conduct CFA irrespective of these ratios (MacCallum, Widaman, Zhang, & Hong, 1999). Results showed a reasonable fit to the data (see Table 3, Model 1): χ² = 335.840 (df = 203), p < .001 (ratio 1.66, which is lower than 2.5); CFI = .943, TLI = .935, SRMR = .051, and RMSEA = .045. Allowing measurement errors to correlate among items showing similar wording (Items 3 and 4, “helping”) or content (Items 5 and 7, “reward”) further significantly improved model fit, which was eventually considered to be good (see Table 3, model 2): χ² = 286.817 (df = 201), p < .001 (ratio 1.43, which is lower than 2.5); CFI = .963 (Δ = .020), TLI = .958, SRMR = .049, and RMSEA = .036. Fitting a model to the data with the two higher order factors derived from the EFA (i.e., positive and negative classroom climate) did not yield a better fit to the data (see Table 3, Model 3), with a similar RMSEA and CFI as in the previous model, but showed a marginally lower Akaike information criterion (AIC), which suggests that the final model is equally valid or may even be preferred (Kline, 2016).

Table 3.

Fit Statistics CFA Models.

	χ²	df	p	χ²/df	RMSEA	TLI	CFI	SRMR	AIC
Model 1	335.840	203	.000	1.66	.045	.935	.943	.051	13760.8
Model 2	286.817	201	.000	1.43	.036	.958	.963	.049	13705.0
Model 3	286.927	202	.000	1.42	.036	.958	.963	.050	13703.2

Note. CFA = confirmatory factor analysis; RMSEA = root mean square error of approximation; TLI = Tucker–Lewis index; CFI = comparative fit index; SRMR = standardized root mean square residual; AIC = Akaike information criterion.

Table 4 presents the factor loadings of all items, ranging from .553 to .752 for Teacher Support, .568 to .811 for Negative Student Interactions, .480 to .815 for Positive Student Affiliation, and .592 to .689 for Unstructured Classroom Environment.

Table 4.

SECCI Standardized Estimates (Loadings), Derived From CFA.

Item No.	Scale/item	Standardized estimates
	Teacher Support
1	The teacher is kind	.655
2	The teachers are listening to us	.752
3	The teachers help us with problems	.774
4	The teachers are helping me when I’m having difficulties with the lessons	.710
5	The teachers tell me when I’m doing fine	.639
6	The teachers are having time for us	.755
7	The teachers often say “nice job”	.560
	Negative Student Interactions
8	Sometimes there will be beating and kicking in the classroom	.586
9	There is arguing in the classroom	.777
10	We call each other names in the classroom	.735
11	We are bullying each other in the classroom	.811
12	We are talking mean about each other in the classroom	.806
13	We are laughing at each other behind our back in the classroom	.725
	Positive Student Affiliation
14	I like my classmates	.815
15	I have friends in my class	.588
16	In the classroom they like me	.673
17	My classmates are nice	.761
18	I like to work together with my classmates	.480
	Unstructured Classroom Environment
19	Students make a lot of noise in the classroom	.623
20	The class is never quiet	.592
21	I cannot pay attention in the classroom	.689
22	It is messy in the classroom	.672

Note. SECCI = Special Education Classroom Climate Inventory; CFA = confirmatory factor analysis.

Ordinal Cronbach’s alpha was good for all scales: Teacher Support, α = .93; Negative Student Interactions, α = .90; Positive Student Affiliation, α = .85; Unstructured Classroom Environment, α = .82.

Table 5 presents the means and standard deviations of the SECCI scales and the two scales for academic self-concept, and all correlations among these scales. Correlations among the SECCI scales were all significant and in the expected direction, ranging r = −.15 (positive student affiliation and unstructured classroom environment) and r = .53 (negative student interactions and unstructured classroom environment).

Table 5.

Means, Standard Deviations and Correlations of all Classroom Climate and Academic Self-Concept (N = 325).

	M	SD	V1	V2	V3	V4	V5	V6
V1. Teacher Support	3.58	.51	1	−.25***	.37***	−.15**	.28***	.26***
V2. Negative Student Interactions	2.78	.68		1	−.19***	.53***	−.23***	−.20***
V3. Positive Student Affiliation	3.49	.55			1	−.15**	.10*	.12*
V4. Unstructured Classroom Environment	2.93	.66				1	−.21***	−.16**
V5. Academic Effort	2.98	.91					1	.67***
V6. Academic Competence	3.30	.88						1

p < .05. **p < .01. ***p < .001 (one-tailed significance).

Convergent Validity: Associations With Academic Effort and Academic Competence

Convergent validity is demonstrated when the SECCI scales are significantly associated with Academic Effort and Academic Competence. We found Teacher Support to have a positive and significant association with Academic Effort and Academic Competence: r = .28 and r = .26 (p < .001), respectively. We also found Positive Student Affiliation to have a positive and significant association with Academic Effort and Academic Competence: r = .10 and r = .12 (p < .05), respectively. Negative Student Interactions showed a negative and significant association with Academic Effort and Academic Competence: r = −.23 and r = −.20 (p < .01), respectively. Finally, Unstructured Classroom Environment showed a negative and significant association with Academic Effort and Academic Competence: r = −.208 and r = −.159 (p < .05), respectively. Given that all correlations were small, these results provide only marginal support for convergent validity of the SECCI.

Discussion

This study was performed to examine the internal structure, convergent validity, and reliability of the SECCI in a group of adolescents and young adults in schools for special education in secure residential settings and youth prisons. Both exploratory and confirmatory factor analyses yielded a four-factor solution, with Teacher Support, Positive Student Affiliation, Negative Student Interactions, and Unstructured Classroom Environment as reliable dimensions. Moreover, significant but small associations between the SECCI scales and academic self-concept provided preliminary but weak support for the convergent validity of the SECCI. We therefore conclude that more research is necessary to obtain robust evidence to support the validity of the SECCI, with the prospect of using the SECCI to validly and reliably assess classroom climate in schools for special education in (secure) residential youth care settings and youth prisons.

The SECCI can be used for several purposes. First, information on classroom climate can be used by teachers and school psychologists as input for the discussion with students about their experiences with other students and their teacher to improve classroom climate (see Broh, 2002; Hattie & Timperly, 2007). Second, the SECCI makes it possible to measure progress in classroom climate, and provides appropriate guidance on which dimensions to target by teachers and school psychologists (Pavletic, 2011; Ramelow et al., 2015). For example, if school psychologists establish that students experience negative student interactions, they may provide teachers with advice on how to stop negative student interactions, and create a more positive classroom climate. Notably, there is some empirical evidence showing that a positive classroom climate can contribute to positive social and academic development of students in regular education (Anderson et al., 2004; Wissink et al., 2014). The SECCI can be used to examine whether the positive results from research on classroom climate in regular education can be generalized to classroom climate in residential special education schools.

The correlations between the SECCI scales and academic self-concept were significant but low, providing only limited support for convergent validity of the SECCI. It is plausible to suggest that classroom climate may only partially contribute to academic self-concept of juveniles attending schools for special education in (secure) residential institutions, because academic self-concept, to be distinguished from self-efficacy, is considered to be affected by several factors, in particular social comparative information and perceived evaluations and appraisals from significant others (Bong & Skaalvik, 2003). Notably, students attending special education in secure residential care often have a history of adverse and inconsistent care, having experienced extremely negative appraisals from significant others during extended periods of time (Souverein, Van der Helm, & Stams, 2013). It is therefore possible that their academic self-concept has become more trait-like than state-like, and therefore less susceptible for rehabilitative environmental influences, such as a positive classroom climate. Apart from this, social comparative information in the context of student relationships, in particular competition (Van der Helm et al., 2011), may affect academic self-concept too. Although many instruments assessing classroom climate assess competition among students (Altaf, 2015), this dimension is not assessed by the SECCI, which may explain the relatively small associations between the SECCI and academic self-concept. In further development of the SECCI, a scale assessing competition should be added, especially because children in secure institutions have difficulties in coping with competition (Van der Helm et al., 2011).

The main limitations of this study relate to characteristics of the sample. Although there are female students in schools of juvenile prisons, male students were overrepresented in these schools. Furthermore, there was no other validated classroom climate instrument available in the Netherlands for this population to compare with the SECCI, and it was not possible to conduct behavioral observations of the classroom climate to establish convergent validity. Moreover, convergent validity was established by only assessing academic self-concept, showing low correlations, and without taking comparative discriminant evidence into account. Future validation studies of the SECCI should also examine associations with intrinsic and extrinsic learning motivation as well as internalizing (anxiety and depression) and externalizing (aggression and norm transgressive behavior) problems. Finally, no predictive validity and test–retest reliability could be established because the present study was cross-sectional.

The present study found preliminary evidence for validity and reliability of the SECCI in a group of adolescents in secure residential youth care and youth prisons attending special education classes. This study needs replication in future validation studies of the SECCI, which also assess classroom climate by means of observation and provide a more comprehensive assessment of validity, in particular establishing discriminant validity. Without discriminant evidence, it is difficult to correctly interpret the value of obtained convergent validity. We assume that repeated measurement of classroom climate, discussing the outcomes of the SECCI with teachers and students by school psychologists, and strengthening competencies and self-efficacy of teachers, informed by a valid and reliable assessment of teacher support, positive student affiliation, negative student interactions, and (un)structured classroom environment, can improve classroom climate and improve the educational and psychosocial outcomes for children at risk.

Footnotes

Appendix A

Appendix B

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Altaf

(2015). Exploring classroom environment through perception. An overview of various inventories. International Journal of Novel Research in Education and Learning, 2, 23-32.

Anderson

Hamilton

R. J.

Hattie

(2004). Classroom climate and motivated behaviour in secondary schools. Learning Environments Research, 7, 211-225.

Beld

M. H. M.

Van der Voort

Van der Helm

G. H. P.

Kuiper

C. H. Z.

Stams

G. J. J. M

. (2017). School attitude, general self-worth, and social identity: The validity of the About Me Questionnaire for students with severe behavioural problems in school. Manuscript submitted for publication.

Bong

Skaalvik

E. M.

(2003). Academic self-concept and self-efficacy: How different are they really? Educational Psychology Review, 15, 1-40.

Breeman

L. D.

Wubbels

Van Lier

P. A. C.

Verhulst

F. C.

Van der Ende

Maras

… Tick

N. T.

(2014). Teacher characteristics, social classroom relationships, and children’s social, emotional, and behavioral classroom adjustment in special education. Journal of School Psychology, 53, 87-103.

Broh

B. A.

(2002). Linking extracurricular programming to academic achievement: Who benefits and why? Sociology of Education, 75, 69-91.

Carman

G. O.

Dorta

Kon

Martin

Zarrilli

M. A.

(2004). Special education in residential treatment. Child and Adolescent Psychiatric Clinics of North America, 13, 381-394.

Clifford

Menon

Gangi

Condon

Hornung

(2012). Measuring school climate for gaugling principal performance: A review of the validity and reliability of publicly accessible measures. Washington, DC: American Institute for Research.

Colins

Vermeiren

Vreugdenhil

van den Brink

Doreleijers

Broekaert

(2010). Psychiatric disorders in detained male adolescents: A systematic literature review. Canadian Journal of Psychiatry, 55, 255-263.

10.

Gadermann

A. M.

Guhn

Zumbo

B. D.

(2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research & Evaluation, 17(3), 1-13.

11.

Goodwin

L. D.

Leech

N. L.

(2003). Assessment in action: The meaning of validity in the new standards for educational and psychological testing: Implications for measurement courses. Measurement and Evaluation in Counseling and Development, 36, 181-191.

12.

Guay

Marsh

H. W.

Boivin

(2003). Academic self-concept and academic achievement: Developmental perspectives on their causal ordering. Journal of Educational Psychology, 95, 124-136.

13.

Hattie

Timperly

(2007). The power of feedback. Review of Educational Research, 7, 81-112.

14.

Hocutt

A. M.

(1996). Effectiveness of special education: Is placement the critical factor? The Future of Children, 6, 77-102.

15.

L. T.

Bentler

P. M.

(1999). Cut-off criteria for fit indices in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modelling, 6(1), 1-55.

16.

Jellesma

F. C.

Zee

Koomen

H. M. Y.

(2015). Children’s perceptions of the relationship with the teacher: Associations with appraisals and internalizing problems in middle childhood. Journal of Applied Developmental Psychology, 36, 30-38.

17.

Kline

R. B.

(2016). Principles and practice of structural equation modeling (4th ed.). New York, NY: Guilford Press.

18.

MacAulay

D. J.

(1990). Classroom environment: A literature review. Educational Psychology, 10, 239-253.

19.

MacCallum

R. C.

Widaman

K. F.

Zhang

Hong

(1999). Sample size in factor analysis. Psychological Methods, 4, 84-99.

20.

Maras

Demetre

Moon

Tolmie

(2012). Psychology and antiscocial behaviour in the classroom. Leicester, UK: The British Psychological Society.

21.

McRobbie

C. J.

Fraser

B. J.

(1993). Associations between student outcomes and psychosocial science environment. The Journal of Educational Research, 87, 78-85.

22.

Moos

R. H.

(1979). Evaluating educational environments: Procedures, measures, findings, and policy implications. San Francisco, CA: Jossey-Bass.

23.

Muthén

L. K.

Muthén

B. O.

(1998-2006). Mplus users guide (4th ed.). Los Angeles, CA: Author.

24.

Pavletic

A. C.

(2011). Connecting with frequent adolescent visitors to the school nurse through the use of intentional interviewing. The Journal of School Nursing, 27, 258-268.

25.

Quinn

M. M.

Rutherford

R. B.

Leone

P. L.

Osher

P. E.

Poirier

J. M.

(2005). Youth with disabilities in juvenile corrections: A national survey. Council for Exceptional Children, 71, 339-345.

26.

Ramelow

Currie

Felder-Puig

(2015). The assessment of school climate: Review and appraisal of published student-report measures. Journal of Psychoeducational Assessment, 33, 731-743.

27.

Revelle

(2016). How to: Use the psych package for factor analysis and data reduction. Evanston, IL: Department of Psychology, Northwestern University.

28.

Rios

Wells

(2014). Validity evidence based on internal structure. Psicothema, 26, 108-116.

29.

Roorda

D. L.

Koomen

H. M. Y.

Spilt

J. L.

Oort

F. J.

(2011). The influence of affective teacher-student relationships on students’ school engagement and achievement: A meta-analytic approach. Review of Educational Research, 81, 493-529.

30.

Schmid

Leiman

J. M.

(1957). The development of hierarchical factor solutions. Psychometrika, 22, 83-90.

31.

Souverein

Van der Helm

G. H. P.

Stams

G. J. J. M

. (2013). Nothing works in secure residential youth care? Children and Youth Services Review, 35, 1941-1945.

32.

Steffgen

Recchia

Viechtbauer

(2013). The link between school climate and violence in school: A meta-analytic review. Aggression and Violent Behavior, 18, 300-209.

33.

Steiger

J. H.

(2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42, 893-898.

34.

Stockard

Mayberry

(1992). Effective educational environments. Newbury Park, CA: Corwin.

35.

Thapa

Cohen

Guffey

Higgins-D’Alessandro

(2013). A review of school climate research. Review of Educational Research, 83, 357-385.

36.

Trickett

E. J.

Leone

P. E.

Fink

C. M.

Braaten

S. L.

(1993). The perceived environment of special education classrooms for adolescents: A revision of the classroom environment scale. Exceptional Children, 59, 411-420.

37.

Van Dam

Nijhof

Scholte

Veerman

J. W

. (2010). Evaluatie nieuw zorgaanbod gesloten jeugdzorg voor jongeren met ernstige gedragsproblemen [Evaluation of a new care supply in secure residential youth care for juveniles with severe behaviour problems]. Nijmegen, The Netherlands: Praktikon.

38.

Van der Helm

G. H. P.

Matthys

Moonen

Giesen

van de Heide

Stams

G. J. J. M

. (2011). Measuring inappropriate responses of adolescents to problematic social situations in secure institutional and correctional youth care: A validation study of the TOPS-A. Journal of Interpersonal Violence, 28, 1579-1595.

39.

Voight

Hanson

(2012). Summary of existing school climate instruments for middle school. San Francisco, CA: REL West at WestEd.

40.

Weber

N. M.

Somers

C. L.

Day

Baroni

B. A.

(2016). Predictors and outcomes of school attachment and school involvement in a sample of girls in residential treatment. Residential Treatment for Children & Youth, 33, 155-174.

41.

Wissink

I. B.

Dekovic

Stams

G. J. J. M.

Asscher

J. J.

Rutten

E. A.

Zijlstra

B. J. H.

(2014). Moral orientation and relationships in school and adolescent pro- and antisocial behaviors: A multi-level study. Journal of School Nursing, 30, 216-225.

42.

Withall

(1949). The development of a technique for the measurement of social-emotional climate in classrooms. The Journal of Experimental Education, 17, 347-361.

43.

Wolff

H. G.

Preising

(2005). Exploring item and higher order factor structure with the Schmid-Leiman solution: Syntax codes for SPSS and SAS. Behavioural Research Methods, 37, 48-58